Prefabricated part production resource intelligent scheduling management method based on reinforcement learning

Abstract

The invention relates to the technical field of reinforcement learning intelligent scheduling, in particular to an intelligent scheduling management method for prefabricated part production resources based on reinforcement learning. The specific implementation process comprises the steps of collecting order demands, material distribution and production pedestals in real time, and mapping the order demands, the material distribution and the production pedestals into production mold load tensors; when a scheduling request is triggered, inputting the production modulo tensor into a resource arrangement network, and performing search and reasoning by using a scheduling strategy based on a multi-head attention mechanism to generate a resource scheduling matching instruction; calculating a state difference tensor, and outputting an efficiency reward signal in combination with a delivery constraint and a cost constraint; and packaging the production modulo tensor, the resource scheduling matching instruction and the reward signal into semantic interaction experience, storing the semantic interaction experience into a scheduling experience playback pool for gradient modulation, and iteratively optimizing a scheduling strategy. According to the method, learning can be carried out from a large amount of historical data by utilizing reinforcement learning, iterative optimization of the scheduling strategy is realized, the calculation time consumption of scheduling instruction generation is reduced, and the flexibility of production scheduling is improved.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning intelligent scheduling technology, specifically to a method for intelligent scheduling and management of prefabricated component production resources based on reinforcement learning. Background Technology

[0002] In the field of precast component production, diverse production tasks are typically handled, including special components using pre-mixed shrinkage-compensating concrete or modular products based on dry-mix concrete processes. These different processes place drastically different demands on production cycle time, curing conditions, and mold turnover. Existing technologies mainly employ computational models such as mathematical programming or metaheuristic algorithms to perform offline calculations throughout the entire production planning cycle to seek the globally optimal solution. Pre-defined scheduling rules also assist in decision-making, improving information transparency and execution efficiency in the production process.

[0003] However, existing technologies have inherent limitations in practical applications. Whether it's offline optimization based on precise computational models or scheduling based on fixed rules, their effectiveness highly depends on a stable and predictable production environment. Once dynamic disturbances such as order changes, equipment failures, or material delays occur in reality, pre-defined static scheduling schemes quickly become ineffective, and the computational lag of re-solving the global problem makes it impossible to meet the demands of real-time response. Furthermore, the inability to effectively utilize the large amounts of historical data generated during production to learn from past successes and failures hinders the self-evolution and iteration of existing scheduling methods in complex and coupled real-world scenarios.

[0004] To address this, a method for intelligent scheduling and management of prefabricated component production resources based on reinforcement learning is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for intelligent scheduling and management of prefabricated component production resources based on reinforcement learning, so as to realize intelligent scheduling of prefabricated components.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A reinforcement learning-based intelligent scheduling and management method for prefabricated component production resources includes: Real-time data collection of premixed shrinkage-compensating concrete resources, including order requirements, material distribution, and production platform data, is mapped to a production load tensor that characterizes the current production status of precast components. When a scheduling request is triggered, the production module load tensor is input as the environment state to the resource orchestration network; the resource orchestration network uses a scheduling strategy based on a multi-head attention mechanism to search and reason within the action space that satisfies the hard constraints of the process, and generates resource scheduling matching instructions that can accumulate rewards. After executing the resource scheduling matching instruction to update the production status of prefabricated components, the state difference tensor is calculated and the constraint value projection mechanism is initiated; combined with the delivery constraints and cost constraints extracted from the resource data, the difference between the production status before and after is calculated in a matrix, and an efficiency reward signal including capacity utilization rate and mold turnover rate is output. The production module load tensor, resource scheduling matching instructions, and performance reward signals are encapsulated into semantic interaction experience and stored in the scheduling experience replay pool; gradient modulation is performed based on the scheduling experience replay pool to iteratively optimize the scheduling strategy of the resource orchestration network.

[0007] Preferably, the specific implementation process of real-time acquisition of premixed shrinkage-compensating concrete resource data, including order requirements, material distribution, and production platform data, and mapping it to a production load tensor characterizing the current precast component production status includes: The system acquires order demand data, including delivery sequence and process path, material distribution data, representing inventory status and logistics timeliness, and production element data, reflecting space occupancy and turnover cycle, from the enterprise resource planning system, production execution system, and IoT sensor terminals. The system performs data cleaning and time-series alignment on the production element data and maps it to a coordinate system with time, space, and resource status as the reference axes, in accordance with preset dimension mapping rules, to form a production module tensor.

[0008] Preferably, when a scheduling request is triggered, the specific implementation process of inputting the production module load tensor as an environmental state into the resource orchestration network includes: Upon receiving a scheduling request generated by dynamic changes in production factors, the production module tensor is sent to the resource orchestration network to mine and capture the dependencies and interactions between order requirements, material distribution, and production platforms.

[0009] Preferably, the resource orchestration network utilizes a multi-head attention-based scheduling strategy to search and reason within the action space that satisfies process hard constraints, generating resource scheduling matching instructions capable of accumulating rewards. The specific implementation process includes: Based on the hard process constraints including component maintenance cycles, mold matching relationships, and process logic associations, a constraint mask is generated to shield invalid scheduling actions. Within the set of valid actions defined by the constraint mask, the resource orchestration network generates scheduling decisions one by one through autoregression. Its internal multi-head attention mechanism dynamically evaluates the correlation weight between the current production state and candidate scheduling tasks in each decision step, and calculates the probability distribution of all valid actions. The scheduling scheme with the highest action value is sampled from the probability distribution and selected, and decoded into a resource scheduling matching instruction containing task identifier, resource code, and execution sequence.

[0010] Preferably, after executing the resource scheduling matching instruction to update the production status of the prefabricated components, the specific implementation process of calculating the state difference tensor and initiating the constraint value projection mechanism includes: The production status of prefabricated components is monitored in real time. When the production status is updated after the resource scheduling matching instruction is completed, the production module load tensor before and after the scheduling execution is automatically captured, the state difference tensor is calculated, and the constraint value projection mechanism is invoked.

[0011] Preferably, the specific implementation process of combining the delivery constraints and cost constraints extracted from the resource data to perform matrix calculations on the differences in production status before and after, and outputting a performance reward signal including capacity utilization and mold turnover rate, includes: Load delivery constraints and cost constraints associated with the current production batch from the production knowledge base, construct a constraint matrix coupled with the delivery constraints and cost constraints, and the weight parameters of the constraint matrix represent the scheduling behavior evaluation criteria under different constraint conditions; perform matrix sliding calculation on the state difference tensor using the constraint matrix to generate an intermediate feature map representing scheduling efficiency; perform a global averaging and aggregation operation on the intermediate feature map to reduce the dimensionality of high-dimensional feature information and map it onto performance index vectors representing capacity utilization and mold turnover rate respectively; perform weighted fusion and scalarization processing on the performance index vectors to generate an efficiency reward signal that can guide the scheduling strategy to optimize in the direction of satisfying the constraint objectives.

[0012] Preferably, the specific implementation process of encapsulating the production module load tensor, resource scheduling matching instructions, and performance reward signals into semantic interaction experience and storing them in the scheduling experience replay pool includes: The production module tensor, which serves as the pre-scheduling state, the executed resource scheduling matching instruction, the generated performance reward signal, and the updated production module tensor, which serves as the post-scheduling state, are combined into a four-element data set containing complete state transition information. This four-element data set is then encapsulated as semantic interaction experience and added to the scheduling experience replay pool.

[0013] Preferably, the specific implementation process of the scheduling strategy for the resource orchestration network based on gradient modulation using a scheduling experience replay pool includes: Semantic interaction experiences are randomly sampled from the scheduling experience replay pool and input into the resource orchestration network. The action probability distribution and state value assessment under the current scheduling strategy are calculated via forward propagation. A strategy loss function and a value loss function are constructed, and the gradient of the loss function with respect to the resource orchestration network parameters is calculated via backpropagation. The calculated gradient is modulated by identifying and amplifying the gradient contribution generated by the reward value semantic interaction experience, while suppressing the gradient influence of related experiences, thus creating a synergistic enhancement in the update directions among different training samples. The modulated gradient is used to update the network parameters of the resource orchestration network, driving the scheduling strategy to iteratively optimize.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention utilizes reinforcement learning, which can learn from a large amount of historical data to achieve self-evolution and iterative optimization, thereby making real-time and intelligent responses to dynamic disturbances and improving the flexibility and robustness of prefabricated component production scheduling.

[0015] 2. This invention, by introducing a resource orchestration network based on a multi-head attention mechanism, can more accurately capture the complex dependencies between ready-mixed shrinkage-compensated concrete order demands, material distribution, and production platforms. This allows scheduling decisions to comprehensively consider multiple constraints such as capacity utilization and mold turnover rate, effectively optimizing resource allocation and improving overall production efficiency by generating scheduling instructions that maximize cumulative returns.

[0016] 3. This invention designs an experience replay and gradient modulation mechanism. By storing successful and unsuccessful scheduling experiences in the replay pool, the gradient contribution of high reward value experiences can be amplified and the influence of low correlation experiences can be suppressed during the iterative optimization process. This accelerates the convergence process to the optimal scheduling strategy and realizes the continuous self-improvement of scheduling capabilities. Attached Figure Description

[0017] Figure 1 This is a flowchart of the intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning proposed in this invention. Figure 2 This is a schematic diagram of the resource orchestration network proposed in this invention; Figure 3 This is a schematic diagram of the gradient modulation mechanism proposed in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It must be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to constitute any limitation on the scope of protection of this invention. Therefore, all equivalent changes or modifications conceived by those skilled in the art based on the content disclosed in this invention without inventive effort should fall within the scope of protection claimed by this invention.

[0019] Reference Figures 1 to 3 This invention provides an intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning. The technical solution is as follows: Example

[0020] Reference Figure 1 This embodiment proposes an intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning, including: Real-time data collection of premixed shrinkage-compensating concrete resources, including order requirements, material distribution, and production platform data, is mapped to a production load tensor that characterizes the current production status of precast components. When a scheduling request is triggered, the production module load tensor is input as the environment state to the resource orchestration network; the resource orchestration network uses a scheduling strategy based on a multi-head attention mechanism to search and reason within the action space that satisfies the hard constraints of the process, and generates resource scheduling matching instructions that can accumulate rewards. After executing the resource scheduling matching instruction to update the production status of prefabricated components, the state difference tensor is calculated and the constraint value projection mechanism is initiated; combined with the delivery constraints and cost constraints extracted from the resource data, the difference between the production status before and after is calculated in a matrix, and an efficiency reward signal including capacity utilization rate and mold turnover rate is output. The production module load tensor, resource scheduling matching instructions, and performance reward signals are encapsulated into semantic interaction experience and stored in the scheduling experience replay pool; gradient modulation is performed based on the scheduling experience replay pool to iteratively optimize the scheduling strategy of the resource orchestration network.

[0021] Furthermore, the specific implementation process of real-time collection of premixed shrinkage-compensating concrete resource data, including order demand, material distribution, and production platform data, and mapping it to a production modulus tensor characterizing the current precast component production status, includes: The system acquires order demand data, including delivery sequence and process path, material distribution data, representing inventory status and logistics timeliness, and production element data, reflecting space occupancy and turnover cycle, from the enterprise resource planning system, production execution system, and IoT sensor terminals. The system performs data cleaning and time-series alignment on the production element data and maps it to a coordinate system with time, space, and resource status as the reference axes, in accordance with preset dimension mapping rules, to form a production module tensor.

[0022] Specifically, it connects to the factory's Enterprise Resource Planning (ERP) system to obtain structured order demand data. For example, the instruction with order number "ORD20251121" specifies in detail the need to produce 50 "Exterior Wall Cladding PC-001" and 30 "Composite Beam PCB-002" units, and specifies the process path that each component must follow, such as: "Rebar Binding - Mold Assembly - Concrete Pouring - Steam Curing - Demolding - Quality Inspection". At the same time, it interacts with the Manufacturing Execution System (MES) to obtain material distribution data representing inventory status. For example, real-time query shows that the current inventory of P.O42.5 grade shrinkage-compensating cement in the raw material warehouse is 150 tons, and the aggregate inventory is sufficient. However, only 10 sets of molds for the specific model (M-001) used to produce "Exterior Wall Cladding PC-001" are currently available, and the rest are in circulation. In addition, Internet of Things (IoT) sensing terminals deployed on the production site, such as RFID readers installed on production benches, temperature and humidity sensors in curing kilns, and GPS positioning modules on material transport vehicles, capture production bench data that reflects space occupancy and turnover cycle. For example, real-time data shows that production bench No. 3 is occupied by component "PCB-001-SN034", and the steam curing process of this component is 80% complete. It is expected to take another 2.5 hours before it can enter the next process, while benches No. 4 and No. 5 are idle and ready to go.

[0023] After acquiring a large amount of multi-dimensional production factor data from ERP, MES, and IoT, data cleaning and time-series alignment were immediately performed. Data cleaning aims to eliminate outliers and fill in missing data, such as correcting momentary erroneous temperature readings caused by sensor signal interference, or supplementing temporarily lost inventory data through interpolation algorithms. Time-series alignment involves uniformly calibrating data streams from different systems, where timestamps may have slight deviations, onto a standard, discrete time axis, for example, using a 5-minute time step, to ensure the synchronicity and consistency of all status information in the time dimension. Subsequently, following a preset dimensional mapping rule, this pre-processed production factor data is uniformly mapped to a multi-dimensional coordinate system with time, space, and resource status as the reference axes. During the mapping process, discrete symbolic data such as order numbers, product types, and mold IDs are processed using entity embedding technology. Specifically, an N×K embedding matrix is ​​constructed (where N is the size of the discrete symbol vocabulary and K is the embedding vector dimension, set to 64 in this embodiment) to map discrete IDs into dense real number vectors. For continuous numerical data such as process progress and inventory, normalization is performed to the (0, 1) interval, and the embedding vector and normalized values ​​are concatenated along the feature channel dimension. In this coordinate system, the time axis of the production module tensor is based on the previously set 5-minute scale, the spatial axis discretizes each physical resource location in the factory, such as "Platform No. 1", "Platform No. 2", "Curing Kiln Area A", etc., and the resource status axis is a multi-dimensional vector, with each dimension corresponding to a status attribute, such as order number, product type, process progress, mold occupancy, material availability, etc. Through this mapping, all discrete data points are organically integrated, ultimately forming a production module tensor that can dynamically and panoramically reflect the complex production status of the entire workshop at any given moment.

[0024] This embodiment improves the flexibility and robustness of production scheduling by constructing a production module tensor, enabling it to respond to production disturbances in real time and intelligently. By fusing multi-source heterogeneous data into a structured tensor, it can more accurately capture the complex dependencies between order requirements, material distribution, and production stations.

[0025] Furthermore, when a scheduling request is triggered, the specific implementation process of inputting the production module load tensor as an environmental state into the resource orchestration network includes: Upon receiving a scheduling request generated by dynamic changes in production factors, the production module tensor is sent to the resource orchestration network to mine and capture the dependencies and interactions between order requirements, material distribution, and production platforms.

[0026] Specifically, a new scheduling request is triggered when the production environment undergoes dynamic changes. For example, a high-priority urgent order may be inserted, or a critical production piece of equipment may experience an unexpected downtime. In this case, the previously constructed production load tensor is input into the resource orchestration network as a snapshot of the environmental state for in-depth situational analysis and correlation mining. In a preferred embodiment, the Enterprise Resource Planning (ERP) system receives an urgent order with order number "ORD20251122-URGENT," requesting the production of 20 units of a specific specification "load-bearing beam PC-003" within 48 hours. This dynamic change in production elements, i.e., the entry of a new order, immediately generates a scheduling request. The current production mold load tensor not only encodes the demand information for this urgent order (product type PC-003, quantity 20 pieces, delivery deadline 48 hours), but also includes all relevant real-time status data in the workshop at that time: Regarding production benches, bench 3 is occupied by component "PCB-001-SN034", and its curing process will take another 2.5 hours to complete; while benches 4 and 5 are idle and can be used immediately. In terms of material distribution, the inventory of M-003 type special molds required for producing PC-003 shows that 5 sets are available, and each set of molds completes a full cycle of component production in 8 hours.

[0027] However, the remaining inventory of compensating shrinkage admixtures was only 500 kg. Based on the material quota, this inventory could only meet the production needs of 10 PC-003 units, while the subsequent 1000 kg of supplementary materials was expected to arrive at the factory in 6 hours. Upon receiving this high-dimensional data stream containing rich spatiotemporal information, the resource orchestration network did not perform isolated data interpretation. Instead, it utilized its complex internal network structure (e.g., multi-head attention mechanism) to deeply mine the data from different dimensions of the tensor, capturing the inherent dependencies and interactions between order demand, material distribution, and production benches. Specifically, the network could identify that although there were idle production benches (No. 4 and No. 5) and a certain number of available molds (5 sets), allowing for the immediate initiation of some production tasks, the completion of the entire order (20 units) was strongly constrained by the critical resource of the admixture inventory (only enough for 10 units). At the same time, the network can also perform correlation analysis between the expected arrival time of materials and the turnover cycle of molds (8 hours), predicting that even if the production of the first 10 pieces starts immediately, the subsequent materials will be in place when the first batch of components is completed and the mold is released, thus achieving seamless production.

[0028] This embodiment feeds the production module load tensor into a resource orchestration network for global correlation mining, enabling the scheduling system to perceive the overall operational status of the production system from a holistic, dynamic evolution perspective. It can identify the true bottlenecks of current production tasks and predict future trends in resource availability. This provides a basis for developing forward-looking scheduling strategies, improving the intelligence level of production scheduling and its responsiveness to dynamic disturbances.

[0029] Furthermore, the resource orchestration network utilizes a multi-head attention-based scheduling strategy to search and reason within the action space that satisfies the hard constraints of the process, generating resource scheduling matching instructions that can accumulate rewards. The specific implementation process includes: Based on the hard process constraints including component maintenance cycles, mold matching relationships, and process logic associations, a constraint mask is generated to shield invalid scheduling actions. Within the set of valid actions defined by the constraint mask, the resource orchestration network generates scheduling decisions one by one through autoregression. Its internal multi-head attention mechanism dynamically evaluates the correlation weight between the current production state and candidate scheduling tasks in each decision step, and calculates the probability distribution of all valid actions. The scheduling scheme with the highest action value is sampled from the probability distribution and selected, and decoded into a resource scheduling matching instruction containing task identifier, resource code, and execution sequence.

[0030] Reference Figure 2Specifically, the resource orchestration network adopts a Transformer-based PointerNetwork architecture. The encoder receives the production module load tensor and extracts global features through a multi-head self-attention mechanism. The decoder uses LSTM units to maintain the decoding state and performs attention calculations between the current decoding state and the encoder output at each step, outputting a Softmax probability distribution for the effective action set to select the pedestal index for the next action to be executed. After receiving the production module load tensor representing the production state after the insertion of an emergency order, the scheduling strategy based on the multi-head attention mechanism within the resource orchestration network begins to execute. For the production task of "load-bearing beam PC-003" with order number "ORD20251122-URGENT", its specific set of process hard constraints is retrieved from the process knowledge base. The set of hard constraints for the process explicitly stipulates: First, the component curing cycle, meaning the component must be continuously cured in a steam curing kiln for no less than 8 hours; second, the mold matching relationship, meaning that the production of PC-003 components must and can only use the dedicated mold of model M-003; third, the logical association of processes, meaning the production process must strictly follow the sequence of "mold assembly - concrete pouring - steam curing - demolding," and any scheduling that skips or reverses the process is invalid. Based on these inviolable rules, a constraint mask is generated in real time. Mathematically, this mask is represented by a Boolean tensor with the same dimension as the action space. It invalidates all scheduling actions that violate the above constraints, such as "assigning PC-003 to a mold other than model M-003" or "performing demolding on platform 4 when its curing time is only 6 hours," thus fundamentally shielding these infeasible solutions during subsequent search and reasoning processes.

[0031] Within the set of valid actions defined by the constraint mask, the resource orchestration network generates scheduling decisions one by one through autoregression. In the first step of the decision sequence, namely deciding the production arrangement of the first PC-003 component, the network's multi-head attention mechanism dynamically evaluates the correlation weights between the current production state (represented by the input production module load tensor) and all candidate scheduling tasks (such as "starting production on pedestal 4 using mold M003-SN01", "starting production on pedestal 5 using mold M003-SN02", etc.) from different dimensions in parallel. For example, one attention head might focus on time urgency, calculating the correlation between the short delivery cycle (48 hours) of urgent orders and the immediate execution capability of each candidate action, assigning higher weights to actions that can start immediately. Another attention head might focus on resource bottlenecks, highlighting the inventory status of only enough special admixtures to produce 10 units, thereby increasing the weights of candidate tasks that can initiate the production of these first 10 components, as this directly relates to whether the task can be started. Another attention head might assess the continuity of resource utilization, predicting that platform 3 will be idle in 2.5 hours and correlating this with the mold's 8-hour turnaround time to evaluate the smoothness of subsequent task transitions. By integrating the assessment results of all attention heads, the network calculates the probability distribution of all effective actions at the current time point. For example, the calculated action value of "starting production on platform 4" is 0.92, while the action value of "starting production on platform 5" is 0.91, and the values ​​of other actions are significantly lower.

[0032] The scheduling strategy samples from this probability distribution and selects the scheduling scheme with the highest action value, namely "start production at platform 4". This is then decoded into a structured, executable resource scheduling matching instruction. This instruction includes the task identifier: "ORD20251122-URGENT-Unit01"; resource codes: "Platform: T04, Mold: M003-SN01, MaterialBatch: MB-20251121-1405"; and detailed execution timing: "StartTime: 2025-11-21-14:05, EstCuringEndTime: 2025-11-21-22:05". This instruction is issued to the production execution system to guide on-site operations. After this decision is made, the system state is updated, and the execution result of this instruction becomes the environmental input for the next round of autoregressive decision-making. The network will continue to make the optimal scheduling arrangement for the second PC-003 component.

[0033] This embodiment reduces the search space for decision-making by generating constraint masks, avoiding the waste of computational power on a large number of invalid solutions, and ensuring the underlying feasibility and efficiency of scheduling decisions. The multi-head attention mechanism can dynamically and multidimensionally weigh the complex interactive effects of all relevant factors, especially when facing complex scenarios such as resource constraints, time pressure, and multi-task parallelism, it can make more accurate and forward-looking judgments.

[0034] Furthermore, after executing the resource scheduling matching instruction to update the production status of the prefabricated components, the specific implementation process of calculating the state difference tensor and initiating the constraint value projection mechanism includes: The production status of prefabricated components is monitored in real time. When the production status is updated after the resource scheduling matching instruction is completed, the production module load tensor before and after the scheduling execution is automatically captured, the state difference tensor is calculated, and the constraint value projection mechanism is invoked.

[0035] Specifically, after the resource scheduling and matching instruction is issued, the calculation of the state difference tensor begins and the constraint value projection mechanism is activated. For example, if the instruction "Start production of the first PC-003 component at 14:05 on November 21, 2025, using mold M003-SN01 on platform 4" is issued, the system actively tracks the physical execution progress of the instruction through deep integration with the Production Execution System (MES) and the IoT platform. When the MES confirms that the mold on platform 4 has been assembled and the concrete has been poured, and officially starts production timing, marking the completion of the scheduling instruction and successful updating of the workshop's physical production state (e.g., at 14:05), the automatic capture and calculation mechanism is instantly triggered. At this time, two actions are automatically executed: First, the last snapshot of the production state before the instruction execution is completely preserved, i.e., the production mold loading tensor at 14:00, referred to as the "previous state tensor"; second, the production mold loading tensor at 14:05 reflecting the latest production state is immediately generated and captured, referred to as the "current state tensor".

[0036] After obtaining two temporally adjacent state tensors, a new, highly sparse data structure—the state difference tensor—is calculated by performing element-wise matrix subtraction (i.e., subtracting the previous state tensor from the current state tensor). When calculating the state difference tensor, for continuous numerical dimensions (such as inventory levels), element-wise subtraction is performed directly; for discrete coded dimensions (such as mold IDs), XOR logic or Hamming distance is used. If the ID codes of the preceding and following states are inconsistent, the corresponding position in the difference tensor is marked as 1 (indicating a state flip); otherwise, it is marked as 0. The resulting difference tensor is a sparse tensor, retaining non-zero values ​​only in dimensions where the state has undergone substantial changes. The vast majority of elements of the state difference tensor are zero, exhibiting non-zero values ​​only in dimensions where the state has changed due to the execution of scheduling instructions. In a preferred embodiment, the state difference tensor specifically manifests as a clear numerical change at the coordinate positions corresponding to the states of pedestal 4 (changing from "idle" to "occupied"), molds M003-SN01 (changing from "available" to "in use"), and the inventory of compensating shrinkage additives (reducing from 500 kg to 450 kg). This state difference tensor precisely isolates all the direct consequences caused by the single scheduling action of "starting production at pedestal 4". After calculating the state difference tensor, the constraint value projection mechanism is immediately invoked. The constraint value projection mechanism predefines a series of constraint vectors strongly correlated with production goals and key performance indicators, such as "timely delivery of urgent orders", "capacity utilization rate", and "mold turnover efficiency". The core function of the constraint value projection mechanism is to project the state difference tensor, which represents the "change in the physical world", onto these constraint vectors, which represent "business value", thereby quantifying the contribution or damage of this scheduling action to achieving the production goal.

[0037] This embodiment achieves real-time quantitative evaluation of the effectiveness of each micro-scheduling decision by calculating the state difference tensor. This immediacy greatly shortens the learning feedback loop, enabling the agent to self-correct and optimize at a very high frequency. By accurately separating the state changes caused by a single action, subsequent rewards or penalties can be clearly attributed to specific decisions, avoiding the erroneous attribution of good results to ineffective actions in complex production processes. The constrained value projection mechanism directly links the underlying physical state changes with the upper-level business objectives, ensuring that the learned scheduling strategy is not only technically efficient but also maximizes production benefits.

[0038] Furthermore, combining the delivery constraints and cost constraints extracted from the resource data, the specific implementation process of performing matrix calculations on the differences in production status before and after, and outputting performance reward signals including capacity utilization and mold turnover rate, includes: Load delivery constraints and cost constraints associated with the current production batch from the production knowledge base, construct a constraint matrix coupled with the delivery constraints and cost constraints, and the weight parameters of the constraint matrix represent the scheduling behavior evaluation criteria under different constraint conditions; perform matrix sliding calculation on the state difference tensor using the constraint matrix to generate an intermediate feature map representing scheduling efficiency; perform a global averaging and aggregation operation on the intermediate feature map to reduce the dimensionality of high-dimensional feature information and map it onto performance index vectors representing capacity utilization and mold turnover rate respectively; perform weighted fusion and scalarization processing on the performance index vectors to generate an efficiency reward signal that can guide the scheduling strategy to optimize in the direction of satisfying the constraint objectives.

[0039] Specifically, when the constraint value projection mechanism is invoked, it accurately loads the delivery constraints and cost constraints closely related to the current production batch from its built-in production knowledge base. The delivery constraint is quantified as a remaining time value function with a 48-hour delivery deadline, indicating that the earlier the production step is completed, the higher its value. The cost constraint includes key economic indicators such as the idle cost of the production rig (500 units per hour) and the opportunity cost of occupying the M-003 model special mold (300 units per hour of occupation).

[0040] Based on these specific quantified business constraints, a constraint matrix deeply coupled with them is dynamically constructed. The weight parameters within this constraint matrix are mathematical representations of the evaluation criteria for scheduling behavior under different constraint conditions. For example, under the strong pressure of urgent delivery constraints, regions in the constraint matrix corresponding to state changes such as activating production platforms and accelerating mold turnover are assigned higher positive weights; conversely, regions corresponding to state changes that increase non-productive occupancy time are assigned negative weights. Then, using this constraint matrix "tailor-made" for the current task, a matrix sliding calculation is performed on the state difference tensor, calculated in the previous step, reflecting the state changes caused by the action of "starting production at platform 4." The matrix sliding calculation uses the constraint matrix as a convolution kernel to perform a two-dimensional convolution operation on the state difference tensor. The size of the convolution kernel is set to match the spatial layout dimensions of the production platforms (e.g., 3*3), the stride is set to 1, and the padding is set to 0. The ReLU activation function is used in the convolution operation to filter out negative state changes (such as resource idleness), retaining only positive contribution features, thereby generating an intermediate feature map representing local scheduling efficiency. In this feature map, the highlighted areas correspond to the state changes that contribute most to meeting the current core constraints (such as on-time delivery). Next, a global average aggregation operation is performed on this intermediate feature map to effectively reduce the dimensionality of the high-dimensional distributed feature information and project it onto two key performance indicator vectors: one representing capacity utilization and the other representing mold turnover rate. For example, if a spare platform is successfully activated during this scheduling, the capacity utilization indicator vector might be assigned a value of (0.95); simultaneously, if a high-value mold enters the production cycle, the mold turnover rate indicator vector might be assigned a value of (0.88). Finally, the two performance indicator vectors are weighted, fused, and scalarized. In this urgent order scenario, capacity utilization is assigned a weight of 0.6, and mold turnover rate is assigned a weight of 0.4 based on business priority. Through weighted summation, a single scalar value is generated, for example, 0.922. This value serves as the performance reward signal that guides the scheduling strategy to continuously optimize towards the combined constraint of meeting urgent delivery and efficient resource utilization.

[0041] This embodiment constructs a dynamic constraint matrix coupled with business constraints to perform in-depth value evaluation of the direct consequences of each micro-scheduling action, generating dense, smooth, and meaningful reward signals. This enables the reinforcement learning agent not only to know whether it "did the right thing to do," but also to accurately understand "how well it did" and "why it did well," effectively accelerating its convergence to the optimal scheduling strategy and thus enabling it to learn more sophisticated and efficient production scheduling strategies.

[0042] Furthermore, the specific implementation process of encapsulating the production module load tensor, resource scheduling matching instructions, and performance reward signals into semantic interaction experience and storing them in the scheduling experience replay pool includes: The production module tensor, which serves as the pre-scheduling state, the executed resource scheduling matching instruction, the generated performance reward signal, and the updated production module tensor, which serves as the post-scheduling state, are combined into a four-element data set containing complete state transition information. This four-element data set is then encapsulated as semantic interaction experience and added to the scheduling experience replay pool.

[0043] Specifically, after calculating that the decision to "start production of the first PC-003 component on pedestal 4" yielded an efficiency reward signal of 0.922, four core data objects directly related to this scheduling decision are collected: First, the production module load tensor as the pre-scheduling state, namely the "previous state tensor" captured at 14:00 on November 21, 2025, representing information such as pedestal 4 being idle and molds M003-SN01 being available; Second, the resource scheduling matching index actually executed by the agent. The instructions include the task identifier "ORD20251122-URGENT-Unit01", the resource code, and the execution timing. The third is the newly generated performance reward signal, which is a scalar value of 0.922. The fourth is the updated production module load tensor, which is captured at 14:05 and accurately reflects the changes such as the No. 4 platform becoming "occupied", the M003-SN01 mold becoming "in use", and the reduction of related material inventory.

[0044] These four data objects—"previous state tensor," "scheduling instruction," "reward signal," and "current state tensor"—are strictly combined into a logically indivisible four-element data set according to the reinforcement learning experience tuple format of (state S_t, action A_t, reward R_{t+1}, new state S_{t+1}). This data set constitutes the smallest knowledge unit containing complete information about state transitions, precisely recording "in which state (S_t), what action (A_t) is performed, what immediate feedback (R_{t+1}) is obtained, and what new state (S_{t+1}) is transitioned to." Subsequently, this four-element data set is encapsulated as a semantic interaction experience. This newly generated semantic interaction experience is added to a large-capacity, high-throughput scheduling experience replay pool. The scheduling experience replay pool can be physically implemented as a first-in-first-out circular queue or database, continuously collecting and storing experience records generated by each scheduling decision.

[0045] This embodiment constructs a scheduling experience replay pool and performs random sampling in subsequent training, which shuffles the temporal order of the data, making the training samples encountered by the model approximately independent and identically distributed. This effectively improves the stability and convergence efficiency of the learning algorithm. By storing semantic interaction experience, it can be repeatedly used in subsequent model iteration and optimization processes, thereby learning robust and high-performance scheduling strategies with extremely high efficiency, achieving continuous self-improvement and iterative optimization of scheduling capabilities.

[0046] Furthermore, the specific implementation process of gradient modulation based on the scheduling experience replay pool and iterative optimization of the scheduling strategy of the resource orchestration network includes: Semantic interaction experiences are randomly sampled from the scheduling experience replay pool and input into the resource orchestration network. The action probability distribution and state value assessment under the current scheduling strategy are calculated via forward propagation. A strategy loss function and a value loss function are constructed, and the gradient of the loss function with respect to the resource orchestration network parameters is calculated via backpropagation. The calculated gradient is modulated by identifying and amplifying the gradient contribution generated by the reward value semantic interaction experience, while suppressing the gradient influence of related experiences, thus creating a synergistic enhancement in the update directions among different training samples. The modulated gradient is used to update the network parameters of the resource orchestration network, driving the scheduling strategy to iteratively optimize.

[0047] Reference Figure 3 Specifically, firstly, from the scheduling experience replay pool that already stores tens of thousands of historical experiences, a unit of semantic interaction experience is randomly sampled in batches, for example, 128 four-element data sets are sampled at once. Because it is a random sampling, this data batch will naturally contain experiences with various performance reward values. There will be successful cases like the previously mentioned case of obtaining a high reward of 0.922 for efficiently initiating emergency orders, and there may also be cases that need improvement, such as obtaining a low reward of -0.15 for minor resource conflicts caused by scheduling decisions.

[0048] This batch of 128 experiences is input as a whole into the resource orchestration network. The network performs forward propagation computation on each experience in the batch via its internal neural network structure. For each experience's "state S_t," the network outputs two key results: first, the preference for all possible actions in that state under the current network parameters, i.e., an action probability distribution; second, a prediction of the long-term potential reward for being in that state, i.e., a state value assessment. Based on the network's output and the real data in the experience records, a policy loss function and a value loss function are constructed. The policy loss function aims to compare the network's predicted action probabilities with the actual actions that are executed and yield high rewards, encouraging the network to choose better actions in the future. The value loss function compares the network's predicted state value with the actual rewards and subsequent state values ​​recorded in the experience, improving the network's accuracy in judging the quality of states. Through automatic differentiation and backpropagation algorithms, the gradients of these two loss functions with respect to each weight parameter in the resource orchestration network can be calculated. This gradient vector indicates the direction and magnitude of the network parameters that should be adjusted to reduce losses.

[0049] This triggers a gradient modulation mechanism, which identifies and modulates each of the 128 gradient vectors in the batch. The gradient modulation formula is as follows: ,in Let R be the gradient vector before modulation, R be the reward signal value, and α be the modulation intensity hyperparameter. This is the modulated gradient vector. When R is positive, the gradient is amplified; when R is negative, the gradient is suppressed. For gradients resulting from successful experiences with reward values, their high positive value is identified and amplified by multiplying them by a coefficient greater than 1 to enhance their contribution to network parameter updates. Conversely, for failed experiences with reward values, which are identified as having low correlation and representing minor trial and error, their gradients are suppressed to avoid the model overreacting to such noise in the early stages of training by multiplying them by a coefficient less than 1. Through this differentiated processing, in the final aggregated gradient used to update network parameters, "correct" update directions derived from high-value successful experiences are synergistically enhanced, while "interference" directions generated by low-correlation or failed experiences are effectively weakened. Using this modulated, more directional aggregated gradient, the network parameters of the resource orchestration network are updated by an optimizer (e.g., the Adam optimizer). This process is repeated continuously within a training cycle, with each update driving the scheduling strategy to iteratively optimize towards better performance.

[0050] This embodiment, by setting a gradient modulation mechanism, endows the learning process with discriminative capabilities, enabling it to intelligently identify and amplify the most instructive successful experiences while filtering out irrelevant background noise. This allows it to concentrate its limited learning energy on the most critical knowledge points. Consequently, it accelerates the convergence speed of the scheduling strategy, and the ultimately learned scheduling strategy exhibits higher robustness and decision-making quality when facing various disturbances in the real production environment.

[0051] Example 2: This embodiment fully deploys the above-mentioned reinforcement learning-based intelligent scheduling and management method for precast component production resources in the precast concrete component production workshop of Factory X, thereby realizing the production scheduling of precast concrete components.

[0052] Furthermore, Factory X primarily produces large building exterior wall panels and composite beams. Its production scheduling faces challenges such as dynamic order insertion, limited mold resources, and multi-process coupling. At a specific production moment, for example, 14:00 on November 21, 2025, real-time data includes: order data from the Enterprise Resource Planning (ERP) system, specifically an urgent order "ORD20251122-URGENT" requiring the production of 20 "Exterior Wall Panel PC-003" units within 48 hours; data from the Manufacturing Execution System (MES), showing that production platform 3 is currently curing "Composite Beam PCB-001" and is expected to be idle in 2.5 hours, while platforms 4 and 5 are available; and data from IoT sensors, showing that the material warehouse has an inventory of 150 tons of "P.O42.5 Compensating Shrinkage Cement" and 5 sets of "M-003" special molds available. This embodiment maps these heterogeneous, multi-source data into a predefined three-dimensional data structure, namely the production mold load tensor. The production load tensor is structurally a coordinate system based on time, spatial resources, and resource status. Specifically, the first dimension is a discrete time axis with a 5-minute step; the second dimension is the physical resource axis within the factory, discretely identifying each production platform, curing kiln, and material storage unit; the third dimension is a feature vector used to describe in detail the specific status at a specific time and resource location, and its vector elements include dozens of quantitative indicators such as order number, product type code, current process progress, mold occupancy information, and material availability. Through this mapping, the complex production panorama of the factory at 14:00 is solidified into a high-dimensional digital snapshot, namely the production load tensor, providing a complete and standardized environmental status input for subsequent intelligent decision-making.

[0053] Furthermore, when the urgent order "ORD20251122-URGENT" triggers a scheduling request, the aforementioned generated production load tensor is input to the resource orchestration network as the current environmental state. Specifically, the resource orchestration network employs an encoder-decoder architecture based on the Transformer model. The encoder consists of multiple stacked encoding layers, each containing a multi-head attention module. After the production load tensor is input, the encoder, through its internal multi-head self-attention mechanism, captures and calculates the intrinsic correlations between all production resources in parallel from multiple perspectives. For example, one attention head might focus on the strong correlation between the idle states of pedestals 4 and 5 and the available state of mold M-003, while another attention head might calculate that although pedestal 3 is currently occupied, its expected idle time has a potential overlap with the time window required for the second batch of production under the urgent order. In this way, the encoder forms a deep and global understanding of the current state and potential of the entire production system. Subsequently, the decoder generates scheduling decision sequences one by one in an autoregressive manner. In the first step of the decision-making process, the decoder uses the urgent order demand (producing 20 PC-003 units) as a query and performs cross-attention calculations with the encoder's output information containing global resource correlations. This assesses the potential value of allocating the production task of the first component to each available resource. During this process, a constraint mask is generated based on the process knowledge base. This mask filters out all actions that do not conform to the hard process constraints, such as invalid combinations that assign PC-003 to molds other than M-003. The network ultimately outputs a probability distribution indicating that, under the current state, "starting production on platform 4 using molds M003-SN01" is the highest-value action. This decision is decoded into a resource scheduling matching instruction containing task identifiers, resource codes, and execution timing, and then sent to the MES system for execution.

[0054] Furthermore, after the MES system confirms the aforementioned instruction (e.g., at 14:05, pedestal 4 and mold M003-SN01 complete assembly and begin pouring) and updates the production status, it immediately captures the "previous state tensor" at 14:00 and the "current state tensor" at 14:05. A highly sparse state difference tensor is calculated by performing element-wise matrix subtraction on these two tensors. This difference tensor has non-zero values ​​only in the dimensions where state changes occur, precisely isolating all direct consequences of this scheduling action, such as: pedestal 4 changing from "idle" to "occupied," mold M003-SN01 changing from "available" to "in use," and a corresponding reduction in cement inventory. Using a constraint matrix deeply coupled with business constraints including delivery and cost constraints, state changes that can advance urgent orders are identified and amplified; simultaneously, state changes that lead to resource idleness or increased costs are identified and penalized. As the constraint matrix slides across the state difference tensor, intermediate feature maps are generated. For example, since this action directly initiates the production of an urgent order, a strong highlight response will appear on the graph. By performing a global average aggregation operation on the graph, the high-dimensional feature information is reduced in dimensionality and mapped onto performance indicator vectors representing capacity utilization and mold turnover rate. Based on the current business scenario (urgent orders take priority), these two performance indicator vectors are weighted, fused, and scalarized. For example, delivery-related performance indicators are assigned a weight of 0.7, and cost-related indicators are assigned a weight of 0.3, ultimately generating a single quantified efficiency reward signal.

[0055] Furthermore, this four-element data set containing complete state transition information before and after scheduling—namely, the previous state tensor at 14:00, the executed scheduling instruction, the performance reward signal of 0.922, and the current state tensor at 14:05—is encapsulated as a semantic interaction experience and stored in a large-capacity scheduling experience replay pool. Then, a batch (e.g., 128 experiences) of semantic interaction experiences is randomly sampled from this replay pool and input into the resource orchestration network for training. After backpropagation calculates the gradient of the loss function with respect to the network parameters, the gradient modulation mechanism discriminates and modulates the gradient generated by each experience within the batch. Specifically, for successful experiences like those in this embodiment that obtain a high reward value of 0.922, the modulation mechanism multiplies them by a coefficient greater than 1 (e.g., 1.5) to amplify their gradient contribution; while for failed experiences that lead to low or even negative rewards, they are multiplied by a coefficient less than 1 (e.g., 0.8) to suppress their gradient impact. This differentiated approach significantly enhances the "correct" update directions derived from high-value successes in the final aggregated gradient used to update network parameters, while effectively weakening the "disruptive" directions derived from low-relevance or failures. The modulated gradient is then used to update the network parameters of the resource orchestration network, thereby driving the entire scheduling strategy to iterate towards a more optimized direction and achieving intelligent production scheduling of precast concrete components.

[0056] This embodiment achieves self-evolution and iterative optimization of scheduling by learning from a large amount of historical data, greatly improving the flexibility and robustness of production scheduling. By introducing a resource orchestration network based on a multi-head attention mechanism, it can capture the complex dependencies between production factors from a global perspective and generate scheduling instructions that maximize cumulative returns, thereby significantly improving capacity utilization and mold turnover rate. The designed experience playback and gradient modulation mechanism can amplify the gradient contribution of high-reward experiences, accelerate the convergence process of the model to the optimal scheduling strategy, and achieve continuous self-improvement of scheduling capabilities.

[0057] It should be clarified that the embodiments described above are merely exemplary and are intended to aid in understanding the present invention, not to limit it. Those skilled in the art can make various changes and modifications after grasping the core ideas of the present invention. Therefore, the scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent scheduling and management of prefabricated component production resources based on reinforcement learning, characterized in that, include: Real-time data collection of premixed shrinkage-compensating concrete resources, including order requirements, material distribution, and production platform data, is mapped to a production load tensor that characterizes the current production status of precast components. When a scheduling request is triggered, the production module load tensor is input as the environment state into the resource orchestration network; The resource orchestration network utilizes a scheduling strategy based on a multi-head attention mechanism to search and reason within the action space that satisfies the hard constraints of the process, generating resource scheduling matching instructions that can accumulate rewards. After executing the resource scheduling matching instruction to update the production status of prefabricated components, the state difference tensor is calculated and the constraint value projection mechanism is initiated; combined with the delivery constraints and cost constraints extracted from the resource data, the difference between the production status before and after is calculated in a matrix, and an efficiency reward signal including capacity utilization rate and mold turnover rate is output. The production module load tensor, resource scheduling matching instructions, and performance reward signals are encapsulated into semantic interaction experience and stored in the scheduling experience replay pool. Gradient modulation is performed based on the scheduling experience replay pool to iteratively optimize the scheduling strategy of the resource orchestration network.

2. The intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning according to claim 1, characterized in that, The specific implementation process of real-time collection of premixed shrinkage-compensating concrete resource data, including order requirements, material distribution, and production platform data, and mapping it to a production load tensor characterizing the current precast component production status includes: The system acquires order demand data, including delivery sequence and process path, material distribution data, representing inventory status and logistics timeliness, and production element data, reflecting space occupancy and turnover cycle, from the enterprise resource planning system, production execution system, and IoT sensor terminals. The system performs data cleaning and time-series alignment on the production element data and maps it to a coordinate system with time, space, and resource status as the reference axes, in accordance with preset dimension mapping rules, to form a production module tensor.

3. The intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning according to claim 1, characterized in that, When a scheduling request is triggered, the specific implementation process of inputting the production module load tensor as an environmental state into the resource orchestration network includes: Upon receiving a scheduling request generated by dynamic changes in production factors, the production module tensor is sent to the resource orchestration network to mine and capture the dependencies and interactions between order requirements, material distribution, and production platforms.

4. The intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning according to claim 1, characterized in that, The resource orchestration network utilizes a multi-head attention-based scheduling strategy to search and reason within an action space that satisfies process hard constraints, generating resource scheduling matching instructions that can accumulate rewards. The specific implementation process includes: Based on the hard process constraints including component maintenance cycles, mold matching relationships, and process logic associations, a constraint mask is generated to shield invalid scheduling actions. Within the set of valid actions defined by the constraint mask, the resource orchestration network generates scheduling decisions one by one through autoregression. Its internal multi-head attention mechanism dynamically evaluates the correlation weight between the current production state and candidate scheduling tasks in each decision step, and calculates the probability distribution of all valid actions. The scheduling scheme with the highest action value is sampled from the probability distribution and selected, and decoded into a resource scheduling matching instruction containing task identifier, resource code, and execution sequence.

5. The intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning according to claim 1, characterized in that, After executing the resource scheduling matching instruction to update the production status of the prefabricated components, the specific implementation process of calculating the state difference tensor and initiating the constraint value projection mechanism includes: The production status of prefabricated components is monitored in real time. When the production status is updated after the resource scheduling matching instruction is completed, the production module load tensor before and after the scheduling execution is automatically captured, the state difference tensor is calculated, and the constraint value projection mechanism is invoked.

6. The intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning according to claim 1, characterized in that, The specific implementation process of combining the delivery constraints and cost constraints extracted from the resource data, performing matrix calculations on the differences in production status before and after, and outputting performance reward signals including capacity utilization and mold turnover rate includes: Load delivery constraints and cost constraints associated with the current production batch from the production knowledge base, construct a constraint matrix coupled with the delivery constraints and cost constraints, and the weight parameters of the constraint matrix represent the scheduling behavior evaluation criteria under different constraint conditions; perform matrix sliding calculation on the state difference tensor using the constraint matrix to generate an intermediate feature map representing scheduling efficiency; perform a global averaging and aggregation operation on the intermediate feature map to reduce the dimensionality of high-dimensional feature information and map it onto performance index vectors representing capacity utilization and mold turnover rate respectively; perform weighted fusion and scalarization processing on the performance index vectors to generate an efficiency reward signal that can guide the scheduling strategy to optimize in the direction of satisfying the constraint objectives.

7. The intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning according to claim 1, characterized in that, The specific implementation process of encapsulating the production module load tensor, resource scheduling matching instructions, and performance reward signals into semantic interaction experience and storing them in the scheduling experience replay pool includes: The production module tensor, which serves as the pre-scheduling state, the executed resource scheduling matching instruction, the generated performance reward signal, and the updated production module tensor, which serves as the post-scheduling state, are combined into a four-element data set containing complete state transition information. This four-element data set is then encapsulated as semantic interaction experience and added to the scheduling experience replay pool.

8. The intelligent scheduling and management method for prefabricated component production resources based on reinforcement learning according to claim 1, characterized in that, The specific implementation process of gradient modulation based on the scheduling experience replay pool and iterative optimization of the scheduling strategy of the resource orchestration network includes: Semantic interaction experiences are randomly sampled from the scheduling experience replay pool and input into the resource orchestration network. The action probability distribution and state value assessment under the current scheduling strategy are calculated via forward propagation. A strategy loss function and a value loss function are constructed, and the gradient of the loss function with respect to the resource orchestration network parameters is calculated via backpropagation. The calculated gradient is modulated by identifying and amplifying the gradient contribution generated by the reward value semantic interaction experience, while suppressing the gradient influence of related experiences, thus creating a synergistic enhancement in the update directions among different training samples. The modulated gradient is used to update the network parameters of the resource orchestration network, driving the scheduling strategy to iteratively optimize.

Images