Methods, apparatus and systems for adaptive generation of production scheduling strategies

Abstract

A method, apparatus, and system for adaptively generating production scheduling strategies include: generating an initial production scheduling strategy scheme and constructing an initial scheduling resource time sequence map corresponding to the initial production scheduling strategy scheme; acquiring the real-time physical state context of the target production unit within the time window triggered by manual adjustment operations; encoding the structural changes in the scheduling resource time sequence map caused by manual adjustment operations into scheduling edit event vectors; constructing production scheduling preference training samples and using the training samples to train a production scheduling preference evaluation model; and outputting an updated production scheduling strategy scheme based on the optimized production scheduling strategy generation model. This application combines unstructured operations on the graphical interface with the real-time state of the underlying physical world, transforming them into preference alignment data that artificial intelligence can understand and learn, thereby achieving closed-loop self-evolution of the production scheduling strategy and solving the problem that implicit human experience in industrial production scheduling systems cannot be effectively learned and absorbed by artificial intelligence systems.

Description

Technical Field

[0001] This application relates to the field of intelligent manufacturing, and in particular to a method, apparatus and system for adaptively generating production scheduling strategies. Background Technology

[0002] In discrete manufacturing and process industries, production scheduling is a core element in ensuring production efficiency and on-time delivery. Traditional Automated Production Scheduling (APS) systems rely primarily on fixed algorithms or rules, resulting in rigid scheduling plans that struggle to adapt to complex and ever-changing production realities (such as equipment failures, material shortages, and urgent order insertions). When automatically generated plans fail to meet the experience of on-site schedulers or fail to adequately address unforeseen circumstances, manual adjustments are necessary. Currently, adjustment records are primarily handled through simple recording or as static case libraries, lacking a closed-loop mechanism that deeply integrates the intelligence of manual adjustments with real-time production status and dynamically feeds feedback to optimize the scheduling decision model. This results in stagnant system intelligence, hindering continuous learning through human-machine collaboration and preventing the development of a scheduling "brain" that becomes increasingly intelligent with use. Summary of the Invention

[0003] This application provides a method, apparatus, and system for adaptively generating production scheduling strategies.

[0004] According to a first aspect of this application, this application provides an adaptive production scheduling strategy generation method, comprising: Generate an initial production scheduling strategy and construct an initial scheduling resource time sequence map corresponding to the initial production scheduling strategy; In response to a manual adjustment operation on the initial scheduling resource timing map, the instantaneous physical state context of the target production unit is obtained within the time window in which the manual adjustment operation is triggered. The structural changes in the scheduling resource time sequence graph caused by the manual adjustment operation are encoded as scheduling edit event vectors; Based on the real-time physical state context, the scheduling and editing event vector, the pre-adjustment production schedule plan, the post-adjustment production schedule plan, and the actual execution result of the post-adjustment production schedule plan, a production schedule preference training sample is constructed, and the production schedule preference evaluation model is trained using the training sample. Using the production scheduling preference evaluation model as the optimization objective, a constrained reinforcement learning algorithm is employed to drive parameter optimization of the production scheduling strategy generation model. Based on the optimized production scheduling strategy generation model, an updated production scheduling strategy scheme is output.

[0005] In the method provided in this application embodiment, the node types of the scheduling resource timing graph include at least one of the following: work order node, process node, equipment node, tooling node, material node, personnel node, time window node, and abnormal event node. In the method provided in this application embodiment, the types of edges in the scheduling resource timing graph include: process dependency edges representing the sequence of processes; equipment capacity edges representing the matching relationship between the processing capacity of processes and available equipment; resource occupancy edges representing the occupancy relationship between processes and materials, tooling, or personnel; mutual exclusion constraint edges representing that the same resource cannot be repeatedly occupied within the same time window; abnormal impact edges representing the impact of abnormal events on work orders, processes, or equipment; and time constraint edges representing delivery dates or maintenance windows. In the method provided in this application embodiment, encoding the structural changes in the scheduling resource time sequence graph caused by the manual adjustment operation into a scheduling edit event vector specifically includes: Obtain the timing graphs of the scheduled resources before and after the adjustment, respectively; The time sequence graph of the scheduled resources before adjustment is converted into a first adjacency matrix or a first adjacency tensor, and the time sequence graph of the scheduled resources after adjustment is converted into a second adjacency matrix or a second adjacency tensor. Calculate the difference between the second adjacency matrix or the second adjacency tensor and the first adjacency matrix or the first adjacency tensor to obtain graph structure difference data; The scheduling and editing event vector is generated by fusing the graph structure difference data with the operational semantics of the manual adjustment operation.

[0006] The method provided in this application embodiment, in which the production scheduling preference training sample is constructed, further includes a sample quality control step: When the manual adjustment operation is performed by a user with low privileges, the real-time physical state context is missing, the actual execution result is abnormally missing, the adjusted production scheduling plan is not actually executed, or the adjusted production scheduling plan causes a significant deterioration in the preset key indicators, the corresponding candidate training data will be marked as low-confidence samples or removed from the training set.

[0007] In the method provided in this application embodiment, the constraints of the constrained reinforcement learning algorithm include: A hard constraint penalty term is introduced to penalize the scheme output by the production scheduling strategy generation model for violating process timing constraints, equipment capacity constraints, material kitting constraints, or resource mutual exclusion constraints. Set a policy update magnitude limit to limit the policy distribution offset of a single model update; The optimized production scheduling strategy generation model is used to perform sandbox simulation verification. If the solution fails the simulation verification, the model version that generates the solution is prohibited from being released to the production scheduling system.

[0008] The method provided in this application embodiment, wherein the step of outputting an updated production scheduling strategy scheme based on the optimized production scheduling strategy generation model, includes: The updated production scheduling strategy was input into the sandbox simulation environment for full-chain simulation verification. When the updated production scheduling strategy is verified, it will be submitted to the management terminal for manual confirmation. Upon receiving a manual confirmation instruction, the updated production scheduling strategy will be published to the production scheduling system or sent to the production terminal for execution.

[0009] According to a second aspect of this application, this application provides a scheduling strategy adaptive generation apparatus, comprising: The strategy generation module is used to generate an initial production scheduling strategy scheme and construct an initial scheduling resource timing graph corresponding to the initial production scheduling strategy scheme; The context acquisition module is used to respond to manual adjustment operations on the initial scheduling resource timing map and acquire the real-time physical state context of the target production unit within the time window when the manual adjustment operation is triggered. The graph difference encoding module is used to encode the structural changes in the scheduling resource time sequence graph caused by the manual adjustment operation into a scheduling edit event vector; The model building module is used to construct scheduling preference training samples based on the real-time physical state context, the scheduling editing event vector, the pre-adjustment production scheduling plan, the post-adjustment production scheduling plan, and the actual execution results of the post-adjustment production scheduling plan, and to train the production scheduling preference evaluation model using the training samples. The model optimization module is used to optimize the parameters of the production scheduling strategy generation model by using the production scheduling preference evaluation model as the optimization objective and employing a constrained reinforcement learning algorithm. The solution output module is used to generate a model based on the optimized production scheduling strategy and output the updated production scheduling strategy solution.

[0010] According to a third aspect of this application, this application provides a production scheduling strategy adaptive generation system, comprising: The upper-level computing node is used to execute the production scheduling strategy adaptive generation method described above; The management terminal is communicatively connected to the host computing node and is used to present the initial production scheduling strategy in a visual form of scheduling resource time sequence graph and capture the user's adjustment operations. The edge gateway communicates with the upper computing node and is used to collect real-time physical status data from the underlying production equipment cluster to form an instantaneous physical status context. The sandbox simulation environment is connected to the host computing node and is used to perform full-process simulation verification of the updated production scheduling scheme generated after model optimization in the host computing node.

[0011] According to a fourth aspect of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the production scheduling strategy adaptive generation method as described above. Due to the adoption of the above technical solutions, the beneficial effects of this application are as follows: The adaptive production scheduling strategy generation method, apparatus, and system provided in this application include: generating an initial production scheduling strategy scheme; converting the initial production scheduling strategy scheme into an initial scheduling resource time sequence graph; acquiring the real-time physical state context of the target production unit within the time window triggered by the manual adjustment operation; encoding the structural changes in the scheduling resource time sequence graph caused by the manual adjustment operation into a scheduling edit event vector; constructing production scheduling preference training samples and using the training samples to train a production scheduling preference evaluation model; using a constrained reinforcement learning algorithm to drive parameter optimization of the production scheduling strategy generation model; and outputting an updated production scheduling strategy scheme based on the optimized production scheduling strategy generation model. This application combines unstructured operations on the graphical interface with the real-time state of the underlying physical world, transforming them into preference alignment data that artificial intelligence can understand and learn, thereby achieving closed-loop self-evolution of the production scheduling strategy and solving the problem that implicit human experience in industrial production scheduling systems cannot be effectively learned and absorbed by artificial intelligence systems. Attached Figure Description

[0012] Figure 1 A flowchart illustrating one implementation of the method provided in this application; Figure 2 A flowchart of sub-steps of the method provided in one embodiment of this application; Figure 3 A flowchart of sub-steps in another implementation of the method provided in this application embodiment; Figure 4 A schematic diagram of the program modules of the apparatus provided in one embodiment of this application. Detailed Implementation

[0013] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0014] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0015] The serial numbers assigned to components in this article, such as "first" and "second", are used only to distinguish the objects being described and have no sequential or technical meaning.

[0016] Furthermore, the technical features and solutions described herein can be combined in any suitable manner in one or more embodiments. It will be readily understood by those skilled in the art that the steps or order of operations related to the embodiments provided herein can also be changed. Therefore, any order in the drawings and embodiments is for illustrative purposes only and does not imply a requirement to follow a particular order, unless explicitly stated otherwise.

[0017] like Figure 1 As shown in the embodiments of this application, one implementation of the adaptive production scheduling strategy generation method includes the following steps: Step 110: Generate an initial production scheduling strategy and construct an initial scheduling resource sequence diagram corresponding to the initial production scheduling strategy.

[0018] In some embodiments, the node types of the scheduling resource sequence graph may include at least one of the following: work order node, process node, equipment node, tooling node, material node, personnel node, time window node, and abnormal event node. In some embodiments, the types of edges in the scheduling resource sequence graph may include: process dependency edges representing the sequence of processes; equipment capacity edges representing the matching relationship between the processing capacity of a process and available equipment; resource occupancy edges representing the occupancy relationship between a process and materials, tooling, or personnel; mutual exclusion constraint edges representing that the same resource cannot be repeatedly occupied within the same time window; abnormal impact edges representing the impact of abnormal events on work orders, processes, or equipment; and time constraint edges representing delivery dates or maintenance windows. Step 120: In response to a manual adjustment operation on the initial scheduling resource timing map, obtain the real-time physical state context of the target production unit within the time window in which the manual adjustment operation is triggered.

[0019] By collecting real-time physical state context within the time window triggered by manual adjustment operations, a strong correlation can be established between manual adjustment actions and the actual on-site state, preserving the equipment, materials, anomalies, work-in-process, and personnel environment at the time of the user's adjustment. This application's embodiment collects real-time physical state context (such as equipment load, material inventory, and work-in-process status) simultaneously with the adjustment event, ensuring that the learned preferences are bound to specific operating conditions, thus improving the model's decision-making accuracy and scenario adaptability.

[0020] In some embodiments, the instantaneous physical state context may include: At least one of the following equipment status information: current operating status of equipment, status of equipment tri-color indicator lights, equipment alarm code, equipment downtime duration, effective output quantity of equipment, equipment load status, and equipment heartbeat status; At least one of the following production progress information: current progress of work order, actual output quantity, planned output quantity, planned deviation, work-in-process location, and work-in-process quantity; At least one of the following material status information: material availability status, estimated arrival time, material shortage work order, material batch, and material delivery status; Information on at least one of the following abnormal events: Andon call type, cause of abnormality, duration of abnormality, status of abnormality response, and person handling the abnormality; Personnel status information including personnel shift, personnel skills, personnel availability status, and job configuration.

[0021] Step 130: Encode the structural changes in the scheduling resource timing graph caused by manual adjustment operations into scheduling edit event vectors.

[0022] By converting production scheduling schemes into scheduling resource time-series graphs and generating graph structure difference data based on the graphs before and after adjustments, manual adjustment operations are no longer just interface operations or result overwriting, but are transformed into computable and trainable scheduling editing events. This application's embodiments encode graph structure changes into scheduling editing event vectors, transforming the originally unstructured, intuition-dependent manual adjustment behavior into machine-quantifiable and learnable data samples, laying the foundation for knowledge transfer.

[0023] In some embodiments, the scheduling edit event vector may include one or more of the following: operation type encoding, object being adjusted encoding, resource encoding before adjustment, resource encoding after adjustment, time window encoding before adjustment, time window encoding after adjustment, constraint change encoding, priority change encoding, anomaly association encoding, and adjustment magnitude encoding.

[0024] like Figure 2 As shown, step 130 may specifically include the following steps: Step 1301: Obtain the timing diagram of the scheduling resources before and after the adjustment, respectively; Step 1302: Convert the time sequence graph of the scheduled resources before adjustment into a first adjacency matrix or a first adjacency tensor, and convert the time sequence graph of the scheduled resources after adjustment into a second adjacency matrix or a second adjacency tensor. Step 1303: Calculate the difference between the second adjacency matrix or the second adjacency tensor and the first adjacency matrix or the first adjacency tensor to obtain graph structure difference data; Step 1304: Based on the graph structure difference data and the operation semantics of manual adjustment operations, generate scheduling and editing event vectors.

[0025] Step 140: Based on the real-time physical state context, scheduling editing event vector, pre-adjustment production schedule plan, post-adjustment production schedule plan, and the actual execution results of the post-adjustment production schedule plan, construct a production schedule preference training sample, and use the training sample to train the production schedule preference evaluation model.

[0026] By constructing scheduling preference training samples together with scheduling editing event vectors, real-time physical state context, pre- and post-adjustment production plans and execution results, the model can learn not only "what the user changed", but also "why the user changed it under what circumstances, and what the effect was after the change".

[0027] In some embodiments, constructing a production scheduling preference training sample may include: The real-time physical state context, the pre-production scheduling plan, the post-production scheduling plan, the scheduling and editing event vector, and the user identity or job information are associated to form candidate training samples; After the adjusted production schedule is implemented, the execution results are obtained. The execution results may include at least one of the following: delivery date achievement rate, equipment utilization rate, number of mold changes, downtime, anomaly recovery time, material waiting time, number of plan changes, and manual confirmation results. Generate sample quality labels based on the execution results; The sample quality labels are associated with candidate training samples to form production scheduling preference training samples.

[0028] In some embodiments, constructing scheduling preference training samples may further include a sample quality control step: When manual adjustment operations are performed by users with low privileges, the real-time physical state context is missing, the actual execution results are missing, the adjusted production scheduling plan is not actually executed, or the adjusted production scheduling plan causes a significant deterioration in the preset key indicators, the corresponding candidate training data will be marked as low-confidence samples or removed from the training set.

[0029] In some embodiments, training a production scheduling preference evaluation model using training samples may include: Step 150: Using the production scheduling preference evaluation model as the optimization objective, a constrained reinforcement learning algorithm is used to drive the parameter optimization of the production scheduling strategy generation model.

[0030] By employing a production scheduling preference evaluation model, constrained optimization, sample filtering, expert confirmation, and sandbox validation mechanisms, the risk of learning low-quality manual adjustments or occasional temporary operational errors into the model is reduced.

[0031] In some embodiments, the constraints of a constrained reinforcement learning algorithm may include: Introduce hard constraint penalty terms to penalize the production scheduling strategy for violating process timing constraints, equipment capacity constraints, material completeness constraints, or resource mutual exclusion constraints in the model output. Set a policy update magnitude limit to limit the policy distribution offset of a single model update; The scheme generated after optimizing the production scheduling strategy generation model is tested in a sandbox simulation. If the scheme fails the simulation verification, the model version that generates the scheme shall not be released to the production scheduling system.

[0032] Step 160: Generate a model based on the optimized production scheduling strategy and output the updated production scheduling strategy scheme.

[0033] By continuously optimizing the production scheduling strategy generation model through real human-computer interaction feedback, the production scheduling system can gradually learn the production scheduling preferences of different factories, workshops, planners, and production scenarios, thereby improving the adaptability, feasibility, and user acceptance of the production scheduling plan.

[0034] like Figure 3As shown, in the method provided in this application embodiment, step 160 may specifically include the following steps: Step 1601: Input the updated production scheduling strategy into the sandbox simulation environment for full-chain simulation verification; Step 1602: When the updated production scheduling strategy is verified, submit the updated production scheduling strategy to the management terminal for manual confirmation. Step 1603: After receiving the manual confirmation instruction, publish the updated production scheduling strategy to the production scheduling system or send it to the production terminal for execution.

[0035] This application provides an adaptive production scheduling strategy generation method. In one specific application example, this method can be executed by a computing node located at the industrial edge or in the cloud. Its hardware environment may include at least a CPU, memory, storage unit, and network interface. Its software environment includes an operating system, a database system, and necessary algorithm frameworks (such as computing libraries and deep learning frameworks).

[0036] This method interacts with an API and service scheduling management system (APS), a production execution system (MES), a supervisory data acquisition system (SCADA / DCS), and a dispatcher management terminal.

[0037] Step S1: Solution Display and Graph Conversion. Generate an initial production scheduling strategy solution and construct an initial scheduling resource time sequence graph corresponding to the initial production scheduling strategy solution.

[0038] First, the production scheduling strategy generation model generates an initial production scheduling plan based on constraints such as the list of work orders to be scheduled, the bill of materials (BOM), process paths, and equipment and resource capabilities. This is achieved using constraint programming, heuristic algorithms, or learned model strategies. The system (in this application example, "system" does not refer to a general computer or software system, but specifically to the real industrial production system scheduled by the production scheduling strategy) then converts this scheduling plan into a structured data representation—a scheduling resource time sequence graph (e.g., G=(V, E)). Here, the node set V represents entities such as work orders, processes, equipment, tooling, materials, personnel, time windows, and abnormal events. The edge set E represents the relationships between these entities, mainly including: Process dependency edge: connects process nodes that have sequential requirements.

[0039] Equipment capability edge: Connects the process node to the equipment node that it can execute, and carries information such as processing time.

[0040] Resource occupation edge: connects the process node with the material, tooling, and personnel nodes that it consumes / occupies.

[0041] Mutually exclusive constraint edges: connect two or more process nodes that are competing for the same resource (such as critical equipment) within the same time window.

[0042] Anomaly Impact Edge: Connects the anomaly event node to the work order, process, or equipment node affected by it.

[0043] The system is also responsible for visually presenting the generated initial scheduling resource timing map on the scheduler's management terminal through a graphical interface.

[0044] Step S2: Interactive Capture and Context Acquisition. In response to manual adjustments to the initial scheduling resource timing graph, acquire the real-time physical state context of the target production unit within the time window triggered by the manual adjustment.

[0045] On the management terminal interface, the dispatcher adjusts nodes (such as a process block) or edges in the graph by dragging, drawing lines, and using right-click menus. The system captures this "manual adjustment operation" and accurately records the operation time, type, object, and state changes before and after the operation. Simultaneously, the system retrieves the real-time physical state context of the target production unit (such as the target workshop or production line) from the edge gateway or MES system within the time window of this adjustment operation (e.g., 10 minutes before and after the adjustment). This context typically includes: real-time equipment operating data (such as speed, temperature, stop / standby / running status); material inventory and online quantity; equipment load rate (idle / busy) and currently executing task ID; the status and location of currently processed work-in-process (WIP); and any alarm or abnormal event information. This data is encapsulated into a structured or semi-structured context data packet and associated with the adjustment operation event.

[0046] Step S3: Graph Structure Differentiation and Event Encoding. The structural changes in the scheduling resource time sequence graph caused by manual adjustment operations are encoded into scheduling edit event vectors.

[0047] The system acquires the scheduling resource time-series graph Gb before adjustment and the scheduling resource time-series graph Ga after adjustment. To quantify the structural changes in the graph, the system converts the graph into an adjacency matrix (or a higher-dimensional adjacency tensor) A. Specifically, for each type of edge, an adjacency matrix can be created, such as Ab_cap and Aa_cap representing the device capability adjacency matrices before and after adjustment.

[0048] Then, the graph structure difference data ΔA = Aa - Ab is calculated. The difference matrix ΔA clearly indicates which edge weights (or existences) have changed: the non-zero elements in ΔA identify edges that have been deleted (negative values), added (positive values), or modified.

[0049] The system parses the semantics of this "manual adjustment operation". For example, it encodes "moving process P01 from equipment M1 to M2" as "operation type: MOVE_PROCESS", "operation object: {P01}", and "changing resources: {from: M1, to: M2}".

[0050] Finally, the graph structure difference data ΔA (if flattened into a vector) is concatenated or fused with the parsed operation semantic vector (after one-hot encoding or embedding encoding) to form a unified scheduling edit event vector E that represents this scheduling adjustment event.

[0051] Step S4: Sample Construction and Quality Filtering, Training the Preference Evaluation Model. Based on the real-time physical state context, scheduling edit event vectors, pre-adjustment production schedule plan, post-adjustment production schedule plan, and the actual execution results of the post-adjustment production schedule plan, a production schedule preference training sample is constructed, and the production schedule preference evaluation model is trained using the training sample.

[0052] The system creates a scheduling preference training sample S, typically structured as: S = (C, E, Pb, Pa, R). Here, C is the immediate physical state context, E is the scheduling edit event vector, Pb and Pa are the pre- and post-adjustment scheduling schemes (or their summary features, such as the critical path length of the scheduling Gantt chart, total equipment load, etc.), respectively. R is the evaluation of the actual execution results of the adjusted scheme, which can be obtained from the MES system after the scheme is executed, such as average equipment utilization, deviation between actual and planned completion time, and delivery date achievement rate.

[0053] To ensure sample validity, the system will implement sample quality control strategies: Check the permissions of the operation initiator. If it is a user with low permissions such as an observer, do not build a sample or reduce its weight.

[0054] Check if C contains critical status information (such as the status of core devices); if it is missing, remove it.

[0055] Check if R is available. If the actual execution result cannot be obtained or is obviously abnormal (such as all devices being idle), then remove it.

[0056] Key Performance Indicator (KPI) Check: If the execution result (R) of the adjusted plan deteriorates significantly in key indicators (such as increased costs and delivery delays) compared to the expected result of the Pb plan based on the original model prediction or historical data, this sample is marked as a negative sample or is directly removed.

[0057] After collecting a large number of production scheduling preference training samples S, a production scheduling preference evaluation model φ is trained offline. This model can be a deep learning-based ranking network (Learning to Rank) or a rating model. Its input is (C, Pb, Px), and its output is a preference rating or preference probability. The training objective of model φ is to make its rating φ(C, Pb, Pa) reflect whether the adjustment from Pb to Pa is better (better R index) or better in line with expert preferences as embodied in S.

[0058] Step S5: Strategy Model Optimization and Security Verification. Using the production scheduling preference evaluation model as the optimization objective, a constrained reinforcement learning algorithm is employed to drive parameter optimization of the production scheduling strategy generation model.

[0059] The trained preference evaluation model φ is used to optimize the core scheduling strategy generation model π. A constrained reinforcement learning framework can be used here. Agent: Production scheduling strategy generation model π.

[0060] Environment: A simulator for the production process, taking π as input for the production scheduling plan and outputting the state changes after execution.

[0061] State: Real-time physical state context C and information on currently unscheduled tasks.

[0062] Action: Generate a complete production scheduling plan Pa′.

[0063] Reward: The core of the reward comes from the score given by the preference evaluation model φ(C, Pb, π(C)), which evaluates the improvement of the new strategy π compared to the baseline strategy (which may be the old model or the solution Pb generated by the heuristic algorithm). Hard constraint penalty term: The hard constraint penalty term is directly calculated as the total penalty value for the solution to violate constraints such as process dependence and resource mutual exclusion, and is deducted from the total reward.

[0064] Learning objective: To maximize the cumulative reward from the preference model φ while satisfying constraints, using RL algorithms such as policy gradient and PPO.

[0065] Policy update magnitude constraint: Use trust domain methods (such as TRPO, PPO-CLIP) or directly add a constraint on the Kullback-Leibler (KL) divergence of the policy distribution to the optimization objective to prevent the model from drifting drastically and losing stability in a single update due to inaccurate φ scores.

[0066] When generating a new solution Pa′ after policy exploration or model optimization in reinforcement learning, sandbox simulation verification is enabled. Solution Pa′ is fed into an offline, high-precision production process simulator (i.e., the sandbox) for rapid simulation of its entire production process. This simulator includes equipment failure models, material supply delay models, etc. If the simulation results violate key constraints (such as material shortages leading to interruptions) or certain KPIs fall below a certain safety threshold, it indicates that model φ may have learned incorrect preferences or the optimization process may have gone out of control. In this case, the new version of the model that generates this solution will be prohibited from being released, and the system can revert to a previous version or mark this optimization as invalid.

[0067] Step S6: Update, Release, and Execute. Based on the optimized production scheduling strategy generation model, output the updated production scheduling strategy solution.

[0068] Once the optimized new strategy generation model has passed multiple rounds of internal verification, and its updated production scheduling plan (which may be the scheduling result for a completely new set of tasks) performs stably and excellently in the final sandbox full-link simulation before release, the system submits the new model version and / or its generated specific plan to a senior scheduler or production line manager for review and manual confirmation on the management terminal. Once approved, the new model will be updated to the online inference service for the next actual production scheduling calculation; or it will be authorized to officially release the verified plan to the production scheduling system and issue specific work instructions to each production terminal for execution.

[0069] like Figure 4 As shown, the production scheduling strategy adaptive generation device provided in this application embodiment may include a strategy generation module 410, a context acquisition module 420, a graph differential encoding module 430, a model building module 440, a model optimization module 450, and a scheme output module 460.

[0070] The strategy generation module 410 is used to generate an initial production scheduling strategy scheme and construct an initial scheduling resource timing diagram corresponding to the initial production scheduling strategy scheme. The context acquisition module 420 is used to respond to manual adjustment operations on the initial scheduling resource timing map and acquire the real-time physical state context of the target production unit within the time window when the manual adjustment operation is triggered. The graph difference encoding module 430 is used to encode the structural changes in the scheduling resource time sequence graph caused by manual adjustment operations into scheduling edit event vectors; The model building module 440 is used to build scheduling preference training samples based on the real-time physical state context, scheduling editing event vector, pre-adjustment scheduling plan, post-adjustment scheduling plan and the actual execution results of the post-adjustment scheduling plan, and to train the scheduling preference evaluation model using the training samples. The model optimization module 450 is used to optimize the parameters of the production scheduling strategy generation model by using a production scheduling preference evaluation model as the optimization objective and employing a constrained reinforcement learning algorithm. The solution output module 460 is used to generate a model based on the optimized production scheduling strategy and output an updated production scheduling strategy solution.

[0071] The production scheduling strategy adaptive generation system provided in this application embodiment may include: The upper-level computing node is used to execute the production scheduling strategy adaptive generation method as described above.

[0072] The upper-level computing node, acting as the core server / cloud server, undertakes all the core computing logic of the adaptive generation method of the above scheduling strategy, namely the generation and transformation of the scheme, event encoding, model training and optimization, etc. The upper-level computing node can be equipped with high-performance GPUs, mainly used for the training and inference of neural network models.

[0073] The management terminal communicates with the host computing node and is used to present the initial production scheduling strategy in a visual form of scheduling resource time sequence graph and capture user adjustment operations.

[0074] The management terminal can be a PC client or a web application, serving as a human-computer interaction interface responsible for visualizing the solution and collecting data from the scheduler's interactions. The management terminal communicates with the host computing node via a stable network connection.

[0075] The edge gateway communicates with the upper-level computing nodes and is used to collect real-time physical status data from the underlying production equipment cluster to form an instant physical status context.

[0076] Edge gateways are deployed in various production workshops or production lines. They are responsible for collecting data such as equipment status, production progress, and material consumption from various industrial PLCs, sensors, barcode readers, etc. After preliminary cleaning and aggregation, the data is sent to the upper computing node in real time / periodically through a private network to form an instant physical state context.

[0077] The sandbox simulation environment communicates with the host computing node and is used to perform full-process simulation verification of the updated production scheduling plan generated after model optimization in the host computing node.

[0078] A sandbox simulation environment can be a production process simulation system deployed on a host computing node or another dedicated server. It simulates the processes, equipment, and logistics rules of the real physical world. After receiving an optimization plan or a plan to be executed, it will quickly simulate the production execution process for hours, days, or even weeks in the virtual environment, evaluate its feasibility and performance, and return the simulation report to the host computing node as the basis for the final decision.

[0079] The electronic device provided in this application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the production scheduling strategy adaptive generation method described above. Those skilled in the art will understand that all or part of the steps of the various methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk or optical disk, etc.

[0080] The above description, in conjunction with specific embodiments, provides a further detailed explanation of this application and should not be construed as limiting the specific implementation of this application to these descriptions. Those skilled in the art to which this application pertains can make several simple deductions or substitutions without departing from the concept of this application.

Claims

1. A method for adaptively generating production scheduling strategies, characterized in that, include: Generate an initial production scheduling strategy and construct an initial scheduling resource time sequence map corresponding to the initial production scheduling strategy; In response to a manual adjustment operation on the initial scheduling resource timing map, the instantaneous physical state context of the target production unit is obtained within the time window in which the manual adjustment operation is triggered. The structural changes in the scheduling resource time sequence graph caused by the manual adjustment operation are encoded as scheduling edit event vectors; Based on the real-time physical state context, the scheduling and editing event vector, the pre-adjustment production schedule plan, the post-adjustment production schedule plan, and the actual execution result of the post-adjustment production schedule plan, a production schedule preference training sample is constructed, and the production schedule preference evaluation model is trained using the training sample. Using the production scheduling preference evaluation model as the optimization objective, a constrained reinforcement learning algorithm is employed to drive parameter optimization of the production scheduling strategy generation model. Based on the optimized production scheduling strategy generation model, an updated production scheduling strategy scheme is output.

2. The method as described in claim 1, characterized in that, The node types of the scheduling resource time sequence graph include at least one of the following: work order node, process node, equipment node, tooling node, material node, personnel node, time window node, and abnormal event node.

3. The method as described in claim 1 or 2, characterized in that, The types of edges in the scheduling resource time sequence graph include: process dependency edges representing the sequence of processes; equipment capacity edges representing the matching relationship between the processing capacity of processes and available equipment; resource occupation edges representing the occupation relationship between processes and materials, tooling, or personnel; mutual exclusion constraint edges representing that the same resource cannot be occupied repeatedly within the same time window; abnormal impact edges representing the impact of abnormal events on work orders, processes, or equipment; and time constraint edges representing delivery dates or maintenance windows.

4. The method as described in claim 1, characterized in that, The process of encoding the structural changes in the scheduling resource time sequence graph caused by the manual adjustment operation into a scheduling edit event vector specifically includes: Obtain the timing graphs of the scheduled resources before and after the adjustment, respectively; The time sequence graph of the scheduled resources before adjustment is converted into a first adjacency matrix or a first adjacency tensor, and the time sequence graph of the scheduled resources after adjustment is converted into a second adjacency matrix or a second adjacency tensor. Calculate the difference between the second adjacency matrix or the second adjacency tensor and the first adjacency matrix or the first adjacency tensor to obtain graph structure difference data; The scheduling and editing event vector is generated by fusing the graph structure difference data with the operational semantics of the manual adjustment operation.

5. The method as described in claim 1, characterized in that, The construction of production scheduling preference training samples also includes a sample quality control step: When the manual adjustment operation is performed by a user with low privileges, the real-time physical state context is missing, the actual execution result is abnormally missing, the adjusted production scheduling plan is not actually executed, or the adjusted production scheduling plan causes a significant deterioration in the preset key indicators, the corresponding candidate training data will be marked as low-confidence samples or removed from the training set.

6. The method as described in claim 1, characterized in that, The constraints of the constrained reinforcement learning algorithm include: A hard constraint penalty term is introduced to penalize the scheme output by the production scheduling strategy generation model for violating process timing constraints, equipment capacity constraints, material kitting constraints, or resource mutual exclusion constraints. Set a policy update magnitude limit to limit the policy distribution offset of a single model update; The optimized production scheduling strategy generation model is used to perform sandbox simulation verification. If the solution fails the simulation verification, the model version that generates the solution is prohibited from being released to the production scheduling system.

7. The method as described in claim 1, characterized in that, The optimized production scheduling strategy generation model outputs an updated production scheduling strategy scheme, including: The updated production scheduling strategy was input into the sandbox simulation environment for full-chain simulation verification. When the updated production scheduling strategy is verified, it will be submitted to the management terminal for manual confirmation. Upon receiving a manual confirmation instruction, the updated production scheduling strategy will be published to the production scheduling system or sent to the production terminal for execution.

8. A production scheduling strategy adaptive generation device, characterized in that, include: The strategy generation module is used to generate an initial production scheduling strategy scheme and construct an initial scheduling resource timing graph corresponding to the initial production scheduling strategy scheme; The context acquisition module is used to respond to manual adjustment operations on the initial scheduling resource timing map and acquire the real-time physical state context of the target production unit within the time window when the manual adjustment operation is triggered. The graph difference encoding module is used to encode the structural changes in the scheduling resource time sequence graph caused by the manual adjustment operation into a scheduling edit event vector; The model building module is used to construct scheduling preference training samples based on the real-time physical state context, the scheduling editing event vector, the pre-adjustment production scheduling plan, the post-adjustment production scheduling plan, and the actual execution results of the post-adjustment production scheduling plan, and to train the production scheduling preference evaluation model using the training samples. The model optimization module is used to optimize the parameters of the production scheduling strategy generation model by using the production scheduling preference evaluation model as the optimization objective and employing a constrained reinforcement learning algorithm. The solution output module is used to generate a model based on the optimized production scheduling strategy and output the updated production scheduling strategy solution.

9. A production scheduling strategy adaptive generation system, characterized in that, include: The upper-level computing node is used to execute the production scheduling strategy adaptive generation method as described in any one of claims 1 to 7; The management terminal is communicatively connected to the host computing node and is used to present the initial production scheduling strategy in a visual form of scheduling resource time sequence graph and capture the user's adjustment operations. The edge gateway communicates with the upper computing node and is used to collect real-time physical status data from the underlying production equipment cluster to form an instantaneous physical status context. The sandbox simulation environment is connected to the host computing node and is used to perform full-process simulation verification of the updated production scheduling scheme generated after model optimization in the host computing node.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the production scheduling strategy adaptive generation method as described in any one of claims 1 to 7.

Images