A method, apparatus, and system for local resolution of shop floor disturbances

Abstract

The application discloses a workshop disturbance local resolution method, device and system. Including based on the production process dimension of workshop, production equipment allocation dimension and logistics transportation dimension, construct three-dimensional analysis graph; based on the three-dimensional analysis graph, construct the time sequence knowledge graph; in response to the disturbance occurring in the workshop, based on the pre-set propagation reasoning rule, the time sequence knowledge graph is reasoned, and the local disturbed subgraph is determined; the local disturbed subgraph is input into the pre-trained deep reinforcement learning intelligent agent, and the workshop local rescheduling strategy is obtained, which is used for realizing the workshop disturbance local resolution. The application integrates the production process dimension, the production equipment allocation dimension and the logistics transportation dimension, and realizes the workshop disturbance local resolution in various disturbance scenes, and realizes the dual balance of high efficiency and high stability.

Description

Technical Field

[0001] This invention relates to a method, apparatus, and system for localizing disturbances in a workshop, belonging to the field of industrial intelligent simulation technology. Background Technology

[0002] In discrete manufacturing workshop environments, workshops face multi-dimensional random disturbances such as equipment failures and plan changes, which pose challenges to the stability and continuity of production processes. Equipment failures often induce plan interruptions, directly affecting delivery schedules; the placement of urgent orders significantly increases the complexity of scheduling decisions, weakening the certainty and controllability of production plans.

[0003] Traditional global rescheduling methods discard the original static scheduling plan when a disturbance occurs, treating all remaining tasks, available equipment, and material status in the workshop at the current moment as a new initial state for rescheduling. While this strategy can find a globally optimal solution, it has extremely high computational costs and significantly disrupts already executed plans, resulting in obvious limitations in actual production. Local rescheduling, on the other hand, focuses on precisely repairing only the subset of tasks directly affected by the disturbance while maintaining the stability of the original plan, achieving rapid recovery within a limited solution space.

[0004] However, existing research has the following shortcomings: First, in terms of information integration and representation, existing studies are mostly limited to a single resource perspective, lacking a systematic integration of the three-dimensional constraints of production process logic, equipment processing, and Automated Guided Vehicle (AGV) transportation, making it difficult to accurately characterize the deep relationships between production entities, dynamic events, and scheduling processes. Second, in terms of disturbance impact assessment, traditional methods rely on empirical rules, making it difficult to accurately define the minimum impact range of disturbances in complex coupled environments. Third, existing deep reinforcement learning scheduling methods are usually trained over long periods for specific environments, lacking the ability to quickly adapt to new disturbance scenarios. Therefore, it is necessary to construct a scheduling method that integrates production processes, equipment, and transportation dimensions to achieve local mitigation of workshop disturbances. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, and system for localizing workshop disturbances, which integrates the dimensions of production process, production equipment allocation, and logistics transportation to locally resolve workshop disturbances under various disturbance scenarios, achieving a dual balance between high efficiency and high stability.

[0006] To achieve the above objectives, the present invention is implemented using the following technical solution.

[0007] In a first aspect, the present invention provides a method for localizing workshop disturbances, comprising:

[0008] A three-dimensional parsing diagram is constructed based on the dimensions of production process, production equipment allocation, and logistics transportation in the workshop.

[0009] Constructing a temporal knowledge graph based on a 3D disjunction graph;

[0010] In response to disturbances occurring in the workshop, the temporal knowledge graph is inferred based on pre-set propagation inference rules to determine locally disturbed subgraphs;

[0011] The local disturbed subgraph is input into a pre-trained deep reinforcement learning agent to obtain a workshop local rescheduling strategy, which is used to achieve local resolution of workshop disturbances.

[0012] In conjunction with the first aspect, optionally, the expression for the three-dimensional disjunctive graph is: ;

[0013] In the formula, N is the set of all production process nodes; A is the set of directed arcs of adjacent production processes for the same workpiece; E is the set of undirected arcs of adjacent production processes that can be processed on the same production equipment; M is the set of production equipment in the workshop; and V is the set of available logistics and transportation equipment.

[0014] In conjunction with the first aspect, the propagation inference rules may optionally include state filtering rules based on time-domain truncation, static data propagation rules, and path-dependent propagation rules.

[0015] The state filtering rule based on time-domain truncation uses the time of disturbance occurrence in the workshop as the truncation benchmark to divide the production process graph nodes in the time-series knowledge graph into completed state graph nodes, interrupted state graph nodes, and pending state graph nodes.

[0016] The static data propagation rules include: in response to production equipment failure, taking the production process graph node affected by the production equipment failure in the time-series knowledge graph as the source point, identifying the alternative production equipment graph node used to replace the production equipment graph node with the failure, as well as the interrupted state graph node and the pending state graph node semantically associated with the alternative production equipment graph node, and marking them as graph nodes affected by the disturbance.

[0017] The static data propagation rule further includes: in response to the insertion of a production order, generating a new production process diagram node, semantically associating the new production process diagram node with a production equipment diagram node, the semantically associating production equipment diagram node is a candidate production equipment diagram node, and marking the new production process diagram node and the candidate production equipment diagram node as diagram nodes affected by disturbance;

[0018] The path-dependent propagation rule is used to identify and mark the termination graph nodes and the production process graph nodes affected by disturbances on the temporal knowledge graph; the termination graph nodes are used to absorb the disturbance effects borne by the production process graph nodes located after the interrupted state graph nodes on the production process path.

[0019] In conjunction with the first aspect, optionally, the method for dividing the completed state graph nodes is as follows: in response to the completion time of the production process node being less than or equal to the disturbance start time, the production process graph nodes mapped on the time-series knowledge graph are divided into completed state graph nodes.

[0020] The method for dividing the interruption state graph nodes is as follows: in response to a disturbance start time that is between the start time and end time of a production process node, and the disturbance involves that production process node, the production process graph nodes mapped on the time-series knowledge graph are divided into interruption state graph nodes.

[0021] The method for dividing the nodes of the state graph to be processed is as follows: in response to the start time of the production process node being greater than or equal to the start time of the disturbance, the production process graph node mapped on the time-series knowledge graph is divided into nodes of the state graph to be processed.

[0022] In conjunction with the first aspect, optionally, the path-dependent propagation rules include: propagation rules dependent on production process paths, propagation rules dependent on production and processing paths, and propagation rules dependent on logistics and transportation paths.

[0023] The propagation rule dependent on the production process path is as follows: in response to the interruption state graph node, along the production process path, identify the production process graph nodes affected by the disturbance after the interruption state graph node, until the termination graph node is identified and recorded.

[0024] The propagation rule that depends on the production and processing path is: mark the nodes in the pending state diagram affected by production equipment failure as nodes in the production process diagram affected by disturbance;

[0025] The propagation rule dependent on the logistics transportation path is as follows: in response to a failure of the logistics transportation equipment, all nodes in the pending state diagram are marked as production process diagram nodes affected by the disturbance; in response to the absence of a backup logistics transportation equipment, the interrupted state diagram nodes affected by the failed logistics transportation equipment are marked as production process diagram nodes affected by the disturbance.

[0026] In conjunction with the first aspect, optionally, the training method for the deep reinforcement learning agent includes:

[0027] A parameterized perturbation generator is constructed based on a temporal knowledge graph to generate a workshop simulation perturbation scenario. The temporal knowledge graph consists of an entity set C, a relation set R, and a fact set F. The entity set C includes workshop scheduling resources and production process objects. The relation set R includes the semantic association between workshop scheduling resources and production process objects. The fact set F is used to structure the workshop scheduling resources, production process objects, and the semantic association between workshop scheduling resources and production process objects in the form of triples.

[0028] Using triples in the fact set F as candidate actions, a local simulated disturbed subgraph is constructed based on the workshop simulation disturbance scenario as the environmental state. An infeasible candidate action is eliminated by introducing a dynamic masking mechanism, and a feasible decision space is constructed.

[0029] The local simulated perturbed subgraph is input into a pre-constructed deep reinforcement learning agent, and under the constraints of the feasible decision space, the action probability distribution and the workshop state value estimate are output.

[0030] In the perturbation scenario of the workshop, the parameters of the deep reinforcement learning agent are updated based on the action probability distribution and the workshop state value estimation.

[0031] In conjunction with the first aspect, optionally, the deep reinforcement learning agent includes a graph attention encoding module, a temporal memory module, and a policy and value output module;

[0032] The graph attention encoding module calculates directed attention coefficients based on graph nodes and neighboring graph nodes in the locally simulated perturbed subgraph, then calculates normalized attention coefficients based on the directed attention coefficients, updates graph nodes based on the normalized attention coefficients, obtains graph embedding features, and inputs them into the temporal memory module.

[0033] The temporal memory module outputs the hidden state of the current time step to the policy and value output module based on the received graph embedding features of the current time step, the hidden state of the previous time step, and the memory cell of the previous time step. In the initial decision time step, the hidden state of the previous time step and the memory cell of the previous time step are both initialized with a preset vector and are updated and saved in each subsequent decision time step.

[0034] The strategy and value output module constructs an Actor-Critic framework based on the received hidden state; the Actor network in the Actor-Critic framework outputs the action probability distribution based on the hidden state; the Critic network in the Actor-Critic decision network outputs the expected cumulative reward that can be obtained by executing the local rescheduling strategy of the workshop based on the hidden state, as an estimate of the workshop state value.

[0035] In conjunction with the first aspect, optionally, the formula for calculating the directed attention coefficient is: ;

[0036] In the formula, For graph nodes With neighborhood graph nodes The directed attention coefficient; To simulate graph nodes in a locally disturbed subgraph; For graph nodes Nodes in the neighborhood graph; For activation functions; A learnable weight vector; It is a linear transformation matrix; This indicates vector concatenation; For graph nodes The initial feature vector; For neighborhood graph nodes The initial feature vector;

[0037] The formula for calculating the normalized attention coefficient is as follows: ;

[0038] In the formula, For graph nodes With neighborhood graph nodes The normalized attention coefficient; It is an exponential function with the natural constant as its base; For graph nodes The first-order neighborhood node index; For graph nodes With graph nodes The unnormalized attention coefficients between them;

[0039] The expression for the graph embedding feature is: ;

[0040] In the formula, For graph nodes Graph embedding features; It is a non-linear activation function;

[0041] The expression for the hidden state is: ;

[0042] In the formula, for The hidden state of the time step; for Memory cells of time steps; It is a long short-term memory network; for The global state vector at each time step; for The hidden state of the time step; for Memory cells of time steps;

[0043] The formula for calculating the global state vector is: ;

[0044] In the formula, for The set of graph nodes in the disturbed subgraph is simulated at a time step.

[0045] In a second aspect, the present invention provides a workshop disturbance localization device, comprising:

[0046] The 3D parsing graph construction module is used to construct 3D parsing graphs based on the production process dimension, production equipment allocation dimension, and logistics transportation dimension of the workshop.

[0047] The temporal knowledge graph construction module is used to construct temporal knowledge graphs based on 3D disjunctive graphs.

[0048] The local disturbed subgraph determination module is used to determine the local disturbed subgraph by reasoning the temporal knowledge graph based on the pre-set propagation reasoning rules in response to disturbances occurring in the workshop.

[0049] The strategy generation module is used to input the local disturbed subgraph into a pre-trained deep reinforcement learning agent to obtain a workshop local rescheduling strategy, which is used to achieve local resolution of workshop disturbances.

[0050] Thirdly, the present invention provides a workshop disturbance localization system, including a storage medium and a processor;

[0051] The storage medium is used to store instructions;

[0052] The processor is configured to operate according to the instructions to implement the method described in any one of the first aspects.

[0053] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0054] This invention constructs a temporal knowledge graph based on a three-dimensional parsing graph, breaking through the limitations of a single resource perspective. It integrates three dimensions: production process sequence, production equipment, and logistics and transportation equipment, realizing the structured representation and associated storage of multi-dimensional information in the workshop, and providing underlying data support for accurately mining the potential impact path of disturbances.

[0055] Furthermore, we propose state filtering rules based on time-domain truncation, static data propagation rules, and path-dependent propagation rules. By defining three states of the production process graph nodes—completed state graph nodes, interrupted state graph nodes, and pending processing state graph nodes—we accurately define the local disturbed subgraphs of disturbances, effectively eliminate redundant nodes unrelated to disturbances, and transform the complex global rescheduling problem into a low-dimensional local disturbed subgraph repair problem, significantly reducing the decision-making scale.

[0056] Furthermore, a deep reinforcement learning agent based on a generative environment is proposed. A parameterized perturbation generator is established to simulate diverse perturbation scenarios. During training, based on the action probability distribution and the workshop state value estimation, a proximal policy optimization algorithm is used to update the parameters of the deep reinforcement learning agent, enabling the reinforcement learning agent to quickly respond to unseen perturbations. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0058] Figure 1 This is a schematic diagram of the overall process of a method for localizing workshop disturbances according to the present invention;

[0059] Figure 2 A schematic diagram of a three-dimensional parsing image provided in an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of a multi-layered knowledge architecture provided for embodiments of the present invention;

[0061] Figure 4 This is a schematic diagram of a time-series knowledge graph provided in an embodiment of the present invention;

[0062] Figure 5 A schematic diagram of the state of each production process node in the state filtering rule based on time domain truncation provided in an embodiment of the present invention;

[0063] Figure 6 This is a schematic diagram of the state changes of each graph node in the time-series knowledge graph after a disturbance occurs, provided in an embodiment of the present invention.

[0064] Figure 7 A schematic diagram of the framework for path-dependent propagation rules provided in an embodiment of the present invention;

[0065] Figure 8 This is a schematic diagram of the motion space design process provided in an embodiment of the present invention;

[0066] Figure 9 This is a schematic diagram of the deep reinforcement learning agent framework provided in an embodiment of the present invention;

[0067] Figure 10 This is a schematic diagram of the scheduling results of the EX72 case provided by the present invention. Detailed Implementation

[0068] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0069] Example 1

[0070] like Figure 1 As shown in the figure, this embodiment introduces a method for localizing workshop disturbances, including:

[0071] A three-dimensional parsing diagram is constructed based on the dimensions of production process, production equipment allocation, and logistics transportation in the workshop.

[0072] Constructing a temporal knowledge graph based on a 3D disjunction graph;

[0073] In response to disturbances occurring in the workshop, the temporal knowledge graph is inferred based on pre-set propagation inference rules to determine locally disturbed subgraphs;

[0074] The local disturbed subgraph is input into a pre-trained deep reinforcement learning agent to obtain a workshop local rescheduling strategy, which is used to achieve local resolution of workshop disturbances.

[0075] In one specific embodiment of the present invention, the expression of the three-dimensional disjunction map is: ;

[0076] In the formula, N is the set of all production process nodes; A is the set of directed arcs of adjacent production processes for the same workpiece; E is the set of undirected arcs of adjacent production processes that can be processed on the same production equipment; M is the set of production equipment in the workshop; and V is the set of available logistics and transportation equipment.

[0077] In one specific embodiment of the present invention, the propagation inference rules include state filtering rules based on time-domain truncation, static data propagation rules, and propagation rules based on path dependence;

[0078] The state filtering rule based on time-domain truncation uses the time of disturbance occurrence in the workshop as the truncation benchmark to divide the production process graph nodes in the time-series knowledge graph into completed state graph nodes, interrupted state graph nodes, and pending state graph nodes.

[0079] The static data propagation rules include: in response to production equipment failure, taking the production process graph node affected by the production equipment failure in the time-series knowledge graph as the source point, identifying the alternative production equipment graph node used to replace the production equipment graph node with the failure, as well as the interrupted state graph node and the pending state graph node semantically associated with the alternative production equipment graph node, and marking them as graph nodes affected by the disturbance.

[0080] The static data propagation rule further includes: in response to the insertion of a production order, generating a new production process diagram node, semantically associating the new production process diagram node with a production equipment diagram node, the semantically associating production equipment diagram node is a candidate production equipment diagram node, and marking the new production process diagram node and the candidate production equipment diagram node as diagram nodes affected by disturbance;

[0081] The path-dependent propagation rule is used to identify and mark the termination graph nodes and the production process graph nodes affected by disturbances on the temporal knowledge graph; the termination graph nodes are used to absorb the disturbance effects borne by the production process graph nodes located after the interrupted state graph nodes on the production process path.

[0082] In one specific embodiment of the present invention, the method for dividing the completed state graph nodes is as follows: in response to the completion time of the production process node being less than or equal to the disturbance start time, the production process graph nodes mapped on the time-series knowledge graph are divided into completed state graph nodes.

[0083] The method for dividing the interruption state graph nodes is as follows: in response to a disturbance start time that is between the start time and end time of a production process node, and the disturbance involves that production process node, the production process graph nodes mapped on the time-series knowledge graph are divided into interruption state graph nodes.

[0084] The method for dividing the nodes of the state graph to be processed is as follows: in response to the start time of the production process node being greater than or equal to the start time of the disturbance, the production process graph node mapped on the time-series knowledge graph is divided into nodes of the state graph to be processed.

[0085] In one specific embodiment of the present invention, the path-dependent propagation rules include: propagation rules dependent on production process paths, propagation rules dependent on production processing paths, and propagation rules dependent on logistics and transportation paths.

[0086] The propagation rule dependent on the production process path is as follows: in response to the interruption state graph node, along the production process path, identify the production process graph nodes affected by the disturbance after the interruption state graph node, until the termination graph node is identified and recorded.

[0087] The propagation rule that depends on the production and processing path is: mark the nodes in the pending state diagram affected by production equipment failure as nodes in the production process diagram affected by disturbance;

[0088] The propagation rule dependent on the logistics transportation path is as follows: in response to a failure of the logistics transportation equipment, all nodes in the pending state diagram are marked as production process diagram nodes affected by the disturbance; in response to the absence of a backup logistics transportation equipment, the interrupted state diagram nodes affected by the failed logistics transportation equipment are marked as production process diagram nodes affected by the disturbance.

[0089] In one specific embodiment of the present invention, the training method of the deep reinforcement learning agent includes:

[0090] 1) Construct a parameterized disturbance generator based on a time-series knowledge graph to generate simulated workshop disturbance scenarios; in the specific implementation process, dynamically generate two types of simulated workshop disturbance scenarios, namely equipment failure or order insertion, to form a diversified training task distribution, so as to prevent the agent from overfitting to specific failure modes.

[0091] The temporal knowledge graph consists of an entity set C, a relation set R, and a fact set F. The entity set C includes shop floor scheduling resources and production process objects. The relation set R includes the semantic associations between shop floor scheduling resources and production process objects. The fact set F is used to structure the representation of shop floor scheduling resources, production process objects, and the semantic associations between shop floor scheduling resources and production process objects in the form of triples. In specific implementation, the relation set R of the temporal knowledge graph is divided into three knowledge layers: static data layer relations, temporal data layer relations, and critical path layer relations. The static data layer relations include the has relation describing that an entity has processes, the can pro relation describing that a device can process processes, the before / after relation describing the sequence of processes, the on relation describing the location of an entity, and the has attr relation describing entity attributes. The temporal data layer relations include the pro on relation describing specific processing information, the pro before / after relation describing the processing sequence, and the trans X relation describing the transportation sequence. The critical path layer relations are the key relations describing the path determined by the maximum completion time.

[0092] 2) Using triples in the fact set F as candidate actions, a local simulated disturbed subgraph is constructed based on the workshop simulation disturbance scenario as the environmental state. Infeasible candidate actions are eliminated by introducing a dynamic masking mechanism, and a feasible decision space is constructed.

[0093] 3) Input the local simulated disturbed subgraph into the pre-constructed deep reinforcement learning agent, and under the constraints of the feasible decision space, output the action probability distribution and the workshop state value estimate;

[0094] 4) In the perturbation scenario of the workshop, based on the action probability distribution and workshop state value estimation, the near-end policy optimization algorithm is used to update the parameters of the deep reinforcement learning agent.

[0095] In one specific embodiment of the present invention, the deep reinforcement learning agent includes a graph attention encoding module, a temporal memory module, and a policy and value output module;

[0096] The graph attention encoding module calculates directed attention coefficients based on graph nodes and neighboring graph nodes in the locally simulated perturbed subgraph, then calculates normalized attention coefficients based on the directed attention coefficients, updates graph nodes based on the normalized attention coefficients, obtains graph embedding features, and inputs them into the temporal memory module.

[0097] The temporal memory module outputs the hidden state of the current time step to the policy and value output module based on the received graph embedding features of the current time step, the hidden state of the previous time step, and the memory cell of the previous time step. In the initial decision time step, the hidden state of the previous time step and the memory cell of the previous time step are both initialized with a preset vector and are updated and saved in each subsequent decision time step.

[0098] The strategy and value output module constructs an Actor-Critic framework based on the received hidden state; the Actor network in the Actor-Critic framework outputs the action probability distribution based on the hidden state; the Critic network in the Actor-Critic decision network outputs the expected cumulative reward that can be obtained by executing the local rescheduling strategy of the workshop based on the hidden state, as an estimate of the workshop state value.

[0099] In one specific embodiment of the present invention, the formula for calculating the directed attention coefficient is as follows: ;

[0100] In the formula, For graph nodes With neighborhood graph nodes The directed attention coefficient; To simulate graph nodes in a locally disturbed subgraph; For graph nodes Nodes in the neighborhood graph; For activation functions; A learnable weight vector; It is a linear transformation matrix; This indicates vector concatenation; For graph nodes The initial feature vector; For neighborhood graph nodes The initial feature vector;

[0101] The formula for calculating the normalized attention coefficient is as follows: ;

[0102] In the formula, For graph nodes With neighborhood graph nodes The normalized attention coefficient; It is an exponential function with the natural constant as its base; For graph nodes The first-order neighborhood node index; For graph nodes With graph nodes The unnormalized attention coefficients between them;

[0103] The expression for the graph embedding feature is: ;

[0104] In the formula, For graph nodes Graph embedding features; It is a non-linear activation function;

[0105] The expression for the hidden state is: ;

[0106] In the formula, for The hidden state of the time step; for Memory cells of time steps; It is a long short-term memory network; for The global state vector at each time step; for The hidden state of the time step; for Memory cells of time steps;

[0107] The formula for calculating the global state vector is: ;

[0108] In the formula, for The set of graph nodes in the disturbed subgraph is simulated at a time step.

[0109] The following uses a flexible work workshop as an example to illustrate the method for localizing workshop disturbance resolution in this invention, with a specific implementation method as an example. Figure 1-9 As shown, the technical solution of the present invention includes the following steps:

[0110] S1: Construct a temporal knowledge graph based on a three-dimensional disjunctive graph; the three-dimensional disjunctive graph integrates three dimensions: production process dimension, production equipment allocation dimension, and AGV transportation dimension (i.e., logistics transportation dimension), to fully depict the three core constraints of production process logic, production equipment allocation, and material transportation in the established scheduling process of the flexible operation workshop;

[0111] Since the established scheduling process in a flexible workshop includes three dimensions—production process dimension, production equipment allocation dimension, and AGV transportation dimension—the 3D disjunctive graph can be obtained using a set. The formula represents the set of all production process nodes; A is the set of directed arcs representing adjacent production processes for the same workpiece; E is the set of undirected arcs representing adjacent production processes that can be processed on the same production equipment; M is the set of production equipment in the workshop; and V is the set of available AGVs. For example... Figure 2 As shown, adjacent production processes for the same workpiece within the production process dimension are connected by solid directed arcs; adjacent production processes that may be processed on the same equipment within the equipment dimension are connected by dashed lines; and the transportation dimension uses dashed lines to represent the AGV handling sequence.

[0112] S2: Constructing a multi-layered knowledge framework based on a 3D disjunctive graph, such as Figure 3 As shown, the multi-layered knowledge framework consists of three parts: static data describing the inherent properties of the local disturbance resolution system in the workshop, time-series data reflecting the scheduling execution process, and critical path data. Together, these three constitute the knowledge system for the local rescheduling problem in flexible work workshops.

[0113] S3: Based on the above multi-layered knowledge framework, construct a temporal knowledge graph, wherein the temporal knowledge graph is defined as follows: Where C is the entity set, R is the relation set, and F is the fact set, the semantic relation definitions of the temporal knowledge graph are shown in [Table 1], and the schematic diagram of the temporal knowledge graph is shown in [Table 1]. Figure 4 As shown.

[0114]

[0115] S4: Based on the propagation inference rules, a local disturbed subgraph is determined. When a disturbance event occurs, disturbance identification and localization are performed first: For equipment failure, the disturbance is mapped to the corresponding production equipment graph node in the temporal knowledge graph. The fault time and duration are determined by combining real-time status data, and the semantic relationships such as can pro, pro on, and pro before / after are located. For order insertion, corresponding workpiece graph nodes and production process graph nodes are created in the temporal knowledge graph. The process sequence relationship (before / after) is constructed, and the new production process is mapped to the resource candidate set through the can pro relationship. The resource candidate set is the set of optional production equipment for a certain production process retrieved based on the can pro relationship of the temporal knowledge graph.

[0116] The propagation inference rules include state filtering rules based on time-domain truncation, static data propagation rules, and path-dependency-based propagation rules. The specific usage methods are as follows:

[0117] 1) State filtering based on temporal truncation: Using the time of disturbance occurrence as the truncation point, the temporal state attributes of each production process node mapped to in the temporal knowledge graph are divided into three categories: completed state (process completed), in-process state (processing-locked / processing-interrupted), and pending state (the main recipient of disturbance propagation). Figure 5 As shown. The processing status needs to be determined to determine whether it involves disturbed resources; if so, it is marked as an interrupted state.

[0118] The workshop disturbance local resolution system implements state locking (freezing) on ​​completed state graph nodes, transforming the determined physical facts into rigid historical constraints for rescheduling; it triggers reprocessing and cascading propagation for interrupted state graph nodes; and it achieves local disturbance resolution for unprocessed state graph nodes by updating timing parameters and resource mappings. The state changes of each node in the timing knowledge graph before and after the disturbance are illustrated below. Figure 6 As shown.

[0119] 2) Static data propagation: It does not depend on the current specific scheduling process, but is based on the inherent constraints of workshop physical attributes, process logic and production equipment capabilities to determine the potential alternative resources of the disturbed nodes and map them as graph nodes in the time-series knowledge graph.

[0120] When production equipment malfunctions, the disrupted production process cannot be executed at the original node. In this case, the workshop disturbance local resolution system uses the disrupted production process graph node as the source point and searches the temporal knowledge graph along its association relationships (i.e., the semantic relationships of graph node quality inspection) to identify all production equipment graph nodes with equivalent processing capabilities as alternative production equipment graph nodes. These retrieved alternative production equipment graph nodes and their current production process task queues will be included in the disturbance scope. For urgently inserted production orders, the workshop disturbance local resolution system first generates new production process graph nodes, and then, based on static processing capability data, uses its association relationships to map the new production process graph nodes to the set of all feasible candidate production equipment graph nodes. At this time, the disturbance scope is defined as all production process graph nodes and all candidate production equipment graph nodes of the new production order. The static processing capability data is inherent workshop attribute data, including the processes that the equipment can process and processing time, etc., derived from process planning.

[0121] 3) Path-dependent propagation: The propagation path includes the production process path, the production and processing path, and the AGV transportation path. The propagation framework is as follows: Figure 7 As shown.

[0122] Production process path propagation: Check the immediate production process diagram nodes along the after relationship. If the delay of the preceding diagram node causes the floating time of the next production process diagram node to be negative, then the next production process diagram node is included in the scope of influence and the recursion continues. If the floating time of a diagram node is sufficient to absorb the preceding delay, then the propagation terminates. This diagram node is the cutoff boundary for local rescheduling on the process chain of this workpiece, that is, the termination diagram node.

[0123] Production path propagation: For faulty equipment and its replacement equipment, production process nodes whose original planned start time is after the disturbance and which have resource conflicts are included in the scope of disturbance; only when a new production process is forcibly inserted and there is insufficient idle time will the production process node that was moved to the later stage be included in the scope of disturbance.

[0124] AGV transport path propagation: When equipment malfunctions, unexecuted transport tasks by AGVs can be reassigned, and all AGV nodes in the workshop are included in the affected range; when inserting an order, the corresponding process is only inserted when there is sufficient idle time, otherwise it is arranged at the end of the equipment task queue.

[0125] S5: Based on the perturbation propagation inference rules, a deep reinforcement learning agent based on generative environment is used to make local scheduling decisions for the locally perturbed subgraph.

[0126] 1) State-space design: At any time step t, the state... Defined as a property graph ,in For the entity nodes within the current local area of ​​influence, The semantic relationships between nodes in the entity graph. This is the graph node feature matrix; the entity nodes refer to the graph nodes of workshop scheduling resources and production process objects. The specific feature extraction dimensions are as follows:

[0127] Features of production process diagram nodes:

[0128] Its feature vectors encompass both physical and dynamic characteristics. The attribute features are constructed based on static workshop data and include the average processing time for each production step in the attribute graph. The dynamic features are used to capture the real-time state under disturbed environments, specifically employing one-hot encoding. The status of each node in the production process diagram is described; a time deviation index is also introduced. The time deviation index is defined as the difference between the current estimated start time and the original planned start time. Based on this, the critical path identifier is used. Indicates whether the production process is on the critical path, helping deep reinforcement learning agents identify key graph nodes; production process graph nodes. eigenvectors Defined as ;

[0129] Production equipment diagram node features:

[0130] The features of production equipment nodes include availability and processing capacity dimensions. In the availability dimension, the feature vector includes the time required to recover to an idle state from the current moment. For production equipment failure scenarios, the estimated repair time of the failed production equipment must be specifically marked. In terms of processing capacity, the feature vector contains the average processing time of the production processes that the production equipment can currently process. For AGV graph nodes, the features include their location codes. This is to support accurate rescheduling of transportation tasks. Production equipment diagram nodes. The feature vector is defined as Similarly, AGV graph nodes The feature vector is defined as ;

[0131] Global context graph node features:

[0132] Introducing a global feature vector enables deep reinforcement learning agents to more effectively acquire overall production information from the workshop. The global feature vector includes a perturbation strength index. The disturbance intensity index is calculated by dividing the total number of nodes in the production process diagram to the total number of nodes in the production equipment diagram in the local disturbed subgraph, thereby quantifying the disturbance intensity at the current moment; the global feature vector also incorporates the global completion rate. This refers to the proportion of completed production processes to the total number of production processes in the order, thus providing the deep reinforcement learning agent with macroscopic feedback on the overall production progress; global context feature vector. Defined as .

[0133] 2) Action Space Design: A joint action space generation mechanism based on temporal knowledge graph association retrieval is proposed. This mechanism utilizes the structured prior knowledge contained in the temporal knowledge graph to construct the action space and introduces dynamic masking technology to eliminate invalid actions. This significantly improves the flexibility of the scheduling strategy and the solution space coverage while ensuring the feasibility of the decision. The specific process is as follows: Figure 8 As shown.

[0134] Time step Actions during Defined as a triple .in, This represents the production process to be scheduled in the current locally disturbed subgraph. For the target processing equipment, AGVs are used to perform material handling tasks in the production process. Their motion space is dynamically generated directly based on semantic relationships in a temporal knowledge graph, traversing the production processes to be processed within the perturbed subgraph. Based on the predefined CAN-PRO relationship in the time-series knowledge graph, retrieve graph nodes of production equipment with processing capabilities. This forms the basic binary tuples. The current set of AGVs in the workshop is added to each binary tuple for expansion, generating the overall motion space. .

[0135] Building upon the action space, a dynamic masking mechanism is introduced to handle complex constraints under perturbed environments. This mechanism eliminates invalid actions from two dimensions: process logic and resource state. Regarding process constraints, each production process graph node is evaluated along the "before" relationship in the temporal knowledge graph. If the preceding graph node of the production process to be scheduled is in a frozen or waiting state, actions related to that process are masked. Regarding resource state, actions associated with faulty equipment at that moment are masked. Through this process, the deep reinforcement learning agent can effectively avoid infeasible solutions at the current moment, ensuring the executability of the scheduling strategy.

[0136] 3) Network framework design: Deep reinforcement learning agents adopt an encoder-decoder architecture, such as... Figure 9 As shown, the system includes a feature embedding layer, a graph attention encoding module, a temporal memory module, and a policy and value output module. The graph attention encoding module uses a graph attention network (GAT) to extract features from locally perturbed subgraphs; the temporal memory module uses a long short-term memory (LSTM) network to capture the temporal dependencies of the scheduling process; and the policy and value output module generates action probability distributions and workshop state value estimates based on the Actor-Critic framework, with the GAT calculating graph nodes. With neighborhood graph nodes The directed attention coefficients are obtained. The above technical solution uses GAT to extract features of heterogeneous nodes, combines LSTM to capture the temporal dependencies of scheduling decisions, and introduces an action masking mechanism to ensure the feasibility of the rescheduling strategy, enabling the reinforcement learning agent to quickly respond to unseen disturbances.

[0137] To ensure comparability of directed attention coefficients between different nodes, the Softmax function is used. Normalization is performed to obtain the normalized attention coefficients. The graph nodes can be calculated using the normalized attention coefficients. Graph embedding features .

[0138] In the temporal memory module, since perturbation is resolved into sequential decisions, the current scheduling action significantly affects the availability of future resources. Therefore, to ensure the effectiveness of scheduling, an LSTM is introduced to record the specific order of the scheduling process. The LSTM unit of the temporal memory module receives the current graph embedding features output from the graph attention encoding module, as well as the hidden state from the previous time step. and memory cells Output the hidden state at the current time step. Since the output of GAT is an independent feature vector for each node, to meet the input dimension requirements of LSTM, an average pooling strategy is used to generate the global state vector at time step t. The formula is:

[0139] ;

[0140] In the formula, for The set of graph nodes in the disturbed subgraph is simulated at a time step.

[0141] global state vector The timing is updated by inputting a standard LSTM cell. The LSTM uses its gating mechanism to dynamically adjust the information flow while preserving key historical scheduling context information. The update process formula is as follows:

[0142] ;

[0143] In the formula, for The hidden state of the time step; for Memory cells of time steps; It is a long short-term memory network; for The global state vector at each time step; for The hidden state of the time step; for Memory cells of time steps;

[0144] After temporal evolution, the output hidden state As a high-dimensional feature vector that integrates current workshop characteristics and historical decision sequences, it is fed into subsequent modules for decision generation.

[0145] In the strategy and value output module, the hidden state is based on the output of the temporal memory module. A policy gradient algorithm based on Actor-Critic is constructed, in which the Actor network is used to generate the action probability distribution and the Critic network is used to generate the workshop state value estimate.

[0146] Both the Actor network and the Critic network employ a Multi-Layer Perceptron (MLP) structure. The Actor network uses the hidden state output by the LSTM. The input is a fully connected layer, which generates normalized scores for all candidate actions through linear transformation and activation functions. Based on the masking results, the scores of invalid actions are forced to negative infinity. Finally, the action scores are transformed into action probability distributions using the Softmax function. The deep reinforcement learning agent samples actions based on this distribution. The Critic network also receives hidden states. As input, a scalar is output after passing through a fully connected layer. , This represents the expected cumulative return that can be obtained by implementing the local rescheduling strategy in the current state, i.e., the value estimate of the shop state.

[0147] 4) Training process of deep reinforcement learning agent: The perturbation generator randomly samples target resources and time windows from the temporal knowledge graph to dynamically generate diverse workshop simulation perturbation scenarios.

[0148] The network parameters of the deep reinforcement learning agent are updated using a proximal policy optimization algorithm. Workshop state value estimation is performed based on the output of the Critic network. The advantage function is calculated using generalized advantage estimation (GAE). To balance bias and variance. The dominance function. The calculation formula is:

[0149] ;

[0150] in, for Temporal difference of time steps; Discount factor; is the GAE smoothing factor; T is the trajectory length.

[0151] in, ;

[0152] In the formula, To enable deep reinforcement learning agents to perform time steps The instant reward received;

[0153] in, ; ; ;

[0154] In the formula, Bonus for timely completion; Rewards will be given for the degree of conformity of the proposed solutions; and These are the weighting coefficients; For state The completion time is as follows; For the set of disrupted processes; For process New distribution equipment; For process The old distribution equipment; For the characteristic function, if the process If the new allocation device is inconsistent with the old allocation device, the function value is 1; otherwise, it is 0.

[0155] For Actor networks, to limit the difference between the old and new policies, a truncation mechanism is introduced to restrict the difference. The policy loss function of the Actor network... The construction is as follows:

[0156] ;

[0157] In the formula, The symbol for expectation indicates the expectation over time step t; Estimated based on experience: The ratio of the old to the new strategies; This is the clipping function; To truncate hyperparameters, used to control the magnitude of policy updates within a certain range. The interval is used to avoid large fluctuations in the training process due to excessively large update increments;

[0158] in, ;

[0159] In the formula, The updated network policy probability; The network probabilities are the old policy probabilities before the update; For time steps Actions at that time; for The state of the time step.

[0160] For the Critic network, the objective is to minimize the mean squared error between the estimated state value and the true return, and the value loss function is... Defined as:

[0161] ;

[0162] In the formula, The target return for time step t.

[0163] Taking the aforementioned flexible work workshop as an example, a verification experiment on the local elimination of workshop disturbances was conducted, and the results are as follows:

[0164] The experiment was conducted on a Windows 11 operating system. The simulation environment was built using the Python 3.11 programming language and the PyTorch deep learning framework. The hardware configuration consisted of an Intel Core i7-13790 processor, 32GB of RAM, and an NVIDIA GeForce RTX 4070 graphics card. The specific hyperparameter settings of the algorithm are shown in [Table 2].

[0165]

[0166] Convergence analysis was conducted on 10 different cases of the standard test set for the Flexible Job Shop Scheduling Problem with Transportation (FJSPT). Random mixed perturbations were introduced through a generative environment, and the overall loss function's trend was recorded. The results show that despite the varying sizes and complexities of the cases, their convergence process exhibits high consistency: the loss decreases rapidly in the early stages of training, oscillates slightly in the middle stages, and finally stabilizes in a low value range for each case. The truncation mechanism effectively controls the policy update magnitude, demonstrating that the local resolution method for job shop perturbations provided in this invention possesses excellent robustness and convergence capability in diverse scheduling problems.

[0167] Comparative experiments were conducted on a classic benchmark dataset of flexible workshops. By randomly inserting two types of mixed disturbances—equipment failure and order insertion—during the execution of a predetermined optimal scheduling scheme, the workshop disturbance local resolution method provided by this invention was compared with traditional scheduling rules and deep reinforcement learning algorithms.

[0168] In [Table 3], each row corresponds to a test case, and each column corresponds to a scheduling algorithm, including CDR1, CDR2, CDR3, and DDQN.

[20] This algorithm is the workshop disturbance local resolution method provided by this invention; seed is a random number seed, and the values ​​in [Table 3] are the scheduling performance evaluation indicators (i.e., objective function values) obtained by each scheduling algorithm under the corresponding test cases. The numbers in parentheses are the number of production process changes before and after workshop disturbance resolution. In cases such as EX71 and EX74, this algorithm achieved zero change in production processes, that is, it completed the absorption and optimization of disturbances without changing the original processing sequence, and successfully achieved a dual balance of high efficiency and high stability. For scheduling problems with completion time as the optimization objective, the scheduling performance evaluation index represents the total time required to complete the processing of all workpieces. The smaller the value, the higher the processing efficiency. The experimental results are shown in [Table 3]. The workshop disturbance local resolution method provided by this invention performs well in both processing efficiency and scheduling stability. The scheduling results of case EX72 in [Table 3] are as follows. Figure 10 As shown.

[0169] exist Figure 10 In the diagram, M1~M4 refer to production equipment 1~4; J1~J10 refer to workpieces 1~10. Some workpieces will utilize multiple production equipment to complete their required production processes; the color blocks on the vertical axis, AGV0 and AGV1, represent automated guided vehicles (AGVs). After a workpiece is processed on one production equipment, the AGV will transport it to the next production equipment. Figure 10 At approximately 20 seconds, an emergency order was inserted in response to the workshop. The arrival of the inserted workpiece caused a disturbance (shown by the red dashed line). The deep reinforcement learning agent locally adjusted the AGV transport and equipment processing sequence. At approximately 30 seconds, equipment M1 malfunctioned (shown by the alternating black and red FAULT area). The affected production process was identified as a locally disturbed subgraph. Under the decision-making of the deep reinforcement learning agent, some production processes were reassigned to alternative equipment such as M2 and M4, or postponed until processing was completed after the M1 malfunction was repaired. Simultaneously, AGV0 and AGV1 collaboratively updated their transport plans. This result directly verifies the invention's ability to locally resolve disturbances while maintaining production efficiency and scheduling stability under multidimensional constraints and complex perturbations.

[0170]

[0171] Example 2

[0172] Based on the same inventive concept as Embodiment 1, this embodiment introduces a workshop disturbance localization device, comprising:

[0173] The 3D parsing graph construction module is used to construct 3D parsing graphs based on the production process dimension, production equipment allocation dimension, and logistics transportation dimension of the workshop.

[0174] The temporal knowledge graph construction module is used to construct temporal knowledge graphs based on 3D disjunctive graphs.

[0175] The local disturbed subgraph determination module is used to determine the local disturbed subgraph by reasoning the temporal knowledge graph based on the pre-set propagation reasoning rules in response to disturbances occurring in the workshop.

[0176] The strategy generation module is used to input the local disturbed subgraph into a pre-trained deep reinforcement learning agent to obtain a workshop local rescheduling strategy, which is used to achieve local resolution of workshop disturbances.

[0177] Example 3

[0178] This invention provides a workshop disturbance localization system, including a storage medium and a processor;

[0179] The storage medium is used to store instructions;

[0180] The processor is configured to operate according to the instructions to implement the method described in any one of Embodiment 1.

[0181] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code, including but not limited to disk storage, CD-ROM, optical storage, etc.

[0182] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, in-vehicle equipment systems, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing in-vehicle equipment to produce a device, such that the instructions, which execute via the processor of the computer or other programmable data processing in-vehicle equipment, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0183] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing vehicle-mounted equipment to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing onboard equipment, causing a series of operational steps to be performed on the computer or other programmable onboard equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable onboard equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0185] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for localizing workshop disturbances, characterized in that, include: A three-dimensional parsing diagram is constructed based on the dimensions of production process, production equipment allocation, and logistics transportation in the workshop. Constructing a temporal knowledge graph based on a 3D disjunction graph; In response to disturbances occurring in the workshop, the temporal knowledge graph is inferred based on pre-set propagation inference rules to determine locally disturbed subgraphs; The local disturbed subgraph is input into a pre-trained deep reinforcement learning agent to obtain a workshop local rescheduling strategy, which is used to achieve local resolution of workshop disturbances.

2. The method for localizing workshop disturbance resolution according to claim 1, characterized in that, The expression for the three-dimensional disjunction map is: ; In the formula, N is the set of all production process nodes; A is the set of directed arcs of adjacent production processes for the same workpiece; E is the set of undirected arcs of adjacent production processes that can be processed on the same production equipment; M is the set of production equipment in the workshop; and V is the set of available logistics and transportation equipment.

3. The method for localizing workshop disturbance resolution according to claim 1, characterized in that, The propagation inference rules include state filtering rules based on time-domain truncation, static data propagation rules, and path-dependent propagation rules. The state filtering rule based on time-domain truncation uses the time of disturbance occurrence in the workshop as the truncation benchmark to divide the production process graph nodes in the time-series knowledge graph into completed state graph nodes, interrupted state graph nodes, and pending state graph nodes. The static data propagation rules include: in response to production equipment failure, taking the production process graph node affected by the production equipment failure in the time-series knowledge graph as the source point, identifying the alternative production equipment graph node used to replace the production equipment graph node with the failure, as well as the interrupted state graph node and the pending state graph node semantically associated with the alternative production equipment graph node, and marking them as graph nodes affected by the disturbance. The static data propagation rule further includes: in response to the insertion of a production order, generating a new production process diagram node, semantically associating the new production process diagram node with a production equipment diagram node, the semantically associating production equipment diagram node is a candidate production equipment diagram node, and marking the new production process diagram node and the candidate production equipment diagram node as diagram nodes affected by disturbance; The path-dependent propagation rule is used to identify and mark the termination graph nodes and the production process graph nodes affected by disturbances on the temporal knowledge graph; the termination graph nodes are used to absorb the disturbance effects borne by the production process graph nodes located after the interrupted state graph nodes on the production process path.

4. The method for localizing workshop disturbance resolution according to claim 3, characterized in that, The method for dividing the completed state graph nodes is as follows: in response to the completion time of the production process node being less than or equal to the disturbance start time, the production process graph nodes mapped on the time-series knowledge graph are divided into completed state graph nodes. The method for dividing the interruption state graph nodes is as follows: in response to a disturbance start time that is between the start time and end time of a production process node, and the disturbance involves that production process node, the production process graph nodes mapped on the time-series knowledge graph are divided into interruption state graph nodes. The method for dividing the nodes of the state graph to be processed is as follows: in response to the start time of the production process node being greater than or equal to the start time of the disturbance, the production process graph node mapped on the time-series knowledge graph is divided into nodes of the state graph to be processed.

5. The method for localizing workshop disturbance resolution according to claim 3, characterized in that, The path-dependent propagation rules include: propagation rules dependent on production process paths, propagation rules dependent on production and processing paths, and propagation rules dependent on logistics and transportation paths. The propagation rule dependent on the production process path is as follows: in response to the interruption state graph node, along the production process path, identify the production process graph nodes affected by the disturbance after the interruption state graph node, until the termination graph node is identified and recorded. The propagation rule that depends on the production and processing path is: mark the nodes in the pending state diagram affected by production equipment failure as nodes in the production process diagram affected by disturbance; The propagation rule dependent on the logistics transportation path is as follows: in response to a failure of the logistics transportation equipment, all nodes in the pending state diagram are marked as production process diagram nodes affected by the disturbance; in response to the absence of a backup logistics transportation equipment, the interrupted state diagram nodes affected by the failed logistics transportation equipment are marked as production process diagram nodes affected by the disturbance.

6. The method for localizing workshop disturbance resolution according to claim 1, characterized in that, The training method for the deep reinforcement learning agent includes: A parameterized perturbation generator is constructed based on a temporal knowledge graph to generate a workshop simulation perturbation scenario. The temporal knowledge graph consists of an entity set C, a relation set R, and a fact set F. The entity set C includes workshop scheduling resources and production process objects. The relation set R includes the semantic association between workshop scheduling resources and production process objects. The fact set F is used to structure the workshop scheduling resources, production process objects, and the semantic association between workshop scheduling resources and production process objects in the form of triples. Using triples in the fact set F as candidate actions, a local simulated disturbed subgraph is constructed based on the workshop simulation disturbance scenario as the environmental state. An infeasible candidate action is eliminated by introducing a dynamic masking mechanism, and a feasible decision space is constructed. The local simulated perturbed subgraph is input into a pre-constructed deep reinforcement learning agent, and under the constraints of the feasible decision space, the action probability distribution and the workshop state value estimate are output. In the perturbation scenario of the workshop, the parameters of the deep reinforcement learning agent are updated based on the action probability distribution and the workshop state value estimation.

7. The method for localizing workshop disturbance resolution according to claim 6, characterized in that, The deep reinforcement learning agent includes a graph attention encoding module, a temporal memory module, and a policy and value output module. The graph attention encoding module calculates directed attention coefficients based on graph nodes and neighboring graph nodes in the locally simulated perturbed subgraph, then calculates normalized attention coefficients based on the directed attention coefficients, updates graph nodes based on the normalized attention coefficients, obtains graph embedding features, and inputs them into the temporal memory module. The temporal memory module outputs the hidden state of the current time step to the policy and value output module based on the received graph embedding features of the current time step, the hidden state of the previous time step, and the memory cell of the previous time step. In the initial decision time step, the hidden state of the previous time step and the memory cell of the previous time step are both initialized with a preset vector and are updated and saved in each subsequent decision time step. The strategy and value output module constructs an Actor-Critic framework based on the received hidden state; the Actor network in the Actor-Critic framework outputs the action probability distribution based on the hidden state; the Critic network in the Actor-Critic decision network outputs the expected cumulative reward that can be obtained by executing the local rescheduling strategy of the workshop based on the hidden state, as an estimate of the workshop state value.

8. The method for localizing workshop disturbance resolution according to claim 7, characterized in that, The formula for calculating the directed attention coefficient is as follows: ; In the formula, For graph nodes With neighborhood graph nodes The directed attention coefficient; To simulate graph nodes in a locally disturbed subgraph; For graph nodes Nodes in the neighborhood graph; For activation functions; A learnable weight vector; It is a linear transformation matrix; This indicates vector concatenation; For graph nodes The initial feature vector; For neighborhood graph nodes The initial feature vector; The formula for calculating the normalized attention coefficient is as follows: ; In the formula, For graph nodes With neighborhood graph nodes The normalized attention coefficient; It is an exponential function with the natural constant as its base; For graph nodes The first-order neighborhood node index; For graph nodes With graph nodes The unnormalized attention coefficients between them; The expression for the graph embedding feature is: ; In the formula, For graph nodes Graph embedding features; It is a non-linear activation function; The expression for the hidden state is: ; In the formula, for The hidden state of the time step; for Memory cells of time steps; It is a long short-term memory network; for The global state vector at each time step; for The hidden state of the time step; for Memory cells of time steps; The formula for calculating the global state vector is: ; In the formula, for The set of graph nodes in the disturbed subgraph is simulated at a time step.

9. A device for locally eliminating workshop disturbances, characterized in that, include: The 3D parsing graph construction module is used to construct 3D parsing graphs based on the production process dimension, production equipment allocation dimension, and logistics transportation dimension of the workshop. The temporal knowledge graph construction module is used to construct temporal knowledge graphs based on 3D disjunctive graphs. The local disturbed subgraph determination module is used to determine the local disturbed subgraph by reasoning the temporal knowledge graph based on the pre-set propagation reasoning rules in response to disturbances occurring in the workshop. The strategy generation module is used to input the local disturbed subgraph into a pre-trained deep reinforcement learning agent to obtain a workshop local rescheduling strategy, which is used to achieve local resolution of workshop disturbances.

10. A workshop disturbance localization system, characterized in that, Including storage media and processor; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to implement the method of any one of claims 1-8.

Images