A method and system for collaborative dynamic scheduling of heterogeneous resources in workshops

Abstract

The present invention provides a method and system for collaborative dynamic scheduling of heterogeneous resources in a workshop, the method comprising: an order task dynamically enters a task issuing agent in a random form; a scheduling event is triggered, and a first-layer contract network negotiation is performed, the first-layer contract network negotiation includes two rounds of negotiation; in the production stage, the task issuing agent and the available production agent conduct a first round of negotiation, and the optimal production equipment for processing the order task is determined according to the negotiation result; in the detection stage, two rounds of negotiation are performed, the task issuing agent and the available detection agent conduct a first round of negotiation, and the optimal detection equipment for processing the order task is determined according to the negotiation result, and when there is a detection equipment with no buffer not full and no bid, the second round of negotiation is triggered, the task issuing agent increases the reward, and stimulates the detection agent to seek assistance from the detection equipment that can be calculated and unloaded through negotiation, so as to complete the task before the order is overdue; with the goal of minimizing the total production delay of the workshop, the heterogeneous resource scheduling of the workshop is completed.

Description

Technical Field

[0001] The present invention relates to the field of resource scheduling, and more specifically, to a method and system for collaborative dynamic scheduling of heterogeneous resources in a workshop. Background Art

[0002] In the production scheduling of smart workshops, meta-heuristic algorithms can obtain approximate optimal solutions for a large number of NP-hard (non-deterministic polynomial, abbreviated as NP) problems. However, heuristic algorithms can only solve the scheduling optimization of static problems, and it is difficult to deal with urgent order insertions and equipment failures in the production process in real time.

[0003] With the widespread use of computing resources represented by edge computing architecture in smart workshops, workshop computing systems can support real-time computing of deep neural networks, reinforcement learning and other methods in the workshop, thereby solving disturbance problems in workshop production and greatly improving the workshop task scheduling capabilities. However, the existing real-time scheduling methods using reinforcement learning are mainly used to solve the scheduling optimization problem of production resources, and to consider computing resources as part of the direct productivity of the workshop, and coordinate scheduling and optimization with production resources to jointly improve workshop production efficiency.

[0004] Therefore, it is very necessary to dynamically coordinate the scheduling of production resources and computing resources in smart workshops and respond to dynamic disturbances in workshops to further improve the production efficiency of smart manufacturing.

[0005] A certain link in PCB (Printed circuit board) manufacturing includes two stages. The first stage is to use production resources to produce a certain stage of PCB. This stage can be any process such as washing, plugging, printing, pre-baking, exposure, and development in the solder mask process in the specific production. The second stage is reflected in the optical AOI (Automated Optical Inspection) line scanning or other visual inspection processes. In traditional PCB manufacturing, AOI is generally used after the PCB surface treatment, which is reflected in the optical scanning of the entire PCB board, comparing it with the original design drawing to check whether it is correct. With the further in-depth application of intelligent manufacturing, the application of visual assisted production is becoming more and more extensive. Manufacturing companies generally believe that the forward movement of AOI assisted detection will help to discover scratches, contamination and other problems introduced in the early production process of PCB as soon as possible, and reduce repair and production costs. That is, AOI inspection can be performed in all links of the solder mask stage, lamination and other stages. In these stages, there is no need to compare with the original file after scanning, but only to identify defects on the PCB board through intelligent calculation.

[0006] These production processes of PCB manufacturing can be abstracted into workshop production processes with first-stage production and second-stage detection, and in the current flexible manufacturing background, they can be used as the problem scenarios solved by the present invention. In the above scenario, the main process of the detection stage is that the optical equipment in the equipment takes the image of the product, and then uses the computing resources in the equipment or workshop to execute a series of algorithms such as image segmentation, target detection, and defect judgment to find scratches, contamination and other problems introduced in the first stage as early as possible. This type of computing task is the main execution task in the detection stage, and the efficiency of task computing execution directly affects the overall completion time of the workshop. The current real-time scheduling algorithm in the workshop is mainly aimed at production resources, and does not take into account the characteristics of computing resources. It does not comprehensively utilize typical computing task scheduling optimization or parallel computing methods to improve detection efficiency, thereby affecting the production efficiency of the entire workshop. Summary of the invention

[0007] The present invention aims to overcome at least one defect (deficiency) of the above-mentioned prior art and provide a method and system for collaborative dynamic scheduling of heterogeneous resources in a workshop, so as to improve the overall production efficiency of the workshop.

[0008] The technical solution adopted by the present invention is a method for collaborative dynamic scheduling of heterogeneous resources in a workshop, and the method is applied to a manufacturing workshop, wherein the manufacturing workshop manufactures products including a production stage and a detection stage, the production stage corresponds to production equipment, the detection stage corresponds to detection equipment, the production equipment and the detection equipment are connected to an inference terminal, and the inference terminal is connected to a server; a virtual machine is divided in the server as a task publishing agent; the production equipment corresponds to a production agent, and the detection equipment corresponds to a detection agent.

[0009] The method comprises:

[0010] Order tasks enter the task publishing agent dynamically in a random form;

[0011] The scheduling event is triggered, and the first-layer contract network negotiation is carried out. The first-layer contract network negotiation includes the first round of negotiation and the second round of negotiation;

[0012] In the production phase, the task issuing agent conducts a first round of negotiation with the available production agents, and determines the optimal production equipment for processing the order task based on the negotiation results, wherein the negotiation process provides the task issuing agent with a group of production equipment that can produce bills of lading;

[0013] The first and second rounds of negotiation are conducted during the detection phase. The task issuing agent conducts the first round of negotiation with the available detection agents, and determines the optimal detection equipment for processing the order task based on the negotiation results. When there is a detection equipment with no buffer that is not full and no bidder, the second round of negotiation is triggered. The task issuing agent increases the reward to stimulate the detection agent to seek assistance from detection equipment that can be calculated and unloaded through negotiation, so as to complete the task before the order expires. In the process of seeking unloadable detection equipment, a second layer of contract network negotiation will be generated.

[0014] According to the results of multiple rounds of negotiation, the task release agent makes intelligent decisions on production equipment selection, testing equipment selection, production stage buffer order selection, testing stage buffer order selection, and unloading testing equipment selection. Through the guidance of negotiation rewards and decision maker selection rewards, the heterogeneous resource scheduling of the workshop is completed with the goal of minimizing the total production delay of the workshop.

[0015] In the above scheme, the calculation formula for minimizing the total production delay of the workshop is:

[0016]

[0017] J represents the total number of orders, j represents the jth order, and C j is the actual completion time of order j, d j is the overdue time of order j, and X represents the total production delay time of the workshop;

[0018] The actual completion time C of order j j The calculation method is:

[0019]

[0020] in, represents the estimated completion time of order j, C j′，of fload represents the list of offloaded computing tasks that the local machine has accepted and maintained by the agreed detection agent, β represents the proportion of computing workload offloaded to other devices by order j, u represents the proportion of transmission overhead of the increased offloaded tasks of order j, S j is the actual start time of production of order j, P m is the processing efficiency of production equipment m that actually produces order j in the production phase, y j is the production workload of the production stage of order j; the calculation frequency of the inference terminal is f n , the computational efficiency is 1 / f n , the workload of the inspection phase of order j is z j .

[0021] In the above scheme, when there is no testing equipment with a non-full buffer and no bidder, the second round of negotiation is triggered, and the task issuing agent increases the reward, which also includes:

[0022] The task publishing agent increases and updates the current task rewards, and republishes tasks when the task publishing time slot arrives;

[0023] After the detection agent determines whether it has obtained the reward, it will calculate whether it can use computational offloading to assign computing tasks to other detection agents based on the current minimum possible overdue interval of the corresponding detection equipment to meet the requirement of completing the inspection before the order is overdue; if the requirement is met, the detection agent will initiate an offloading request to other detection agents in the same workshop, the offloading object is 1, and the reward = (current task reward - original order reward) * offloading computing amount / total computing amount.

[0024] In the above solution, the process of dispatching calculations to other detection agents also includes:

[0025] This calculation process will trigger the second-layer contract network negotiation, that is, the detection agent negotiates with other detection equipment with offloading capabilities in the workshop whether it can accept the parallel calculation of the offloading task;

[0026] The detection agent corresponding to the detection device with other offloading capabilities calculates whether it can accommodate the offloading computing volume based on the offloading occupancy queue it maintains and the possible minimum overdue interval. If it can, it returns an offloading discount ratio based on the proportion of its own time occupancy. The reward required by the detection agent corresponding to the detection device with other offloading capabilities to the detection agent that initiates the computing offloading = the original reward * (1 + discount ratio).

[0027] In the above scheme, the specific calculation method for accommodating the unloading calculation amount is "the minimum waiting processing time of the uninstalled detection device + the minimum overdue time of the unprocessed orders of the uninstalled detection device + the existing calculated unloading waiting time of the uninstalled detection device > the minimum waiting processing time of the calculated unloading detection device + the expected processing time of the most urgent order - the overdue time of the most urgent order - the existing calculated unloading waiting time of the calculated unloading detection device".

[0028] In the above solution, the intelligent decision-making for production equipment selection is specifically:

[0029] The state space of production equipment selection is expressed as {[production buffer capacity, sum of operation time of all orders in the production buffer, sum of waiting time of all orders in the production buffer, sum of remaining time of all orders in the production buffer, production equipment occupancy time]*M+[expected processing time, order overdue time]};

[0030] The probability distribution of the action space includes {standby, select production equipment 1, ..., select production equipment M};

[0031] The reward for production equipment selection is the actual completion time of the order minus the overdue time;

[0032] and / or,

[0033] The intelligent decision-making for buffer order selection in the production stage is specifically as follows:

[0034] The buffer order selection state space of the production equipment is expressed as {total time of unproduced orders, [expected processing time of unproduced orders, overdue time of unproduced orders, maximum task time for accepting unloading]*J M , minimum waiting time (equipment occupancy time)}; J M Indicates the total number of orders that the production equipment buffer can accommodate;

[0035] The action space is [select the order that enters the production buffer earliest, select the order that enters the production buffer latest, select the order with the shortest processing time, select the order with the longest processing time, select the order with the most urgency]; the reward for order selection in the production stage is the actual completion time of the order minus the overdue time;

[0036] and / or,

[0037] The intelligent decision-making for selecting the detection equipment is specifically:

[0038] The task issuing agents in all detection phases use and maintain the same training model;

[0039] During the detection phase, the order may offload part of the computing workload to other detection devices. According to the negotiation mechanism, it may be offloaded to a single detection device. In this case, the local detection computing workload needs to be split into the tasks to be offloaded to other detection devices, and the transmission overhead of the offloaded tasks and the offloaded computing workload from other detection devices will be increased.

[0040] and / or,

[0041] The intelligent decision-making for buffer order selection in the detection phase is specifically as follows:

[0042] The rewards for selecting the testing equipment during the testing phase include rewards for order testing and negotiated price increases, as well as rewards for undertaking computing offloading tasks during the duration of the order;

[0043] and / or,

[0044] The specific intelligent decision-making for uninstall detection equipment selection is:

[0045] The status space of the uninstall detection device is [(minimum overdue time of undetected orders, minimum waiting time, uninstalled waiting time)*J N ]; J N Indicates the total number of orders that the detection device buffer can accommodate;

[0046] All compute offload publishing agents use and maintain the same model;

[0047] The probability distribution of the action space includes {standby, select detection device 1, ..., select detection device N}; the standby action indicates that there is no device that meets the offloading condition and is not suitable for computation offloading.

[0048] The rewards for equipment selection during the detection phase include rewards for order production affected by offloading and rewards for undertaking the offloading volume, as well as rewards for undertaking computational offloading tasks during the duration of the order.

[0049] In the above scheme, when it is found that all detection devices are not suitable for computing offloading, the process also includes:

[0050] The task issuing agent is required to perform mandatory allocation of orders. The task issuing agent executes the device selection decider to decide the most suitable detection device, insert the order into the buffer of the detection device, mark the order, and when the order completes the detection, no additional negotiation timeout penalty will be received.

[0051] The present invention also provides a workshop heterogeneous resource collaborative dynamic scheduling system, the system is applied to a manufacturing workshop, the manufacturing workshop manufactures products including a production stage and a detection stage, the system comprises:

[0052] Production equipment corresponding to the production stage;

[0053] Testing equipment corresponding to the testing stage;

[0054] Reasoning terminals connected to production equipment and testing equipment respectively;

[0055] A server connected to the inference terminal, wherein the server is divided into a virtual machine as a task publishing agent, and the task agent is used to receive order tasks dynamically entered in a random form;

[0056] Production agent for corresponding production equipment;

[0057] Detection agent corresponding to the detection equipment;

[0058] The first-tier contract negotiation network includes the first round of negotiation and the second round of negotiation mechanism. The first round of negotiation is used to bid for production equipment and testing that can handle orders, and the second round of negotiation is used to increase the price and reward the testing equipment that can handle urgent orders;

[0059] The second-tier contract negotiation network negotiates and allocates additional rewards according to the amount of tasks that can be offloaded, bids for testing equipment that can perform computational offloading, and reduces production delays through parallel computing.

[0060] The intelligent decision maker for equipment selection gives appropriate equipment selection results based on the equipment and order status of the workshop and guided by historical rewards;

[0061] The buffer order selection intelligent decision maker gives appropriate order selection results according to the buffer order status and equipment occupancy status, under the guidance of historical rewards.

[0062] Uninstall the device selection intelligent decision maker, and give appropriate device selection results according to the device status and order status under the guidance of historical rewards.

[0063] The model parameter update module uses the DQN network parameter update method to update the target network parameters and value network parameters according to the number of decision executions. It also maintains a playback pool to record state-action parameters, listens to order execution results, and stores the results in the corresponding order location.

[0064] A device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for collaborative dynamic scheduling of heterogeneous resources in a workshop as described above.

[0065] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for collaborative dynamic scheduling of heterogeneous resources in a workshop as described above.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] Compared with the single production scheduling or computing scheduling method in the smart manufacturing workshop in the prior art, in the present invention, the heterogeneous resources such as production and computing in the workshop are collaboratively scheduled, and a dynamic scheduling algorithm based on the contract network is constructed by combining the two, which further improves the production efficiency of the smart manufacturing workshop. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;

[0069] Figure 2 A schematic diagram of a flow chart of an embodiment of a method for collaborative dynamic scheduling of heterogeneous resources in a workshop according to the present invention;

[0070] Figure 3 A schematic diagram of a contract network negotiation process of an embodiment of a method for collaborative dynamic scheduling of heterogeneous resources in a workshop of the present invention;

[0071] Figure 4 A schematic diagram of reasoning and training of a decision maker model in an embodiment of a method for collaborative dynamic scheduling of heterogeneous resources in a workshop of the present invention; DETAILED DESCRIPTION

[0072] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0073] Example 1

[0074] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0075] The device in the embodiment of the present invention may be a distributed cluster composed of computers, servers or other computing devices. The device in this embodiment may be used to implement resource scheduling and efficient production in an intelligent manufacturing workshop.

[0076] like Figure 1 As shown, the cloud-side server and the edge-side reasoning terminal in the distributed cluster may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can be used to connect a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and the optional user interface 1003 can also include a standard wired interface and a wireless interface. The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0077] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation on the device for collaborative dynamic scheduling of heterogeneous resources, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0078] In the specific implementation process, the cloud-side server and edge-side reasoning terminal are based on the Atlas800 training server and the Atlas200 reasoning terminal respectively, and virtual machines are divided in the Atlas800 training server to connect to the task input as a task publishing agent. Other resources in the Atlas800 training server are used as necessary computing resources for target network model parameter update training and storage of state-action-reward information playback pool in subsequent intelligent decision makers. Atlas200 terminals are connected to each production equipment and testing equipment in the original workshop for contract network negotiation and intelligent decision processing. In this way, a set of intelligent workshop cloud-edge architecture is formed.

[0079] On top of the above-mentioned cloud-edge architecture, the workshop adopts a two-stage production system. The first stage is the production stage. When manufacturing PCB boards, the production stage is the PCB board manufacturing process. The second stage is the inspection stage. When manufacturing PCB boards, the inspection stage is the quality inspection process of the products of the previous stage through AOI technology, which may include image scanning and visual intelligent processing parts.

[0080] like Figure 1 As shown, the memory 1005 as a computer storage medium in the cloud-side server and the edge-side inference terminal may include an operating system, a network communication module, a user interface module, and a resource collaborative scheduling program. The operating system is a program that manages and controls the hardware and software resources of the device, and supports the operation of the resource collaborative scheduling program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the resource collaborative scheduling program stored in the memory 1005, and Figure 2 As shown, do the following:

[0081] S101. The order task dynamically enters the task publishing agent in a random form;

[0082] S102. The scheduling event is triggered, and the first-layer contract network negotiation is performed. The first-layer contract network negotiation includes the first round of negotiation and the second round of negotiation:

[0083] S1021. In the production phase, the task issuing agent conducts a first round of negotiation with the available production agents, and determines the optimal production equipment for processing the order task based on the negotiation results, wherein the negotiation process provides the task issuing agent with a group of production equipment that can produce bills of lading;

[0084] S1022. The first round of negotiation and the second round of negotiation are carried out in the detection phase. The task issuing agent conducts the first round of negotiation with the available detection agents, and determines the optimal detection equipment for processing the order task according to the negotiation results. When there is a detection equipment with no buffer not full and no bid, the second round of negotiation is triggered. The task issuing agent increases the reward to stimulate the detection agent to seek assistance from the detection equipment that can be calculated and unloaded through negotiation, so as to complete the task before the order expires. In the process of seeking the detection equipment that can be unloaded, the second layer of contract network negotiation will be generated;

[0085] S103. Based on the results of multiple rounds of negotiations, the task issuing agent makes intelligent decisions on the selection of production equipment, the selection of testing equipment, the selection of production phase buffer orders, the selection of testing phase buffer orders, and the selection of unloading testing equipment. Through the guidance of negotiation rewards and decision-maker selection rewards, the heterogeneous resource scheduling of the workshop is completed with the goal of minimizing the total production delay of the workshop.

[0086] It should be noted that, in this embodiment, the decision makers related to order manufacturing are divided into two categories, namely, equipment selection decision makers and order selection decision makers.

[0087] Both types of decision makers play a role in both the production and testing stages.

[0088] In the production stage, the equipment selection decision maker runs on the cloud-side server, and there is only one. The order selection decision maker runs on the Atlas200 inference terminal associated with each production device. There are multiple inference models, but only one training model is maintained on the cloud-side server. Figure 3 In order to prevent multiple inference terminals from triggering the parameter update process of the training model at the same time, a spin lock mechanism is used to allow the inference terminal to seize the only training opportunity during the model training process.

[0089] In the detection phase, both the device selection decision maker and the order selection decision maker run on the edge inference terminal. There are multiple inference models, but only the same training model is maintained on the cloud server. To prevent multiple inference terminals from triggering the parameter update process of the training model at the same time, a spin lock mechanism is used so that the inference terminal seizes the only training opportunity during the model training process.

[0090] The calculation formula for minimizing the total production delay of the workshop is:

[0091]

[0092] J represents the total number of orders, j represents the jth order, and C j is the actual completion time of order j, d j is the overdue time of order j, and X represents the total production delay time of the workshop;

[0093] The actual completion time C of order j j The calculation method is:

[0094]

[0095] in, represents the estimated completion time of order j, C j′，offload represents the list of offloaded computing tasks that the local machine has accepted and maintained by the agreed detection agent, β represents the proportion of computing workload offloaded to other devices by order j, μ represents the proportion of transmission overhead of the increased offloaded tasks of order j, and S j is the actual start time of production of order j, P m is the processing efficiency of production equipment m that actually produces order j in the production phase, y j is the production workload of the production stage of order j; the calculation frequency of the inference terminal is f n , the computational efficiency is 1 / f n , the workload of the inspection phase of order j is z j .

[0096] Specifically, before executing the step of collaborative dynamic scheduling of heterogeneous resources in a workshop, the processor 1001 located in the cloud-side server can be used to call the resource collaborative scheduling program stored in the memory 1005, and perform the following operations:

[0097] According to the total number of orders currently required to be produced and the total number of products to be produced in the orders, a set of input conditions is constructed, a two-stage intelligent workshop system model is constructed, and the production time of a single product in the two stages is constructed using formulas;

[0098] Obtain and use formulas to represent the start time, completion time, and order deadline of production of a specific device (production equipment or testing equipment) occupied by an order.

[0099] Furthermore, the processor 1001 in the cloud-side server may be used to call the heterogeneous resource collaborative dynamic scheduling program stored in the memory 1005, and perform the following operations:

[0100] Get the newly arrived production requirements from the task input interface;

[0101] The task release agent packages the demand into an order. Specifically, the task release agent packages the order delivery date of the production stage into a task announcement and sends the task announcement to all production agents in the workshop. The sending is to release the highest priority order through the task release agent. At this time, the scheduling event is triggered, and the task release agent negotiates with the available production agents in the next step. The production agent evaluates the results according to the corresponding production equipment's own production capacity (processing efficiency, equipment buffer size, etc.) and order attributes (estimated completion time, order urgency, etc.), and then bids (production capacity).

[0102] like Figure 4 As shown:

[0103] Furthermore, the processor 1001 in the edge reasoning terminal can be used to call the heterogeneous resource collaborative dynamic scheduling program stored in the memory 1005, and perform the following operations:

[0104] After receiving the task release announcement, the attributes of the order to be scheduled (including the estimated completion time and the urgency of the order) are obtained. Then the estimator maintained by this production agent is called to evaluate whether the equipment can complete the task execution before the order is expected, combining the status of device cache idleness, device production capacity, remaining time for processing orders, order simulation priority, etc. If the device buffer is full or the estimated production capacity is insufficient to meet the order completion within the expected time, the task evaluation result is that the production equipment is not suitable for undertaking new production tasks, and the production agent will not bid to the issuing agent. The production agents participating in the bidding measure the performance of completing the production tasks based on the status of device cache idleness, device production capacity, remaining time for processing orders, order simulation priority, etc., and encapsulate the evaluation results into the bid information and feedback to the task issuing agent.

[0105] Optionally, the workshop negotiation and scheduling time slots are divided according to Δt, and one negotiation is completed within one Δt time slot.

[0106] Optionally, before the step of producing the agent to bid, the method further includes:

[0107] If no valid response is obtained within a negotiation Δt time slot, the negotiation starts again in the next Δt time slot. This step is repeated for a number of Δt time slots, such as 3.

[0108] If there is still no valid response, the task issuing agent requests the equipment buffer status from the production agents of the entire workshop. If all are full, it waits for a certain number of time slots (the default is 20 Δt time slots) and then starts negotiation again.

[0109] If it is not full, it means that the order urgency is too high and all equipment cannot complete production within the order overdue time. Optionally, in the production task stage, the workshop equipment buffer is not full and the order urgency is too high. Before selecting the optimal production equipment to process the order, it also includes:

[0110] The task release agent is required to perform mandatory allocation of orders. The task release agent executes the equipment selection decider to produce the most suitable equipment and insert the order into the equipment buffer, but marks the order, and does not receive additional negotiation timeout penalties when the order is completed.

[0111] Furthermore, the processor 1001 in the cloud-side server may be used to call the heterogeneous resource collaborative dynamic scheduling program stored in the memory 1005, and perform the following operations:

[0112] The task release agent collects all bid information received within a certain time window and analyzes the content of these bids. Specifically, the task release agent constructs the equipment selection state space based on the production agent's bid information. According to the equipment selection decision maker maintained by the task release agent, the probability distribution of equipment selection in the production stage is given, and the optimal production equipment for processing the order is selected according to the action strategy. At the same time, the workshop status is updated and written into the playback pool.

[0113] Optionally, before the step of constructing the device selection state space, the method further includes:

[0114] Each order status includes [expected processing time, order overdue time]. The expected processing time of an order is the total production volume of the order / standard equipment processing capacity.

[0115] Each equipment state includes [production buffer capacity, sum of operation time of all orders in the production buffer, sum of waiting time of all orders in the production buffer, sum of remaining time of all orders in the production buffer, and production equipment occupancy time]. The production occupancy time indicates the remaining processing time of the order being processed on the production equipment, that is, the waiting time required to execute a new order on the production equipment. Since each production equipment contains two states, the length of the state space of the decision maker is 5*M+2, where M represents the total number of production equipment in the workshop.

[0116] Optionally, before the step of providing the probability distribution of equipment selection in the production phase according to the equipment selection decider maintained by the task issuing agent, the method further includes:

[0117] Set the neural network model structure to: (5*M+2)×[(5*M+2)*4]×[(5*M+2)*4]×(M+1).

[0118] All network nodes are fully connected.

[0119] Optionally, before the step of providing the probability distribution of equipment selection in the production phase, the method further includes:

[0120] The scheduler fits the Q value table according to the input state and obtains the probability distribution of the action space, including {standby, select production equipment 1, ..., select production equipment M}. The standby action means that the buffers of the production equipment are full and are not suitable for accepting new orders. It is necessary to wait for the production equipment to complete the buffer orders before accepting new orders.

[0121] Select the action space as [select the order that enters the production buffer earliest, select the order that enters the production buffer latest, select the order with the shortest processing time, select the order with the longest processing time, select the order with the most urgency].

[0122] In action selection, the ε-greedy algorithm is used to select the action with the largest action probability distribution with probability P, and randomly select actions with probability (1-P) (randomly select from devices whose buffers are not full). P is set to 0.9 based on experience.

[0123] Optionally, before the step of calculating the reward according to the task completion result, the method further includes:

[0124] The optimization goal is to minimize the total delay in order production in the workshop, so the reward needs to take into account the cumulative early completion time of the equipment. However, after the production equipment selects the action, the order only enters the buffer of the production equipment, and will not be produced immediately and the completion time of the order will not be obtained. Therefore, the present invention adopts a delayed reward method to record the SA-S'-R information in the algorithm. After the action is obtained, the decision maker only records the current state and the selected action into the playback pool, and the reward is empty at this time. The playback pool continuously monitors the execution of the order. Only after the order is completed, the reward is generated according to the formula and stored in the playback pool.

[0125] The reward for equipment selection in the production phase is the actual completion time of the order minus the overdue time.

[0126] Optionally, before the step of the production agent constructing the state space according to the buffer information, the method further includes:

[0127] When the production equipment completes the production of the current order, it needs to select an order from the local buffer for production. Therefore, all order states in the production equipment buffer constitute the state space of the order selection decision maker.

[0128] The order status in the buffer of each production equipment includes [average total time of unproduced orders, expected processing time of unproduced orders*J M , J M represents the total number of orders that the production equipment buffer can accommodate, and the minimum waiting time (production equipment occupancy time)]. The length of the state space is JM +2. Considering the unity of the states in the production and testing phases, the testing phase needs to add additional calculation unloading flags. Therefore, the state space is expanded to {total time of unproduced orders, [expected processing time of unproduced orders, overdue time of unproduced orders, maximum task time for accepting unloading]*J M , minimum waiting time (equipment occupancy time)}, the state space length is 3*J M +2. In the specific implementation process, since the production equipment does not involve unloading calculations, in the production stage, the maximum task time for accepting unloading is usually set to 0.

[0129] It should be noted that in the state space, the time variables of the order and other states are normalized (Min-Max Scaling) and the values ​​are linearly mapped to [0,1].

[0130] Optionally, before the step of calling the order selection decider, the method further includes:

[0131] Set the neural network model structure to: (3*J M +2)×[(3*J M +2)*4]×[(3*J M +2)*4]×(5).

[0132] All network nodes are fully connected.

[0133] Since the delayed reward method is adopted in the present invention, the task issuing agent needs to continuously monitor the completion of the order task, and then calculate the reward according to the task completion result, and fill it into the corresponding position of the replay pool. When the order execution reaches the agreed number of times, SA-S'-R is selected from the replay pool, where S represents the state, A represents the action, S' is the updated state, and R is the reward obtained after the current action is executed and converted to S'; the complete replay sample is used as the mini-batch of this training, and the target network parameters are updated using the DQN method. And the value network parameters before the update are assigned to the target network.

[0134] The order enters the order buffer of the assigned device. When the device completes the production of the on-board order, the scheduling time is triggered. The production agent builds the state space based on the buffer information, calls the order selection decider, gives the probability distribution of order selection in the production stage, and selects the order that is optimal for the system production delay based on the action strategy.

[0135] Furthermore, the processor 1001 in the edge server may be used to call the heterogeneous resource collaborative dynamic scheduling program stored in the memory 1005, and perform the following operations:

[0136] When the order completes the first stage of production, the production equipment at this time acts as the task release agent for the inspection and sends the order announcement to all inspection agents in the workshop according to the contract network protocol. The negotiation process in the inspection stage is divided into two rounds:

[0137] The first round of negotiations is consistent with the negotiation process in the production phase: the testing agent evaluates the results based on its own testing capabilities (testing efficiency, equipment buffer size, etc.) and order attributes (estimated completion time, order urgency, etc.), and then bids (testing capabilities).

[0138] When all detection agents evaluate that they are unable to complete the detection before the order expires, they will not bid in the first round of negotiation. If it is found that the non-device buffer is full, the detection task issuing agent will initiate a second round of negotiation.

[0139] The second round of negotiation is the negotiation process of increasing the reward, specifically:

[0140] At this time, the task release agent updates the current task reward = original order reward * 10, and re-releases the task when the task release time slot arrives. After the detection agent determines whether it can use computational offloading to dispatch computing tasks to other detection agents based on the preset reward threshold of the detection agent and the minimum possible overdue interval of the current equipment, so as to meet the requirement of completing the inspection before the overdue date. This calculation process will trigger the second-layer contract network negotiation, that is, the monitoring agent preparing to bid for computational offloading negotiates with other detection equipment with offloading capabilities in the workshop whether it can accept parallel computing of offloading tasks. The computational offloading negotiation process needs to be completed within the computational offloading negotiation time slot Δτ, Δτ∈ scheduling time slot Δt. The device that becomes the computational offloading release agent will no longer respond to the offloading inquiries initiated by other computational offloading agents in this round of inquiries.

[0141] If the requirements are met, the detection agent will initiate an unloading request to the detection agents of other detection equipment in the same workshop. The unloading object is 1, and the reward = (current task reward-original order reward)*unloading calculation amount / total calculation amount.

[0142] Other detection agents calculate whether they can accommodate the offloading calculation volume based on the offloading occupancy queues they maintain and the possible minimum overdue interval. The specific calculation method is "the minimum waiting processing time of the offloaded detection device + the minimum overdue time of the unprocessed orders of the offloaded detection device + the existing calculation offloading waiting time of the offloaded detection device > the minimum waiting processing time of the calculation offloading detection device + the expected processing time of the most urgent order - the overdue time of the most urgent order - the existing calculation offloading waiting time of the calculation offloading detection device".

[0143] If it can be accommodated, an offloading discount ratio is returned according to the proportion of its own time occupied. The reward required by the detection agent to initiate the calculation offloading detection agent = original reward * (1 + discount ratio).

[0144] The number of unloading negotiations is still performed according to the aforementioned negotiation Δt time slots, but multiple negotiations are not repeated to ensure that the detection task allocation can be completed within 3 Δt time slots.

[0145] After receiving the bid, the offload agent constructs the offload state space, calls the offload decision maker, gives the probability distribution of offload detection equipment selection, and selects the optimal detection equipment to handle the offloaded computation according to the action strategy. At the same time, the workshop status is updated and written into the playback pool.

[0146] The order is placed in the order buffer of the assigned detection device. When the detection device completes the on-board order detection, the scheduling time is triggered. The detection agent builds the state space based on the buffer information, calls the order selection decider, gives the probability distribution of order selection in the detection phase, and selects the order that is optimal for the system production delay based on the action strategy.

[0147] Optionally, before the step of constructing the computation offloading state space, the method further includes:

[0148] After the detection agent bids to the computing task offloading agent, the computing task offloading agent forms the offloading state space of each detection device in the detection stage according to the bidding result. The state space is [(minimum overdue time of undetected orders, minimum waiting time, offloaded waiting time)*J N ], J N Indicates the total number of orders that the detection equipment buffer can accommodate. In the state space, the time variables of the order and other states are normalized (Min-Max Scaling), and the values ​​are linearly mapped to [0,1]. In addition, the minimum overdue time of the untested orders of the unbid detection equipment is set to 0, the minimum waiting time is set to 0, and the unloaded waiting time is set to 1.

[0149] Optionally, before calling the step of calculating the offloading decision maker, the method further includes:

[0150] Set the neural network model structure to: (3*J N )×[(3*J N )*4]×[(3*J N )*4]×(N+1).

[0151] All detection agents that offload calculations use and maintain the same model. To prevent multiple detection agents from training model parameters at the same time, a spin lock mechanism is set for the model. When a detection agent enters the model parameter update phase, it first reads the spin lock state. If it is 0, it performs a parameter update operation and sets the spin lock to 1. After the parameter update is completed, the spin lock is released and reset to 0. If the spin lock state is read and found to be, the parameter update operation is exited.

[0152] All network nodes are fully connected.

[0153] Optionally, before the step of calculating the probability distribution of the offloading device selection is given, the method further includes:

[0154] The scheduler fits the Q value table according to the input state and obtains the probability distribution of the action space, including {standby, select detection device 1, ..., select detection device N}. The standby action means that there is no device that meets the offloading conditions and is not suitable for computation offloading.

[0155] In action selection, the ε-greedy algorithm is used to select the action with the largest action probability distribution with probability P, and randomly select actions with probability (1-P) (randomly select among uninstallable devices). P is set to 0.9 based on experience.

[0156] Optionally, before the step of calculating the reward according to the task completion result, the method further includes:

[0157] The order selection reward also uses a delayed reward method to record the SA-S'-R information in the algorithm. After the action is obtained, the decision maker only records the current state and the selected action into the replay pool, and the reward is empty at this time. The replay pool continuously monitors the execution of the order. Only after the order is completed, the reward is generated according to the formula and stored in the replay pool.

[0158] When the number of scheduling executions of each decision maker reaches a multiple of 10, the program takes out 20 groups of SA-S'-R historical samples from the revisit pool, trains the value network of the decision maker, updates its network parameters, and uses the old parameters of the value network to update the target network of the decision maker.

[0159] The rewards for the selection of detection equipment in the detection phase include the rewards for the production of orders affected by unloading, the rewards for undertaking the unloading volume, and the rewards for undertaking the calculation of unloading tasks during the duration of the order. In order to unify the magnitude of the three types of rewards, the reward branches are normalized (Min-Max Scaling) and the values ​​are linearly mapped to [0,1].

[0160] Optionally, in the detection phase, if the buffer of the workshop detection equipment is not fully full, the order urgency is too high, and the unloading negotiation is unsuccessful, before the step of selecting the optimal detection equipment for processing the order, the method further includes:

[0161] The task issuing agent is required to perform mandatory allocation of orders. The task issuing agent executes the device selection decider, selects the most suitable detection device for the detection, inserts the order into the detection device buffer, but marks the order. When the order completes the detection, it will not receive additional negotiation timeout penalties.

[0162] Optionally, before the step of constructing the device selection state space in the detection phase, the method further includes:

[0163] Each order status includes [expected processing time, order overdue time]. The expected processing time of an order is the total production volume of the order / standard equipment processing capacity.

[0164] The state of each testing device includes [buffer capacity, sum of operation time of all orders in the buffer, sum of waiting time of all orders in the buffer, sum of remaining time of all orders in the buffer, and equipment occupancy time]. The occupancy time indicates the remaining processing time of the order being processed on the testing device, that is, the waiting time required to execute a new order on the testing device. Since each testing device contains two states, the state space length of the decision maker is 5*N+2, where N represents the total number of testing devices in the workshop.

[0165] Optionally, before the step of providing the probability distribution of the detection device selection in the detection phase according to the device selection decider maintained by the task issuing agent, the method further includes:

[0166] Set the neural network model structure to: (5*N+2)×[(5*N+2)*4]×[(5*N+2)*4]×(N+1).

[0167] All detection task publishing agents use and maintain the same model. To prevent multiple detection agents from training model parameters at the same time, a spin lock mechanism is set for the model. When a detection agent enters the model parameter update phase, it first reads the spin lock status. If it is 0, it performs the parameter update operation and sets the spin lock to 1. After the parameter update is completed, the spin lock is released and reset to 0. If the spin lock status is read, the parameter update operation is exited.

[0168] All network nodes are fully connected.

[0169] Optionally, before the step of providing the probability distribution of device selection in the detection phase, the method further includes:

[0170] The scheduler fits the Q value table according to the input state and obtains the probability distribution of the action space, including {standby, select detection device 1, ..., select detection device N}. The standby action means that the device buffer is full and is not suitable for accepting new orders. It is necessary to wait for the detection device to complete the buffer order before accepting new orders.

[0171] In action selection, the ε-greedy algorithm is used to select the action with the largest action probability distribution with probability P, and randomly select actions with probability (1-P) (randomly select from detection devices whose buffers are not full). P is set to 0.9 based on experience.

[0172] Optionally, before the step of calculating the reward according to the task completion result in the detection phase, the method further includes:

[0173] During the detection phase, the order may offload part of the computation to other devices. According to the negotiation mechanism, it may be offloaded to a single device. Then the detection computation of the local machine needs to be split into the task amount offloaded to other devices, and the transmission overhead of the offloaded task is increased.

[0174] During the detection phase, there may be offloaded computations from other detection devices. This computational load may occur during the processing of order i, and the completion time of order i is affected by the offloaded computations. Each detection agent maintains a computation offload queue. The local offload computation task list maintained by the agreed detection agent is represented by D offload , which is represented by a list with list as its element, for example {...,{s i C ,s i C '},...},s i C Indicates the time when an unloading starts, s i C 'Indicates the time when the unloading volume ends, and its calculation method is as follows:

[0175]

[0176]

[0177] h i′ To indicate the overdue time of the order that needs to be calculated for unloading, the default method used for unloading is to start from the overdue time and subtract the amount of tasks that can be unloaded.

[0178] Compared to the duration of order i, D i,offload It can be expressed as

[0179]

[0180] α k Represents the proportion of unloading volume among multiple testing devices participating in the same order unloading volume. In the specific implementation process, it can be simplified to unloading the same order unloading volume to only one testing device, so it can be considered that α k =1,s i They respectively represent the start time of the unloaded orders.

[0181] Therefore, the actual completion time of order i needs to be modified to

[0182]

[0183] Among them, order i′ is the computing amount offloaded from other orders on device n.

[0184] Optionally, before the step of the detection agent constructing the state space according to the buffer information, the method further includes:

[0185] When the detection device completes the current order detection, it needs to select an order from the local buffer for detection. Therefore, all order states in the detection device buffer constitute the state space of the order selection decision maker.

[0186] The order status in each detection device buffer includes [average total time of undetected orders, expected processing time of undetected orders*J N , minimum waiting processing time (equipment occupancy time)]. The length of the state space is J N +2. Considering the unity of the states in the production and testing phases, the testing phase needs to add additional calculation unloading flags. Therefore, the state space is expanded to {total time of untested orders, [expected processing time of untested orders, overdue time of untested orders, maximum task time for accepting unloading]*J N , minimum waiting time (equipment occupancy time)}, the state space length is 3*J N +2.

[0187] It should be noted that in the state space, the time variables of the order and other states are normalized (Min-Max Scaling) and the values ​​are linearly mapped to [0,1].

[0188] Optionally, before the step of providing the probability distribution of order selection in the detection phase, the method further includes:

[0189] Select the action space as [select the order that enters the detection buffer earliest, select the order that enters the detection buffer latest, select the order with the shortest detection time, select the order with the longest detection time, select the order with the most urgency].

[0190] In action selection, the ε-greedy algorithm is used to select the action with the largest action probability distribution with probability P, and randomly select actions with probability (1-P) (randomly select from devices whose buffers are not full). P is set to 0.9 based on experience.

[0191] The rewards for equipment selection during the detection phase include rewards for order production and negotiated price increases, as well as rewards for undertaking computational offloading tasks during the duration of the order. To unify the magnitude of the three types of rewards, the reward branches are normalized (Min-Max Scaling) and the values ​​are linearly mapped to [0,1].

[0192] Optionally, in the steps of all intelligent agents calling the decision maker, the following steps are also included:

[0193] The decision maker runs on the Atlas200 inference terminal in the workshop cloud-edge architecture. After the terminal completes the calculation, the result is directly given. The replay pool is stored on the Atlas800 training server. After the decision maker runs the result, SA-S'-[] is transmitted to the Atlas800 training server. After the R calculation is completed, the intelligent agent responsible for listening fills in the corresponding position in the replay pool.

[0194] Optionally, in the steps of all intelligent agents using the DQN method to update the target network parameters, the following steps are further included:

[0195] The value network in all intelligent agent decision makers runs on the Atlas800 training server. The target network is stored on the Atlas800 training server and the Atlas200 inference terminal. When the target network update is triggered, the model training is performed on the Atlas800 training server, and after completion, the updated model parameters replace the old parameters of the target network on the Atlas200 inference terminal.

[0196] In the present invention, a method for dynamic collaborative scheduling of heterogeneous resources in a workshop is implemented, which includes: designing a two-layer "contract negotiation network" + a three-layer deep reinforcement learning "decision maker" to organize multiple agents in the workshop to perform self-organized collaborative scheduling. The first layer of negotiation network: when the workshop order arrives dynamically, the workshop task release agent negotiates with the production inspection or inspection agent based on the contract network to obtain the production equipment, and uses the equipment selection order selection "decision maker" to obtain the optimal equipment to arrange the order into the equipment buffer; when the equipment completes the production of the current order, the deep reinforcement learning model is used according to the buffer order status to select the priority order that best meets the needs for subsequent production. The second layer of negotiation network: when an abnormal disturbance occurs in the workshop and an urgent inspection order is generated, and the first layer of negotiation scheduling cannot eliminate the order default, the task release agent increases the bidding reward, and the inspection agent negotiates with other inspection agents in the workshop that can use computational offloading to share the computing tasks, and obtains the computational offloading bid, and then negotiates the bid with the first layer; when the task marked for computational offloading starts production, the offloading "decision maker" is called to select the inspection agent that bid in the early stage in real time, offload part of the computing tasks, eliminate order default as much as possible, and further minimize the total delay of workshop production.

[0197] Example 2

[0198] The present invention also provides a workshop heterogeneous resource collaborative dynamic scheduling system, the system is applied to a manufacturing workshop, the manufacturing workshop manufactures products including a production stage and a detection stage, the system comprises:

[0199] Production equipment corresponding to the production stage;

[0200] Testing equipment corresponding to the testing stage;

[0201] Reasoning terminals connected to production equipment and testing equipment respectively;

[0202] A server connected to the inference terminal, wherein the server is divided into a virtual machine as a task publishing agent, and the task agent is used to receive order tasks dynamically entered in a random form;

[0203] Production agent for corresponding production equipment;

[0204] Detection agent corresponding to the detection equipment;

[0205] The first-tier contract negotiation network includes the first round of negotiation and the second round of negotiation mechanism. The first round of negotiation is used to bid for production equipment and testing that can handle orders, and the second round of negotiation is used to increase the price and reward the testing equipment that can handle urgent orders;

[0206] The second-tier contract negotiation network negotiates and allocates additional rewards according to the amount of tasks that can be offloaded, bids for testing equipment that can perform computational offloading, and reduces production delays through parallel computing.

[0207] The intelligent decision maker for equipment selection gives appropriate equipment selection results based on the equipment and order status of the workshop and guided by historical rewards;

[0208] The buffer order selection intelligent decision maker gives appropriate order selection results according to the buffer order status and equipment occupancy status, under the guidance of historical rewards.

[0209] Uninstall the device selection intelligent decision maker, and give appropriate device selection results according to the device status and order status under the guidance of historical rewards.

[0210] The model parameter update module uses the DQN network parameter update method to update the target network parameters and value network parameters according to the number of decision executions. It also maintains a playback pool to record state-action parameters, listens to order execution results, and stores the results in the corresponding order location.

[0211] The expanded content of the specific implementation method of the workshop heterogeneous resource collaborative dynamic scheduling system of the present invention is basically the same as the above-mentioned workshop heterogeneous resource collaborative dynamic scheduling embodiments, and will not be repeated here.

[0212] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for collaborative dynamic scheduling of heterogeneous resources in a workshop, characterized in that: The method is applied to a manufacturing workshop, wherein the manufacturing workshop manufactures products including a production phase and a testing phase, the production phase corresponds to production equipment, the testing phase corresponds to testing equipment, the production equipment and the testing equipment are connected to an inference terminal, and the inference terminal is connected to a server; a virtual machine is divided in the server as a task publishing agent; There are production agents for production equipment and testing agents for testing equipment. The method comprises: Order tasks enter the task publishing agent dynamically in a random form; The scheduling event is triggered, and the first-layer contract network negotiation is carried out. The first-layer contract network negotiation includes the first round of negotiation and the second round of negotiation. In the production phase, the task issuing agent conducts a first round of negotiation with the available production agents, and determines the optimal production equipment for processing the order task based on the negotiation results, wherein the negotiation process provides the task issuing agent with a group of production equipment that can produce bills of lading; The first and second rounds of negotiation are conducted during the detection phase. The task issuing agent conducts the first round of negotiation with the available detection agents, and determines the optimal detection equipment for processing the order task based on the negotiation results. When there is a detection equipment with no buffer that is not full and no bidder, the second round of negotiation is triggered. The task issuing agent increases the reward to stimulate the detection agent to seek assistance from detection equipment that can be calculated and unloaded through negotiation, so as to complete the task before the order expires. In the process of seeking unloadable detection equipment, a second layer of contract network negotiation will be generated. According to the results of multiple rounds of negotiation, the task issuing agent makes intelligent decisions on the selection of production equipment, testing equipment, production phase buffer order, testing phase buffer order, and unloading testing equipment. Through the guidance of negotiation rewards and decision-maker selection rewards, the heterogeneous resource scheduling of the workshop is completed with the goal of minimizing the total production delay of the workshop. When there is no buffer that is not full and no testing equipment that has responded to the bid, the second round of negotiation is triggered. The task issuing agent increases the reward in the process, which also includes: The task publishing agent increases and updates the current task rewards, and republishes tasks when the task publishing time slot arrives; After the detection agent determines that it has obtained the reward, it calculates whether it can use computational offloading to assign computational tasks to other detection agents based on the current possible minimum overdue interval of the corresponding detection equipment to meet the requirement of completing the inspection before the order is overdue; if the requirement is met, the detection agent initiates an offloading request to other detection agents in the same workshop, and the offloading object is 1, and the reward = (current task reward - original order reward) * offloading computation amount / total computation amount; The process of distributing computations to other detection agents also includes: This calculation process will trigger the second-layer contract network negotiation, that is, the detection agent negotiates with other detection equipment with offloading capabilities in the workshop whether it can accept the parallel calculation of the offloading task; The detection agent corresponding to the detection device with other offloading capabilities calculates whether it can accommodate the offloading computing volume based on the offloading occupancy queue it maintains and the possible minimum overdue interval. If it can accommodate, it returns an offloading discount ratio based on the proportion of its own time occupancy. The reward required by the detection agent corresponding to the detection device with other offloading capabilities to the detection agent that initiates the computing offloading = the original reward * (1 + discount ratio).

2. The method for collaborative dynamic scheduling of heterogeneous resources in a workshop according to claim 1, characterized in that: The calculation formula for minimizing the total production delay of the workshop is: J represents the total number of orders, j represents the jth order, and C j is the actual completion time of order j, d j is the overdue time of order j, and X represents the total production delay time of the workshop; The actual completion time C of order j j The calculation method is: in, represents the estimated completion time of order j, Indicates the list of offloaded computing tasks that the agreement detection agent maintains on this machine. represents the proportion of the computational workload of order j offloaded to other devices, They represent the proportion of increased offloading task transmission overhead of order j, S j is the actual start time of production of order j, is the processing efficiency of production equipment m that actually produces order j during the production phase, is the production workload of the production stage of order j; the calculation frequency of the inference terminal is , the computational efficiency is , the workload of the detection phase of order j is .

3. The method for collaborative dynamic scheduling of heterogeneous resources in a workshop according to claim 1, characterized in that: The specific calculation method for accommodating the unloading calculation amount is "the minimum waiting processing time of the unloaded detection device + the minimum overdue time of the unprocessed orders of the unloaded detection device + the existing calculated unloading waiting time of the unloaded detection device > the minimum waiting processing time of the calculated unloading detection device + the expected processing time of the most urgent order - the overdue time of the most urgent order - the existing calculated unloading waiting time of the calculated unloading detection device".

4. The method for collaborative dynamic scheduling of heterogeneous resources in a workshop according to claim 1, characterized in that: The intelligent decision-making for production equipment selection is specifically as follows: The state space of production equipment selection is expressed as {[production buffer capacity, sum of operation time of all orders in the production buffer, sum of waiting time of all orders in the production buffer, sum of remaining time of all orders in the production buffer, production equipment occupancy time]*M+[expected processing time, order overdue time]}; The probability distribution of the action space includes {standby, select production equipment 1, ..., select production equipment M}; The reward for production equipment selection is the actual completion time of the order minus the overdue time; and / or, The intelligent decision-making for buffer order selection in the production stage is specifically as follows: The buffer order selection state space of the production equipment is expressed as {total time of unproduced orders, [expected processing time of unproduced orders, overdue time of unproduced orders, maximum task time for accepting unloading]* , minimum waiting processing time (equipment occupancy time)}; Indicates the total number of orders that the production equipment buffer can accommodate; The action space is [select the order that enters the production buffer earliest, select the order that enters the production buffer latest, select the order with the shortest processing time, select the order with the longest processing time, select the order with the most urgency]; The reward for order selection in the production stage is the actual completion time of the order minus the overdue time; and / or, The intelligent decision-making for selecting the detection equipment is specifically: The task issuing agents in all detection phases use and maintain the same training model; During the detection phase, the order may offload part of the computing workload to other detection devices. According to the negotiation mechanism, it may be offloaded to a single detection device. In this case, the local detection computing workload needs to be split into the tasks to be offloaded to other detection devices, and the transmission overhead of the offloaded tasks and the offloaded computing workload from other detection devices will be increased. and / or, The intelligent decision-making for buffer order selection in the detection phase is specifically as follows: The rewards for selecting the testing equipment during the testing phase include rewards for order testing and negotiated price increases, as well as rewards for undertaking computing offloading tasks during the duration of the order; and / or, The intelligent decision-making for selecting uninstall detection equipment is as follows: The status space of the uninstall detection device is [(minimum overdue time for undetected orders, minimum waiting time, uninstalled waiting time)* ]; Indicates the total number of orders that the detection device buffer can accommodate; All compute offload publishing agents use and maintain the same model; The probability distribution of the action space includes {standby, select detection device 1, ..., select detection device N}; the standby action indicates that there is no device that meets the offloading condition and is not suitable for computing offloading; The rewards for equipment selection during the detection phase include rewards for order production affected by offloading and rewards for undertaking the offloading volume, as well as rewards for undertaking computational offloading tasks during the duration of the order.

5. The method for collaborative dynamic scheduling of heterogeneous resources in a workshop according to any one of claims 1 to 4, characterized in that: When all the detection equipment are found to be unsuitable for computational offloading, the process also includes: The task publishing agent is required to perform mandatory allocation of orders; the task publishing agent executes the device selection decider to decide the most suitable detection device, insert the order into the buffer of the detection device, mark the order, and when the order completes the detection, no additional negotiation timeout penalty will be received.

6. A workshop heterogeneous resource collaborative dynamic scheduling system, characterized in that: The system is used to execute the workshop heterogeneous resource collaborative dynamic scheduling method according to any one of claims 1 to 5; The system is applied to a manufacturing workshop, and the manufacturing workshop manufactures products including a production stage and a testing stage. The system includes: Production equipment corresponding to the production stage; Testing equipment corresponding to the testing stage; Reasoning terminals connected to production equipment and testing equipment respectively; A server connected to the inference terminal, wherein the server is divided into a virtual machine as a task publishing agent, and the task agent is used to receive order tasks dynamically entered in a random form; Production agent for corresponding production equipment; Detection agent corresponding to the detection equipment; The first-tier contract negotiation network includes the first and second rounds of negotiation. The first round of negotiation is used to bid for production equipment and testing that can handle orders, and the second round of negotiation is used to increase the price and offer rewards for testing equipment that can handle urgent orders. The second-tier contract negotiation network negotiates and allocates additional rewards according to the amount of tasks that can be offloaded, bids for testing equipment that can perform computational offloading, and reduces production delays through parallel computing. The intelligent decision maker for equipment selection gives appropriate equipment selection results based on the equipment and order status of the workshop and guided by historical rewards; The intelligent decision maker for buffer order selection gives appropriate order selection results based on the buffer order status and equipment occupancy status, guided by historical rewards; Uninstall the intelligent decision maker for equipment selection, and give appropriate equipment selection results based on equipment status and order status, guided by historical rewards; The model parameter update module uses the DQN network parameter update method to update the target network parameters and value network parameters according to the number of decision executions; it also maintains a replay pool to record state-action parameters, listens to order execution results, and stores the results in the corresponding order location.

7. A device, characterized in that: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for collaborative dynamic scheduling of heterogeneous resources in a workshop as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for collaborative dynamic scheduling of heterogeneous resources in a workshop according to any one of claims 1 to 5 are implemented.

Images