A cloud manufacturing scheduling method and system based on green benefit ratio

Abstract

This invention proposes a cloud manufacturing scheduling method and system based on green efficiency ratio, belonging to the field of cloud manufacturing service scheduling technology, to solve existing multi-objective optimization methods. The method includes: acquiring scheduling data of order tasks from the cloud platform; defining each device as a different agent based on the Independent DQN architecture; constructing a state space vector and action space; detecting device faults; calculating energy efficiency variation values ​​if a fault occurs; triggering rescheduling when the variation exceeds a preset threshold; transferring affected processes to available equipment; using the green efficiency ratio as the comprehensive optimization objective; using its change in adjacent time intervals as immediate rewards; and training multiple agents; iterating cyclically until all processes are completed, outputting the optimal scheduling scheme. This invention transforms multiple objectives into a single green efficiency ratio, avoiding subjective weighting and improving the adaptability and accuracy of cloud manufacturing scheduling.

Description

Technical Field

[0001] This invention belongs to the field of cloud manufacturing service scheduling technology, specifically relating to a cloud manufacturing scheduling method and system based on green benefit ratio. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Cloud manufacturing service scheduling addresses the needs of multiple customers within a cloud service platform. By utilizing scheduling algorithms and dynamic adjustment strategies, it manages distributed machine tools and allocates manufacturing tasks, significantly improving a company's market responsiveness and competitiveness.

[0004] However, cloud manufacturing service scheduling needs to consider multiple objective factors such as economic cost, production efficiency and environmental impact. These objectives influence each other and may even have contradictory relationships. However, most existing multi-objective optimization methods use weighted summation or constraint transformation to perform multi-objective optimization. Such methods cannot describe the coupling relationship between multiple objectives and are prone to problems such as objective imbalance and deviation from the optimal solution in scheduling results. Summary of the Invention

[0005] This invention provides a cloud manufacturing scheduling method and system based on green benefit ratio, which solves the problem that existing multi-objective optimization methods are prone to scheduling result imbalance and deviation from the optimal solution.

[0006] To address the above problems, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a cloud manufacturing scheduling method based on green benefit ratio, comprising: Obtain scheduling data for cloud platform order tasks. The scheduling data includes a set of equipment, a set of manufacturing enterprises, a set of workpieces, and a set of processes. Based on the Independent DQN architecture, each device in the set of equipment is defined as an agent. The decision model of each agent is a deep Q-network. Based on the scheduling data, construct the state space vector and action space of each agent at time t. Determine whether a device malfunctions based on the current state space vector of the agent; If a fault occurs, the energy efficiency change value is calculated. When the energy efficiency change value is greater than or equal to a preset threshold, a rescheduling mechanism is triggered to allocate the affected process to other available agents in the equipment set. The other available agents are used as the current agents. The state space vector of the current agent at time t+1 is updated and an action is selected and executed. If no fault occurs or the energy efficiency change value is less than the preset threshold, select the original action and execute it, and update the state space vector of the current agent at time t+1. The green benefit ratio is calculated based on the state space vectors at time t and t+1, and the change in the green benefit ratio is used as an immediate reward. The state space vector, action, and immediate reward at time t, and the state space vector and completion flag at time t+1 are stored as interaction experience, and the parameters of the deep Q network are updated based on the interaction experience. Repeat the above process until all processes are completed, and use the decision results of each agent as the final scheduling scheme.

[0007] Furthermore, the calculation of the green benefit ratio is as follows:

[0008] in, GER t Indicates time The green benefit ratio U Indicates equipment utilization rate. U min and U `max` represents the minimum and maximum values ​​of equipment utilization during the normalization process, respectively. Q Indicates service quality. Q min and Q max These represent the minimum and maximum values ​​of service quality during the normalization process, respectively. R Indicates service reliability. R min and R max These represent the minimum and maximum values ​​of service reliability during the normalization process, respectively. T Indicates service time. T min and T max These represent the minimum and maximum service times during the normalization process, respectively. C Indicates service cost, C min and C max These represent the minimum and maximum values ​​of the service cost during the normalization process, respectively. E Indicates service energy consumption. E min and E max These represent the minimum and maximum values ​​of service energy consumption during the normalization process, respectively. Energy efficiency variation values ​​are calculated using the following formula:

[0009] in, Indicates the cumulative energy consumption deviation. Indicates the cumulative cost deviation. Indicates the cumulative time deviation; The state space vector includes the agent's processing time, current utilization, queue length of pending processes, fault status, remaining processing time of the workpiece, and the ratio of average waiting time to the average waiting time of all processes. The action specifically involves selecting a process from the current agent's processing queue, and the action space consists of the set of all process positions in the processing queue. An ε-greedy strategy is used for action selection and execution. The ε-greedy strategy sets a high exploration rate in the early stage of training, and ε gradually decays exponentially as training iterations proceed. The deep Q-network introduces a target network, and a dual-network structure consisting of an online Q-network and a target Q-network is constructed for each agent. The online network is responsible for action value estimation and parameter updates, while the target network is responsible for calculating the target Q-value and gradually synchronizing the network parameters using a soft update method. The second aspect of the present invention provides a cloud manufacturing scheduling system based on green benefit ratio, including a data acquisition and initialization module, a rescheduling module, a green benefit ratio calculation module, an experience storage and training module, and a scheduling output module; The data acquisition and initialization module is configured to: acquire scheduling data of order tasks from the cloud platform, the scheduling data including a set of equipment, a set of manufacturing enterprises, a set of workpieces and a set of processes; define each device in the set of equipment as an agent based on the Independent DQN architecture; the decision model of each agent is a deep Q network; and construct the state space vector and action space of each agent at time t based on the scheduling data. The rescheduling module is configured to determine whether a device has malfunctioned based on the current agent's state space vector. If a fault occurs, the energy efficiency change value is calculated. When the energy efficiency change value is greater than or equal to a preset threshold, a rescheduling mechanism is triggered to allocate the affected process to other available agents in the equipment set. The other available agents are used as the current agents. The state space vector of the current agent at time t+1 is updated and an action is selected and executed. If no fault occurs or the energy efficiency change value is less than the preset threshold, select the original action and execute it, and update the state space vector of the current agent at time t+1. The green benefit ratio calculation module is configured to calculate the green benefit ratio based on the state space vectors at time t and time t+1, and use the change in the green benefit ratio as an immediate reward. The experience storage and training module is configured to: store the state space vector, action, immediate reward at time t and the state space vector and completion flag at time t+1 as interaction experience, and update the parameters of the deep Q network based on the interaction experience; The scheduling output module is configured to repeat the above process until all processes are completed, and use the decision results of each agent as the final scheduling scheme.

[0010] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a cloud manufacturing scheduling method based on green efficiency ratio as described in the first aspect of the present invention.

[0011] The fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a cloud manufacturing scheduling method based on green efficiency ratio as described in the first aspect of the present invention.

[0012] The above one or more technical solutions have the following beneficial effects: The green benefit ratio calculation proposed in this invention transforms multi-objective optimization into a single ratio of efficiency output and resource consumption, avoiding the highly subjective weighted summation of multiple objectives. At the same time, the ratio of the two includes the inherent constraint relationship between economic benefits and environmental objectives, reflecting the characteristics of mutual constraints due to conflict between the two, improving the accuracy of cloud manufacturing scheduling schemes, enabling cross-enterprise collaborative manufacturing to achieve the optimization of unit resource input under a unified green benefit index, and providing low-carbon, economical, and efficient green scheduling capabilities for cloud manufacturing services.

[0013] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0014] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0015] Figure 1 This is an overall flowchart of the present invention; Figure 2 This is the original scheduling Gantt chart of the present invention without any faults; Figure 3 This is a Gantt chart showing that only the affected process is shifted to the right after a failure occurs in this invention; Figure 4 This is a Gantt chart for rescheduling after a failure in this invention. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0018] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0019] Example 1 This embodiment discloses a cloud manufacturing scheduling method based on green benefit ratio. like Figure 1 As shown, a cloud manufacturing scheduling method based on green efficiency ratio includes: Step S1: Obtain the scheduling data of order tasks from the cloud platform; The cloud platform accepts order tasks submitted by customers. Each customer submits one order, and each order contains one workpiece to be processed. The order task includes n workpieces to be processed, i.e., a set of workpieces. Each workpiece needs to go through multiple processes. O ij Indicates workpiece J i The j-th process, workpiece J i process set , i.e., workpiece J i There are a total of S processes, of which i =1,2,....,n; The cloud manufacturing platform also includes a collection of equipment. These devices are distributed in L In a collection of different manufacturing enterprises and factories, that is, a group of manufacturing enterprises Each enterprise E k The included device subset is Each process can be completed by different machine tools. Indicates workpiece J i The set of processing equipment that can be selected for the j-th process. .

[0020] In addition, dynamic scheduling in a cloud manufacturing environment must also meet the following constraints and assumptions: 1) Assume the process route for the workpiece has been determined; 2) The set of optional equipment for each process is known; 3) There is no temporal dependency between workpieces, and they can be processed and manufactured in parallel. However, the processes of the same workpiece must meet the sequential constraints, that is, the next process can only be started after the previous process is completed; 4) All available processing equipment shall run continuously from the first process of the first workpiece until the last process of the last workpiece is completed before shutting down. No interruption is allowed during this period (except for sudden equipment failure). 5) A single machine can only process one operation at a time; 6) Each process can only be performed by one piece of equipment from its set of selectable equipment; 7) Machine tool equipment occurs with a certain probability of failure (the probability distribution of the failure probability is known based on historical statistical values); Assuming the type of equipment failure occurring is the most common in the factory, the average repair time and cost per repair can be calculated based on historical statistics.

[0021] Based on the scheduling data obtained above, this embodiment defines the scheduling data in more detail, as shown in Table 1, for use in the subsequent construction of state space vectors and calculation of green benefit ratio.

[0022] Table 1 Parameters and their definitions

[0023] Because cloud manufacturing services are characterized by the wide distribution of manufacturing resources, diverse participants, and complex personalized needs, they are essentially a typical NP-hard problem. As production scales up, computational complexity and computation time costs rise sharply, and traditional algorithms are difficult to adapt to real-time scheduling requirements. Therefore, this embodiment specifically adopts a multi-agent scheduling algorithm framework based on deep Q-network (DQN).

[0024] Step S2: Based on the Independent DQN architecture, each device in the device set is defined as an agent, and the decision model of each agent is a deep Q network. Based on the scheduling data, the state space vector and action space of each agent at time t are constructed. Each machine tool, acting as an independent intelligent agent, constructs a state space vector to reflect its production environment. This vector characterizes the operating environment of a single machine at the current moment, thus reflecting the dynamic operating environment of machine tools in a distributed workshop. This provides feedback for the intelligent agent to learn scheduling strategies and optimize overall green efficiency. Mu The state-space vector at time t is constructed as follows: (1) in, P u ( t This represents the processing time of the machine tool, used to record the total cumulative processing time completed by the machine tool from the start of scheduling to time t. It is calculated by summing the times of all processing operations performed by the machine tool, providing a clear reflection of the equipment's historical workload and offering fundamental data for load balancing during scheduling. The specific formula is: (2) in, , indicating process O ij The machining is completed on machine tools, and the completion time is... ;otherwise ; U u ( t The current utilization rate of a machine tool is represented by the percentage of cumulative processing time of a single machine tool at the current time t. It is used to depict the real-time busy / idle status of the machine tool. The specific formula is: (3) L u ( t The denoted represents the queue length of the machine tool's pending operations, which is the number of unprocessed operations waiting on that machine tool at time t. Let a certain machine tool... M u The queue of processes at time t K u ( t )for: (4) in, p Indicates the number of elements in the queue. p An unprocessed step, i.e., a location index. This represents the first unprocessed step in the queue; it belongs to the workpiece. The j-th process, This indicates the number of operations to be processed in the queue at time t. Therefore, the machine tool... M u The length of the queue of processes to be processed can be expressed as: (5) f u ( tThis indicates the fault state of the machine tool equipment. At time t, the machine tool equipment... M u The fault state is represented by a binary variable to reflect the impact of equipment availability on scheduling decisions. The specific formula is as follows: (6) RT i ( t Let represent the remaining processing time for each workpiece. At time t, the sum of the processing times for the unprocessed steps within each workpiece is used to measure the total processing workload required for the subsequent steps of the task. Let the workpiece... The set of processes that have not yet been completed at time t is: (7) in This represents the set of indices for incomplete operations. Therefore, the workpiece... The remaining processing time is: (8) W ( t The denoted represents the average waiting time ratio across all processes. At time t, for all processes that have arrived and entered the waiting or processing queue in the scheduling process, the ratio of their actual waiting time to the current global time t is calculated and averaged across all processes. Specifically, the waiting time of a process is determined by the difference between its start processing time and the completion time of the preceding process. This indicator reflects the average queuing delay of each process during dynamic scheduling, and its formula is as follows: (9) The agent's decision-making involves selecting a processing step from the queue of machines with processing queues. The action space is defined as a finite set of all processing positions in the current queue of the machine. Specifically, let the machine tool equipment... M u The queue of processes at time t is K u ( t Then the action space It consists of all the process positions in the queue, and each action corresponds to selecting a specific process for processing. There is an upper limit to the queue length. Therefore, the action dimension of each device is uniformly limited to within, when L u ( t When )=1, directly select that position; when L u ( tWhen )=0, no selection is made and the device is skipped. When the actual queue length is insufficient, the action space is padded with zeros to ensure the dimensionality consistency of the input and output of the deep Q-network. During execution, the agent follows the state space vector. Choosing actions with an ε-greedy strategy This determines the scheduling order of processes on the machine tool, and subsequently updates the green benefit ratio. Specifically, at each decision moment, the agent first inputs the current state vector into the online Q-network. The network outputs the Q-values ​​corresponding to all candidate actions in the current action space, i.e., each process to be processed corresponds to a Q-value. Then, an ε-greedy strategy is used to select actions: with a probability of 1... ε selects the action with the highest Q-value, that is, prioritizes the action that the network currently considers to have the highest benefit. It also randomly selects an action within the current action space with probability ε to increase policy exploration capability and avoid getting stuck in local optima in the early stages of training. As training progresses, ε gradually decreases exponentially, allowing the agent to gradually transition from random exploration in the early stages to optimal policy utilization in the later stages.

[0025] For each agent, a dual-network structure consisting of an online Q-network and a target Q-network is constructed. The online network is responsible for action value estimation and parameter updates, while the target network is responsible for calculating the target Q-value. The network parameters are gradually synchronized using a soft update method, thereby alleviating the estimation bias caused by frequent changes in the target value during the training process of traditional DQN. The deep Q-network consists of an input layer, hidden layers, and an output layer. The input layer receives the state space vector of the current machine tool agent, which includes state information such as processing time, equipment utilization, queue length, fault status, and average waiting time. The hidden layer uses a fully connected neural network to extract state features through a nonlinear activation function. The output layer outputs the Q-values ​​corresponding to each candidate action in the current action space. Each output node corresponds to a scheduling action, i.e., selecting a process from the current queue to be processed. The agent selects the final action to execute based on the Q-values ​​obtained from the output layer and an ε-greedy strategy.

[0026] Furthermore, in cloud manufacturing services, machine tool equipment failure is a highly unpredictable and destructive problem. When equipment failure occurs, it affects the initial scheduling scheme of the cloud service platform and the progress of enterprise tasks. In existing rescheduling schemes, rescheduling is triggered immediately whenever equipment fails, and the energy consumption of machine tools during state transitions such as processing, idle, and start-up / shutdown is ignored. This scheduling method not only significantly increases computational costs but also frequently disrupts the originally optimal global process arrangement, easily leading to excessive rescheduling. Therefore, this application adopts the rescheduling method in steps S3-S4.

[0027] Step S3: Determine whether the device has malfunctioned based on the current state space vector of the agent; The machine tool equipment failures considered in this embodiment refer to random failures, mainly manifested as mechanical failures such as the failure of components like lead screws and guide rails, and electrical failures such as malfunctions of control components and sensors in the electrical system. These failures are typically irregular and recoverable. The occurrence of machine tool equipment failures will cause the operating environment of the original scheduling scheme to exhibit significant randomness, nonlinearity, and strong dynamic characteristics.

[0028] Based on the fault state variables in the state space vectors of each agent f u ( t Perform fault diagnosis when f u ( t When )=1, it indicates that the device M u The fault occurs at time t, when f u ( t When )=0, it indicates that the equipment is operating normally. If a fault is detected, proceed to step S4 to calculate the energy efficiency change value and determine the rescheduling; if no fault is detected, directly execute the original action and update the status.

[0029] Step S4: Calculate the energy efficiency change value, determine whether the rescheduling mechanism is triggered, and execute the action. A rescheduling mechanism based on manufacturing energy efficiency variations is adopted. Threshold comparisons determine whether machine tool rescheduling should be performed after a disturbance event, ensuring that the energy consumption, cost, and time losses to the machining system are within acceptable limits. When a machine tool fails, the energy efficiency variation value determines whether to maintain the original schedule. One option is for the workpiece to wait for the faulty machine to be repaired and continue processing on that machine. Another option is that the original scheduling scheme fails, requiring the workpiece to be rescheduled to another available machine for processing. The energy efficiency variation value is calculated as follows: (10) Among them, , , These represent the deviations in cumulative energy consumption, cost, and time, respectively. These deviations arise from the difference between the original scheduling scheme and the original scheduling scheme in which the machine tool continues processing without changing the processing steps after the fault is repaired, and the original scheduling scheme in which the baseline object has not experienced a fault.

[0030] Indicates energy consumption per unit cost. Indicates energy consumption per unit time; After the energy efficiency variation value is calculated, it is compared with a preset threshold δ. Preferably, the preset threshold δ is set to 22. When the calculated energy efficiency variation value is lower than the preset threshold, it is determined that the impact of the fault is small, the original scheduling scheme remains unchanged, and processing continues after the equipment is repaired, and the state space vector of the current agent at time t+1 is updated. When the change value exceeds the threshold, the fault has a significant impact, triggering a rescheduling mechanism to transfer the affected process to other optional equipment with the same processing capacity. At this time, the state space vector at time t+1 is updated, and an action is selected and executed.

[0031] Step S5: Calculate the green benefit ratio based on the state space vectors at time t and time t+1, and use the change in the green benefit ratio as the immediate reward; The green benefit ratio refers to the ratio of efficiency output indicators to resource consumption indicators, reflecting the comprehensive green benefits generated per unit of resource input by an enterprise. The basic expression is as follows: (11) This comprehensive ratio transforms multi-objective combination optimization into a single ratio value, effectively avoiding the subjective weighted summation of multiple objectives and the difficulty in balancing conflicting indicators. Since the six green benefit indicators have different units, they need to be normalized to eliminate dimensions. The detailed process is as follows: Service time T refers to the entire time period from when an enterprise receives a batch of customer orders through a cloud platform, executes machine tool production scheduling, until all pending tasks are completed in a distributed manner through collaborative manufacturing and logistics transportation. Service time is determined by the longest-running task and mainly includes processing time, internal and external logistics time, and machine tool failure repair time.

[0032] (12) (13) (14) Service cost C refers to the various costs incurred by a distributed enterprise in scheduling machine tools to collaboratively complete order processing tasks, mainly including machine tool processing costs. C 1. Transportation costs C 2 and failure costs C 3. The processing cost includes raw material costs and process costs; inter-enterprise transportation costs are calculated based on existing ordinary parcel rates and are borne by the sending and receiving cities. D The corresponding initial weight price (a), subsequent weight price (b), and workpiece weight The decision is made jointly. Intra-enterprise transportation costs are calculated based on the unit distance transportation cost of the AGV (Automated Guided Vehicle) and the workshop spacing. Failure costs include repair costs and delay costs incurred beyond the planned service completion time. Equipment failure rate can be calculated based on the number of failures occurring in the workshop over a period of time.

[0033] (15) (16) (17) (18) (19) Service energy consumption E refers to the energy consumption of machine tools and logistics equipment during the execution of all processing tasks, mainly including processing energy consumption. E 1. Idle energy consumption E 2 and transportation energy consumption E 3. Set the processing power of each type of machine tool equipment as A of the rated power value, and the no-load power as B of the rated power. Preferably, A is 30% and B is 70%. Transportation energy consumption mainly includes two parts: the power consumption generated by AGV automated logistics vehicles within the enterprise; and the transportation energy consumption between enterprises, which can be calculated based on the average lower heating value of energy used by various transportation modes in the existing energy statistics yearbook.

[0034] (20) (twenty one) (twenty two) (twenty three) Equipment utilization rate U is calculated by summing the actual processing times of all machine tools and then dividing by the total service time. It reflects the overall utilization of machine tools in the scheduling system, rather than focusing solely on the performance of individual machines.

[0035] (twenty four) Service quality (Q) refers to the service quality of an individual task, calculated by the pass rate of each processing step, and then averaged across all tasks to measure the overall service quality and reflect the overall performance of the scheduling system. The pass rate of a processing step can be calculated based on the historical number of pass processing steps on the machine tools in the workshop and the total number of processing steps.

[0036] (25) Service reliability R is calculated by weighting the on-time completion rate and the failure-free rate for each process, then averaging the reliability of each process to obtain the overall reliability of a single task, and finally averaging the reliability of all tasks to reflect the reliability of the entire service process. The on-time completion rate of the processing steps can be calculated based on the number of processes historically completed on time by the machine tools in the workshop and the total number of processing steps.

[0037] (26) Substituting the formulas (12)-(26) for the six green benefit indicators above into formula (11), we obtain the final formula for calculating the green benefit ratio: (27) Using the green benefit ratio between adjacent time steps as the reward function for the deep Q-network, the GER calculation employs a normalization method based on six key indicators, eliminating the influence of dimensions through dynamic normalization of historical data's minimum and maximum values. By perceiving the magnitude of the GER, the agent can clearly determine the degree of impact of its scheduling decisions on the overall green benefit, thus being guided to explore optimization strategies that can improve efficiency-output indicators while reducing resource-consumption indicators. The GER-based reward mechanism aligns the agent's learning objective with the system's optimization objective, and provides precise learning signals for the optimization direction through changes in the GER.

[0038] Step S6: Store the state space vector, action, immediate reward at time t, and the state space vector and completion flag at time t+1 as interaction experience, and update the parameters of the deep Q network based on the interaction experience; The agent represents the interaction experience at each moment as a quintuple consisting of a state space vector, an action, an immediate reward, and the state space vector at time t+1 and a completion flag. Store to based In the experience replay pool of the structure, the experience pool adopts a first-in-first-out management strategy. When the buffer is full, the old experience will be automatically removed to make room for new experience, so as to maintain the timeliness and diversity of the sample, covering multiple machine tools, different task structures and other working conditions, so as to make decisions.

[0039] Each agent uses a deep Q-network, and a target network is introduced to alleviate the overestimation of Q-values ​​in traditional DQN. The structure of the target network is the same as that of the current network, and its parameters are copied and updated from the current network every fixed number of steps C.

[0040] After each training round, the system checks the experience pool capacity of each agent. When the number of samples reaches the preset batch size, the learning update process is triggered, using a Q-value update method based on temporal difference (TD). After each training round, the system checks the number of samples in the experience replay pool of each agent. When the number of samples reaches the preset batch size, a small batch of samples is randomly sampled from the experience pool. Network training is then performed. First, an online network is used to calculate the current state-action value. Then, the target network is used to calculate the target Q value for the next state, and a time difference (TD) target value is constructed: (28) Subsequently, a loss function is constructed using the time difference error (TD Error) between the Q-value output by the online network and the target Q-value, and the parameters of the online network are updated through the backpropagation algorithm. Finally, the parameters of the online network are synchronized to the target network to ensure that the target Q-value changes relatively smoothly, thereby improving the stability and convergence performance of the training process.

[0041] Step S7: Repeat steps S3-S6 until all processes of all workpieces are completed, and obtain the final scheduling scheme.

[0042] Case Analysis The cylinder barrel of a crane's luffing cylinder is the main pressure-bearing component of the hydraulic cylinder. The luffing cylinder controls the elevation angle of the crane boom, while the cylinder barrel serves as the main pressure vessel, housing the piston, seals, and other components, and bearing high hydraulic pressure. These cylinder barrels are widely used in construction machinery such as truck cranes, crawler cranes, tower cranes, and gantry cranes. Due to differences in structural forms, various types of cylinder barrels differ significantly in material selection, stress characteristics, and maintainability, resulting in diverse manufacturing processes.

[0043] The cloud platform received six orders from customers for different crane luffing cylinder barrels to be processed. The workpieces are: integral cold-drawn tube processing (workpiece 1), integral hot-rolled tube processing (workpiece 2), welded sheath structure cold-drawn tube processing (workpiece 3), welded sheath structure hot-rolled tube processing (workpiece 4), welded flange structure processing (workpiece 5), and integral flange structure processing (workpiece 6). The process routes are shown in Table 2.

[0044] Table 2. Manufacturing process routes for the cylinder barrels of six types of crane luffing cylinders

[0045] The machine tools are categorized into seven types based on process requirements: lathes, sawing machines, drilling machines, shot blasting machines, welding machines, straightening machines, and honing machines. These seven types of equipment are distributed across three companies. Table 3 shows the processing operations of each machine tool and the distribution of the number of machines in each company, totaling 30 machine tools. Each company has its workshop layout based on process nature and equipment function. Workshop A is allocated to lathes, sawing machines, and drilling machines; workshop B is allocated to welding machines and straightening machines; and workshop C is allocated to shot blasting machines and honing machines. The no-load power of the machine tools can be calculated empirically using fixed power values. The no-load power accounts for a large proportion of the total machine power (approximately 17%~83%), and the standby power is even higher for complex and large machine tools. Auxiliary power can also be calculated empirically using fixed power values, mainly including the power of auxiliary devices such as cutting fluid spraying, tool changing, and chip removal (accounting for approximately 1%~46%). Therefore, it is assumed that the processing power of each machine tool is 30% of its rated power, and the no-load power is 70% of its rated power.

[0046] Table 3. Number of Machine Tools and Service Capabilities of the Three Companies

[0047] All three companies are assumed to be located within the same administrative region. The transfer of workpieces within the distributed workshop includes both inter-company and intra-company transport. Inter-company transport utilizes medium-sized trucks, with transportation costs calculated based on China Post's domestic ordinary parcel rates. Intra-company transport only considers the use of AGVs (Automated Guided Vehicles) between different workshops.

[0048] Table 4, using workpiece 4 as an example, shows the basic machine tool processing data used for scheduling calculations. This data is based on the actual value range (maximum-minimum range) of each processing parameter provided by the enterprise. For the sake of simplifying the model, the value range of each processing parameter is regarded as an empirical interval that follows a uniform distribution, and simulation data is generated by random sampling within this interval.

[0049] Table 4. Process-Machine Tool Machining Data Information for Workpiece 4

[0050] The relevant parameter settings for the embodiments are shown in Tables 5 and 6. The raw material cost, AGV speed, single-failure repair cost, and workshop distance are data from the enterprises; the distances between the three enterprises and the weight of raw materials are assumed; the AGV energy consumption per unit time and transportation cost are statistical averages; according to equipment reliability theory, machine tool failures can generally be considered as a random process following an exponential distribution. The number of mechanical and electrical failures occurring in the machine tool within a certain period is taken as the prior probability to describe the likelihood of random failures occurring within a unit of time; the transportation energy consumption per unit distance is assumed to be the diesel consumption generated by truck transportation within the province. The fuel consumption per 100 kilometers of trucks is calculated based on the average lower heating value of diesel fuel using existing energy statistics yearbooks to obtain the corresponding energy consumption per unit distance.

[0051] Table 5 Scheduling Model and Training Parameter Settings

[0052] Table 6 Hyperparameters of the training algorithm

[0053] Furthermore, we can obtain the following: Figure 2 The original scheduling Gantt chart shown is for when no machine tool failure occurs. Figure 2 An initial scheduling scheme was presented, involving 30 machines (belonging to companies E1, E2, and E3) processing 6 workpieces and 58 operations. Machine resources from different companies were effectively utilized during the scheduling process. Machines on E1 and E2 had a relatively denser distribution of operations, handling a larger number of workpieces. Looking at the workpiece processing paths, it was observed that many operations for some workpieces were concentrated within the same company. The scheduling strategy tended to reduce unnecessary cross-company processing to lower transfer costs and scheduling complexity. However, several typical workpieces had different operations assigned to different companies and workshops, such as workpieces 4, 5, and 6, whose processing spanned multiple companies. Overall, this scheduling scheme achieved a reasonable allocation and effective utilization of manufacturing resources under multi-company collaboration conditions. Ultimately, the cloud manufacturing system completed the processing of all workpieces in approximately 1159 minutes.

[0054] To determine the threshold for energy efficiency variation that triggers rescheduling, simulation experiments were conducted under different machine tool fault durations and probabilities. Through multiple rounds of simulation and calculation of energy efficiency variation, the average energy efficiency variation under common fault disturbances was found to be approximately 22. Therefore, [the following is a continuation of the previous sentence, but the context is unclear]. This serves as the rescheduling trigger threshold. When the change in manufacturing system energy efficiency caused by a machine tool failure exceeds this threshold, the dynamic rescheduling mechanism should be activated to reallocate unprocessed operations to the optimal machine tool.

[0055] Through machine tool failure uncertainty simulation, machine tools M13, M14, and M15 failed at time 900.52. Figure 3 As shown, the directly affected processes are marked with red boxes: Jobs 5-8, 4-9, and 6-8; the indirectly affected processes are marked with blue boxes: Jobs 6-9, 5-9, and 4-10. The initial scheduled completion time was 1159 minutes. If the plan is to wait for the machine tools to be repaired before continuing processing without changing the processing sequence, then the processes must be strictly shifted to the right. Due to downtime and process priority constraints, the system completion time will be delayed. The completion time increases to 1184.52 minutes. Compared to the initial scheduling plan, the time variable... =25.52 minutes, energy consumption variable =593.24MJ, cost variable =1200 yuan. Since the delivery date has not been exceeded and there is no delay cost, only the fixed repair cost for a single failure needs to be added. Therefore, the manufacturing energy efficiency variation is... .visible The system will reject the initial scheduling and trigger the rescheduling mechanism, which will reassign the unprocessed processes to other optional equipment sets for processing, so that the energy consumption, cost, and time loss caused to the processing system are within an acceptable range.

[0056] When the rescheduling mechanism is triggered, such as Figure 4 As shown, all unfinished processes after minute 900.52 were rescheduled, resulting in a service time of 1187.37 minutes, a service cost of 8868.65 yuan, and a service energy consumption of 37309.145 MJ. Compared to the initial scheduling scheme, the time variable... =28.37 minutes, cost variable =1293 yuan, energy consumption variable =586.45MJ, Compared to the strategy of simply shifting the affected process to the right after a failure, the service time increased by 0.24%, service energy consumption decreased by 0.018%, service cost increased by 1.06%, and manufacturing energy efficiency increment decreased by 9.16%. The rescheduling scheme effectively reduced the adverse impact of equipment failure uncertainty on the overall green benefit ratio, kept manufacturing energy efficiency fluctuations within the threshold range, and improved the adaptive scheduling capability for uncertain equipment failure events in the cloud manufacturing distributed workshop environment.

[0057] Example 2 This embodiment discloses a cloud manufacturing scheduling system based on green benefit ratio, including a data acquisition and initialization module, a rescheduling module, a green benefit ratio calculation module, an experience storage and training module, and a scheduling output module; The data acquisition and initialization module is configured to: acquire scheduling data of order tasks from the cloud platform, the scheduling data including a set of equipment, a set of manufacturing enterprises, a set of workpieces and a set of processes; define each device in the set of equipment as an agent based on the Independent DQN architecture; the decision model of each agent is a deep Q network; and construct the state space vector and action space of each agent at time t based on the scheduling data. The rescheduling module is configured to determine whether a device has malfunctioned based on the current agent's state space vector. If a fault occurs, the energy efficiency change value is calculated. When the energy efficiency change value is greater than or equal to a preset threshold, a rescheduling mechanism is triggered to allocate the affected process to other available agents in the equipment set. The other available agents are used as the current agents. The state space vector of the current agent at time t+1 is updated and an action is selected and executed. If no fault occurs or the energy efficiency change value is less than the preset threshold, select the original action and execute it, and update the state space vector of the current agent at time t+1. The green benefit ratio calculation module is configured to calculate the green benefit ratio based on the state space vectors at time t and time t+1, and use the change in the green benefit ratio as an immediate reward. The experience storage and training module is configured to: store the state space vector, action, immediate reward at time t and the state space vector and completion flag at time t+1 as interaction experience, and update the parameters of the deep Q network based on the interaction experience; The scheduling output module is configured to repeat the above process until all processes are completed, and use the decision results of each agent as the final scheduling scheme.

[0058] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0059] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a cloud manufacturing scheduling method based on green efficiency ratio as described in Embodiment 1 of this disclosure.

[0060] Example 4 The purpose of this embodiment is to provide an electronic device.

[0061] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in a cloud manufacturing scheduling method based on green efficiency ratio as described in Embodiment 1 of this disclosure.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A cloud manufacturing scheduling method based on green benefit ratio, characterized in that, Obtain scheduling data for cloud platform order tasks. The scheduling data includes a set of equipment, a set of manufacturing enterprises, a set of workpieces, and a set of processes. Based on the Independent DQN architecture, each device in the set of equipment is defined as an agent. The decision model of each agent is a deep Q-network. Based on the scheduling data, construct the state space vector and action space of each agent at time t. Determine whether a device has malfunctioned based on the current state space vector of the agent; If a fault occurs, the energy efficiency change value is calculated. When the energy efficiency change value is greater than or equal to a preset threshold, a rescheduling mechanism is triggered to allocate the affected process to other available agents in the equipment set. The other available agents are used as the current agents. The state space vector of the current agent at time t+1 is updated and an action is selected and executed. If no fault occurs or the energy efficiency change value is less than the preset threshold, select the original action and execute it, and update the state space vector of the current agent at time t+1. The green benefit ratio is calculated based on the state space vectors at time t and t+1, and the change in the green benefit ratio is used as an immediate reward. The state space vector, action, and immediate reward at time t, and the state space vector and completion flag at time t+1 are stored as interaction experience, and the parameters of the deep Q network are updated based on the interaction experience. Repeat the above process until all processes are completed, and use the decision results of each agent as the final scheduling scheme.

2. The cloud manufacturing scheduling method based on green efficiency ratio as described in claim 1, characterized in that, The calculation of the green benefit ratio is as follows: in, GER t Indicates time The green benefit ratio U Indicates equipment utilization rate. U min and U `max` represents the minimum and maximum values ​​of equipment utilization during the normalization process, respectively. Q Indicates service quality. Q min and Q max These represent the minimum and maximum values ​​of service quality during the normalization process, respectively. R Indicates service reliability. R min and R max These represent the minimum and maximum values ​​of service reliability during the normalization process, respectively. T Indicates service time. T min and T max These represent the minimum and maximum service times during the normalization process, respectively. C Indicates service cost, C min and C max These represent the minimum and maximum values ​​of the service cost during the normalization process, respectively. E Indicates service energy consumption. E min and E max These represent the minimum and maximum values ​​of service energy consumption during the normalization process, respectively.

3. The cloud manufacturing scheduling method based on green efficiency ratio as described in claim 1, characterized in that, The energy efficiency variation value is calculated according to the following formula: in, Indicates the cumulative energy consumption deviation. Indicates the cumulative cost deviation. This indicates the cumulative time deviation.

4. The cloud manufacturing scheduling method based on green efficiency ratio as described in claim 1, characterized in that, The state space vector includes the processing time corresponding to the agent, the current utilization rate, the length of the queue of processes to be processed, the fault status, the remaining processing time of the workpiece and the ratio of the average waiting time of all processes.

5. The cloud manufacturing scheduling method based on green efficiency ratio as described in claim 1, characterized in that, The action specifically involves selecting a process from the current agent's processing queue, and the action space consists of the set of all process positions in the processing queue.

6. The cloud manufacturing scheduling method based on green benefit ratio as described in claim 5, characterized in that, An ε-greedy strategy is used for action selection and execution. In the early stages of training, a high exploration rate is set, and as the training iteratively progresses, ε gradually decays exponentially.

7. The cloud manufacturing scheduling method based on green efficiency ratio as described in claim 1, characterized in that, The deep Q-network incorporates a target network, constructing a dual-network structure consisting of an online Q-network and a target Q-network for each agent. The online network is responsible for action value estimation and parameter updates, while the target network is responsible for calculating the target Q-value and gradually synchronizing network parameters using a soft update method.

8. A cloud manufacturing scheduling system based on green efficiency ratio, characterized in that, It includes a data acquisition and initialization module, a rescheduling module, a green benefit ratio calculation module, an experience storage and training module, and a scheduling output module; The data acquisition and initialization module is configured to: acquire scheduling data of order tasks from the cloud platform, the scheduling data including a set of equipment, a set of manufacturing enterprises, a set of workpieces and a set of processes; define each device in the set of equipment as an agent based on the Independent DQN architecture; the decision model of each agent is a deep Q network; and construct the state space vector and action space of each agent at time t based on the scheduling data. The rescheduling module is configured to determine whether a device has malfunctioned based on the current agent's state space vector. If a fault occurs, the energy efficiency change value is calculated. When the energy efficiency change value is greater than or equal to a preset threshold, a rescheduling mechanism is triggered to allocate the affected process to other available agents in the equipment set. The other available agents are used as the current agents. The state space vector of the current agent at time t+1 is updated and an action is selected and executed. If no fault occurs or the energy efficiency change value is less than the preset threshold, select the original action and execute it, and update the state space vector of the current agent at time t+1. The green benefit ratio calculation module is configured to calculate the green benefit ratio based on the state space vectors at time t and time t+1, and use the change in the green benefit ratio as an immediate reward. The experience storage and training module is configured to: store the state space vector, action, immediate reward at time t and the state space vector and completion flag at time t+1 as interaction experience, and update the parameters of the deep Q network based on the interaction experience; The scheduling output module is configured to repeat the above process until all processes are completed, and use the decision results of each agent as the final scheduling scheme.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the cloud manufacturing scheduling method based on green efficiency ratio as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the cloud manufacturing scheduling method based on green efficiency ratio as described in any one of claims 1-7.

Images