Digital twin-driven reprocessing scheduling optimization method

Abstract

The invention belongs to the technical field of remanufacturing process, and discloses a digital twin-driven reprocessing scheduling optimization method, which comprises the following steps: initializing and generating a reprocessing initial scheduling scheme in a physical workshop, and moving the reprocessing initial scheduling scheme to a third window of a virtual workshop; based on a rolling window mechanism, moving the task which is being reprocessed currently and the task which is about to be reprocessed from the third window to the second window; if it is detected that the emergency task is inserted, all reprocessed tasks in the second window are moved to the first window; moving the emergency task to a second window; based on the target function of minimizing the total cost and the total time, a whale optimization algorithm is adopted for solving, and rescheduling schemes of all tasks in the second window are obtained; and sending the rescheduling scheme to the physical workshop for execution until no task needing rescheduling exists in the third window. According to the method, insertion of emergency tasks is considered, and rescheduling and response efficiency are considered.

Description

Technical Field

[0001] This invention belongs to the field of remanufacturing process technology, specifically relating to a digital twin-driven reprocessing scheduling optimization method. Background Technology

[0002] The remanufacturing process typically includes three core stages: first, the recycled product is disassembled in the disassembly workshop to obtain multiple components that can be used for subsequent processing; second, these components are reprocessed in the reprocessing workshop; and finally, the reassembled components are reassembled into a complete product in the reassembly workshop, with necessary performance optimizations performed. Among these, the reprocessing stage plays a crucial role in the remanufacturing process. Through the rational scheduling and optimization of production resources, it helps improve resource utilization, enhance production flexibility, and increase overall manufacturing efficiency. With the continuous development of the remanufacturing industry, the reprocessing scheduling problem has gradually become a research hotspot.

[0003] Current research on reprocessing scheduling mainly focuses on two major research directions: one is to explore the impact of different reprocessing production line or workshop configurations on the scheduling process, such as parallel assembly line reprocessing production lines, flexible workshop reprocessing workshops, and parallel and dedicated hybrid reprocessing production lines; the other is to explore the impact of various production disturbances on the scheduling process, such as random rework, new task insertion, and outsourcing.

[0004] These studies typically assume significant differences between components in different recycled products, thus requiring remanufactured products to wait for all components from the same original product to be reprocessed before assembly, resulting in substantial waiting times. However, in real-world remanufacturing scenarios, some components can be functionally restored after reprocessing, and there is no essential difference between components with only cosmetic differences (such as labels or colors). These components possess a degree of versatility, can be used across products for assembly, and do not require prior differentiation before reassembly. This processing strategy is called "difference delay." Through the difference delay strategy, reprocessed universal components can be pre-stored in inventory for immediate use, thereby saving reassembly waiting time.

[0005] Furthermore, remanufacturing processes often face the insertion of urgent tasks. To effectively schedule these urgent tasks, the remanufacturing scheduling model needs to possess high flexibility and dynamic response capabilities to achieve real-time scheduling updates. However, existing research on dynamic scheduling is mostly based on physical space, making it difficult to achieve rapid response to urgent tasks. Digital twin technology improves response speed by integrating physical and virtual spaces to achieve information interaction. Research shows that digital twins have been widely used to solve dynamic scheduling problems. Nevertheless, most existing digital twin-based dynamic scheduling methods employ global rescheduling strategies and rely on a fixed rolling frequency, making it difficult to balance rescheduling and system response efficiency.

[0006] The reprocessing scheduling problem, as a subproblem of remanufacturing scheduling, is a typical NP-hard problem. Swarm intelligence optimization algorithms have shown strong adaptability to this problem and have been widely applied by scholars. Besides traditional swarm intelligence optimization algorithms (such as ant colony optimization and artificial bee colony optimization), the whale optimization algorithm has attracted widespread attention due to its good solution space search capability and algorithmic simplicity. Currently, an improved whale optimization algorithm based on ensemble methods has been proposed to solve dynamic workshop scheduling models. There are also algorithms that combine whale optimization with genetic algorithms to solve production scheduling models more efficiently. Additionally, there is an improved whale optimization algorithm that combines adaptive weights and quadratic interpolation techniques to solve cloud remanufacturing scheduling models. However, the solution representation of the basic whale optimization algorithm is relatively simple, its adaptability to discrete problems is limited, and it is prone to getting trapped in local optima, making it difficult to directly apply to representing and updating complex solutions of models. Summary of the Invention

[0007] The purpose of this invention is to provide a digital twin-driven reprocessing scheduling optimization method that takes into account the insertion of urgent tasks and balances rescheduling and response efficiency.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A digital twin-driven reprocessing scheduling optimization method, based on digital twin technology, maps the physical workshop of reprocessing to generate a virtual workshop, and the virtual workshop has three windows. The digital twin-driven reprocessing scheduling optimization method includes:

[0010] Step 1: Initialize and generate the initial reprocessing schedule in the physical workshop, and move the initial reprocessing schedule to the third window of the virtual workshop;

[0011] Step 2: After the scheduled task begins execution, the virtual workshop executes a rescheduling strategy, including:

[0012] Step 2-1: Based on the reprocessing data collected from the physical workshop, move the currently reprocessing tasks and the tasks about to be reprocessed from the third window to the second window using a scrolling window mechanism;

[0013] Step 2-2: If no emergency task insertion is detected, proceed to step 2-5; otherwise, proceed to step 2-3.

[0014] Steps 2-3: After the currently reprocessing task is completed, move all completed reprocessing tasks in the second window to the first window;

[0015] Steps 2-4: Move the emergency task to the second window;

[0016] Steps 2-5: Based on the objective function of minimizing total cost and total time, the whale optimization algorithm is used to solve the problem and obtain the rescheduling scheme for all tasks in the second window.

[0017] Steps 2-6: Send the rescheduling plan to the physical workshop for execution;

[0018] Step 2-7: Clear the first and second windows of the virtual workshop and return to step 2-1 to continue execution until there are no tasks that need to be rescheduled in the third window.

[0019] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0020] Preferably, the scrolling window mechanism includes:

[0021] When no urgent tasks are inserted, the current scrolling frequency is adjusted based on the baseline scrolling frequency and the current load of the second window.

[0022] When an urgent task is inserted, scroll the window immediately.

[0023] Preferably, the solution of the whale optimization algorithm is represented by a matrix-based two-stage solution.

[0024] The first stage is the allocation of reprocessing lines. The allocation of reprocessing lines for each component is represented by a one-dimensional matrix. If the element at the corresponding position in the one-dimensional matrix is ​​set to 1, it means that the component is reprocessed on the reprocessing line with the corresponding sequence number; if the element at the corresponding position in the one-dimensional matrix is ​​set to 0, it means that the component is not reprocessed on the reprocessing line with the corresponding sequence number.

[0025] The second stage is reprocessing sorting, represented by a two-dimensional matrix with three rows. The first row of the two-dimensional matrix represents the product index, the second row represents the component index of the product, and the third row represents the sequence index of the reprocessing line of the component in the product obtained from the first stage. The order of the index values ​​in the third row of the two-dimensional matrix corresponds to the execution order of reprocessing. Each column of the two-dimensional matrix represents the component of the product being reprocessed on the corresponding sequence reprocessing line.

[0026] Preferably, in the whale optimization algorithm, the whale swarm is divided into leaders, followers, and scouts. During initialization, the whale with the best fitness value is set as the leader, and the remaining whales are randomly assigned as followers or scouts. Followers perform local searches based on the leader, while scouts perform global searches in the solution space.

[0027] As a preferred approach, the dynamic adjustment strategy for the whale's role in each iteration is as follows:

[0028] If there is a whale among the followers or scouts with a fitness value higher than that of the current leader, then set that whale as the new leader and randomly assign the original leader to a follower or scout.

[0029] If the optimal fitness value of a follower remains unchanged throughout multiple iterations, then the follower is designated as a scout.

[0030] Preferably, the whale position update strategy in the whale optimization algorithm is as follows:

[0031] For the leader, when it gets stuck in a local optimum, the Levy flight strategy is used for position update; otherwise, the spiral search strategy is used for position update.

[0032] For followers, the position update strategy is determined by a reinforcement learning algorithm, which can be a spiral search strategy, a Levy flight strategy, or a random search strategy.

[0033] For the scouts, the Levy flight strategy is used for position updates.

[0034] Preferably, the position update strategy is determined by a reinforcement learning algorithm, including:

[0035] State definition: The change in a whale's role and individual fitness value is defined as a state;

[0036] Action definition: The position update strategy is defined as an action, which includes the spiral search strategy, the Levy flight strategy, and the random search strategy;

[0037] Reward definition: If the current leader's fitness value is greater than the updated fitness value of the current individual, the reward value is twice the difference between the current leader's fitness value and the updated fitness value of the current individual; if the current individual's fitness value before the update is greater than the updated fitness value of the current individual, the reward value is the difference between the current individual's fitness value before the update and the updated fitness value of the current individual; otherwise, the reward value is the difference between the current individual's fitness value before the update and the updated fitness value of the current individual.

[0038] Based on the defined state, action, and reward, generate the optimal position update strategy for each follower.

[0039] Preferably, the reinforcement learning algorithm employs - Greedy strategy, based on probability Choose a random search strategy, with probability Select the action with the highest Q value.

[0040] This invention provides a digital twin-driven reprocessing scheduling optimization method that considers component differential delays and the insertion of urgent tasks. It introduces a general component differential delay strategy to reduce wasted waiting time and constructs a rescheduling framework based on a rolling window mechanism. Specifically, based on a rolling window mechanism with dynamic rolling frequency, a hierarchical dynamic rescheduling strategy is introduced to model the urgent task insertion problem, thus more efficiently addressing the dynamic scheduling problem of urgent task insertion. To achieve efficient solution, a hybrid whale optimization algorithm is proposed, which integrates solution representation methods, the role-switching mechanism of the artificial bee colony algorithm, and a Q-learning-based strategy selection mechanism to improve the algorithm's optimization performance. Finally, simulation experiments, comparative experiments, and real-world case studies verify the superiority of the proposed method in solving the reprocessing scheduling model. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the execution flow of the reprocessing scheduling model of the present invention;

[0042] Figure 2 This is a schematic diagram illustrating an example of the combination of physical and virtual workshops in this invention;

[0043] Figure 3 This is a flowchart of the digital twin-driven reprocessing scheduling optimization method of the present invention;

[0044] Figure 4 This is a flowchart illustrating the execution of the whale optimization algorithm of the present invention.

[0045] Figure 5 This is a schematic diagram of the matrix-based two-stage solution representation method of the present invention;

[0046] Figure 6 The comprehensive objective function value of each algorithm as the population size increases in the experiments of this invention. The change curve;

[0047] Figure 7 The combined objective function value of each algorithm increases with the number of iterations in the experiments of this invention. The change curve;

[0048] Figure 8 The average value obtained by the HWOA-QL algorithm under different discount factors and learning rates in the experiments of this invention is... Value diagram;

[0049] Figure 9 These are the results obtained by each algorithm on the P10-U3 instance under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values;

[0050] Figure 10 These are the results obtained by each algorithm on instance P21-U3 under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values;

[0051] Figure 11 These are the results obtained by each algorithm on the P68-U8 instance under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values;

[0052] Figure 12 These are the results obtained by each algorithm on instance P100-U24 under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0055] The remanufacturing process includes three key stages: disassembly, reprocessing, and reassembly. For example... Figure 1 As shown, this invention proposes a digital twin-driven reprocessing scheduling model (DT-RSS-DD-UT model) that considers differential delays and the insertion of urgent tasks, focusing on modeling the scheduling optimization problem in the reprocessing stage. First, the DT-RSS-DD-UT model assumes that the common components for reprocessing are non-differentiated; the identification, appearance, and other characteristics of the components do not need to be differentiated before reassembly and can be applied across products in the reassembly process. Second, the DT-RSS-DD-UT model considers multiple equivalent parallel reprocessing production lines, allowing components to be reprocessed on any available line. Furthermore, the model defines a common component warehouse to store common components not immediately used for reassembly for later retrieval, while non-common components are directly transferred to the reassembly workshop after reprocessing to await assembly.

[0056] During the scheduling process, urgent tasks may be inserted at any time, and these tasks have a higher priority than other tasks. Once an urgent task is detected, it needs to be reprocessed immediately. This invention achieves this process through a digital twin-driven dynamic scheduling architecture. This architecture establishes a real-time connection between the physical workshop and the virtual workshop, thereby enabling dynamic response to the demand for urgent task insertion. The virtual workshop continuously monitors and collects scheduling data from the physical workshop in real time. When an urgent task is detected, the virtual workshop will reschedule the task based on existing data, using models and optimization algorithms. Subsequently, the optimized rescheduling scheme is transmitted back to the physical workshop, which then rearranges the reprocessing tasks accordingly, thus effectively addressing the dynamic scheduling challenges posed by the insertion of urgent tasks.

[0057] The costs incurred from reprocessing parts and storing common components directly affect the economic benefits of remanufacturing. Meanwhile, actual remanufacturing companies typically prioritize ensuring timely completion of production tasks to achieve on-time product delivery. Therefore, the DT-RSS-DD-UT model constructed in this invention is a bi-objective optimization model, aiming to simultaneously minimize total cost and total reprocessing time. The following assumptions were considered when constructing the DT-RSS-DD-UT model: (1) The reprocessing line will not be interrupted during operation. (2) The idle time cost of the reprocessing line is negligible. (3) When an urgent task is inserted, it will be prioritized through a rolling window mechanism. (4) There is no priority distinction between urgent tasks. (5) There is no priority distinction between non-urgent tasks. (6) All reprocessing lines are available at the start of reprocessing. (7) All components required for the product can immediately enter the reassembly stage after reprocessing is completed.

[0058] Specifically, the objective function is constructed as follows: The DT-RSS-DD-UT model aims to minimize the total cost and total time of part reprocessing. Minimizing the total cost... The objective function is shown in formula (1).

[0059] (1)

[0060] Minimize total time The objective function is shown in formula (2).

[0061] (2)

[0062] The comprehensive objective function of the DT-RSS-DD-UT model is shown in Equation (3).

[0063] (3)

[0064] in, It is the value of the comprehensive objective function. and These are the weight coefficients of the two objective functions, with values ​​ranging from [0, 1] and constrained by formula (4). For components The cost required for reprocessing. The unit storage cost for general-purpose components. Let n be the quantity of the nth type of general-purpose component in the inventory. For products The start time of reassembly. For the first One product ,in It represents the total number of products. For products The Each component ,in It is a product The total number of components.

[0065] (4)

[0066] Meanwhile, the DT-RSS-DD-UT model satisfies the following constraints:

[0067] Formula (5) ensures that a component only needs to be reprocessed on one reprocessing line.

[0068] (5)

[0069] Formula (6) ensures that a reprocessing line can only process one part at a time.

[0070] (6)

[0071] in, On the reprocessing line Above another component The end time for reprocessing of this component. The reprocessing was completed previously.

[0072] Formulas (7) and (8) define the start and end times of reprocessing for each part on the reprocessing line.

[0073] (7)

[0074] (8)

[0075] Set B is used to record products for which the inventory quantity is insufficient to support reassembly. The general component types are shown in formula (9).

[0076] (9)

[0077] Because reassembly can only begin after all the necessary components have been reprocessed, this affects the product. The start time for reassembly is the latest end time for all parts of the product that need to be reprocessed, as shown in formula (10).

[0078] (10)

[0079] in, Indicates product The latest completion time for reprocessing of required non-standard parts. Indicates product The latest completion time for reprocessing of the required common components. This represents ordinal statistics in reassembled products. The time is used to calculate the end time for reprocessing common parts that are in short supply, as shown in formula (11).

[0080] (11)

[0081] in, From a set The selected dynamic threshold satisfies the following conditions: Choose the smallest value from all possible values ​​such that at least indivual The value is less than or equal to . For the first One reprocessing line, ,in This represents the total number of reprocessing lines. For the first A type of general-purpose component, ,in This represents the total number of common component types. For binary decision variables, where 1 represents a component. For the first One type of general-purpose component, otherwise 0. For binary decision variables, where 1 represents a component. On the reprocessing line If it is not processed further, then it is 0. For binary decision variables, where 1 represents a component. It is an urgent task; otherwise, it is 0. For reassembling products The required first The number of common components. For disassembling the product The end time. For components The time required for further processing. For components The start time for reprocessing. For components The end time for further processing.

[0082] For example, there are 4 types of general-purpose components: , , and The end time for reprocessing these components , , and The values ​​are {11, 17, 7, 9}. Reassembling a certain product requires 3 of these common parts, but currently only 1 of these common parts is in stock. Therefore, This indicates that at least two more general-purpose parts of this type need to be manufactured. To ensure timely completion... At least two reprocessing of this type of common parts must be completed within the specified time. The possible values ​​are {9, 11, 17}. The minimum value is 9, which means that the earliest end time for processing two more general-purpose parts of this type is 9, corresponding to the second order statistic of the set {11, 17, 7, 9}.

[0083] The digital twin system architecture consists of two interconnected parts: a physical space and a virtual space. Monitoring, analysis, and optimization of the physical space are achieved through information interaction between the two spaces. In this invention, the physical space refers to the physical workshop, and the virtual space refers to the virtual workshop generated by mapping the physical workshop. In the digital twin-driven dynamic scheduling architecture, data transmission and interaction between the physical and virtual workshops are achieved through advanced communication technologies and network protocols. This process mainly includes three stages: First, the virtual workshop acquires data from the physical workshop in real time through a data sensing and acquisition system (including sensors, IoT devices, and other monitoring devices); then, the scheduling model within the virtual workshop adjusts and optimizes the scheduling scheme based on the acquired data using dynamic scheduling strategies, responding to the insertion of urgent tasks through timely rescheduling; finally, the optimized scheduling scheme generated by the virtual workshop is transmitted back to the physical workshop to guide the implementation of actual reprocessing operations.

[0084] To solve the dynamic scheduling problem of urgent task insertion more efficiently, this invention proposes an improved digital twin-driven hierarchical dynamic rescheduling strategy based on a rolling window mechanism. First, to balance global and local optimization, the scheduling problem is processed in layers, with the physical workshop serving as the global layer L. globalResponsible for long-term planning, while the virtual workshop serves as a local layer L. local It is responsible for real-time adjustments. Secondly, to avoid redundant global calculations and improve scheduling efficiency, a rolling window mechanism is applied and extended in the rescheduling strategy. In the optimized rolling window mechanism, the reprocessing of a component represents a task. For example, a component... The reprocessing is represented as Task 3-2. The virtual workshop contains three task windows: the first window (W1) represents the window of completed tasks; the second window (W2) represents the window of tasks being scheduled, including tasks that are being reprocessed and tasks that are planned to be reprocessed; and the third window (W3) represents the window of tasks waiting to be scheduled. Figure 2 This is a diagram illustrating the combination of physical and virtual windows. The inserted emergency tasks are 5-1, 5-2, and 5-3.

[0085] Based on the aforementioned preparatory work, this embodiment proposes a digital twin-driven reprocessing scheduling optimization method, such as... Figure 3 As shown, the specific steps include:

[0086] Step 1: Initialize and generate the initial reprocessing schedule in the physical workshop, and move the initial reprocessing schedule to the third window of the virtual workshop.

[0087] Step 2: After the scheduling task begins execution, the virtual workshop executes the rescheduling strategy.

[0088] Step 2-1: Based on the reprocessing data collected from the physical workshop, move the currently reprocessing tasks and the tasks about to be reprocessed from the third window to the second window using a scrolling window mechanism.

[0089] Because the time required for reprocessing each component varies, and the task load within each window also differs at different times, existing scrolling window mechanisms use a fixed scrolling frequency, which cannot dynamically change according to the window load. To overcome these shortcomings, this embodiment proposes a dynamic scrolling frequency adjustment strategy.

[0090] Current scrolling frequency when no urgent tasks are inserted At the reference rolling frequency Based on this, adjustments are made according to the current load of window W2, as shown in formulas (12) and (13).

[0091] (12)

[0092] (13)

[0093] in, It is the time required to complete the task within the current window W2. It is the time required to complete the task within the initial window W2, that is, the time required to execute the task first moved into window W2 is taken as a reference. It is an adjustment factor with a value range of [1, 2]. Step 2-1 of the next iteration is triggered based on the current rolling frequency.

[0094] When an urgent task is inserted, the window will immediately scroll and update the scheduling scheme, that is, immediately start the next iteration and execute step 2-1.

[0095] Step 2-2: If no emergency task insertion is detected, proceed to step 2-5; otherwise, proceed to step 2-3.

[0096] Steps 2-3: After the currently reprocessing task is completed, move all completed reprocessing tasks in the second window to the first window.

[0097] Steps 2-4: Move the emergency task to the second window.

[0098] Steps 2-5: Based on the objective function of minimizing total cost and total time (i.e., the comprehensive objective function of the DT-RSS-DD-UT model), the whale optimization algorithm is used to solve the problem and obtain the rescheduling scheme for all tasks within the second window.

[0099] The basic whale optimization algorithm solves mathematical optimization problems by simulating the hunting behavior of whale pods. It mainly consists of three basic stages: surrounding the prey, bubble net attack, and random search. Due to its superior global search ability and adaptability, the whale optimization algorithm has been widely used to solve scheduling optimization problems. However, the basic whale optimization algorithm is suitable for solving continuous optimization problems and needs to be improved to solve discrete optimization problems. Furthermore, the basic whale optimization algorithm has relatively weak local search ability, which can easily lead to premature convergence to a local optimum. To effectively solve the DT-RSS-DD-UT model, this embodiment proposes a hybrid whale optimization algorithm (HWOA-QL algorithm), with the following main improvements:

[0100] (1) A matrix-based two-stage solution representation method is proposed to reasonably encode the feasible solutions of the DT-RSS-DD-UT model. During the position update process, the discreteness of the solution is maintained by a rounding mechanism.

[0101] (2) Introduce the roles of leader, follower and scout in artificial bee colony algorithm into whale pods, divide individual whales into one of these three roles, and construct a role switching mechanism to balance global and local search.

[0102] (3) A strategy selection mechanism based on Q-learning was designed to adaptively select strategies such as Levy flight, spiral search and random search to enhance the algorithm's solution space search capability.

[0103] like Figure 4 As shown, the execution process of the whale optimization algorithm in this embodiment is as follows:

[0104] (1) Population initialization is performed based on a random strategy, where each individual in the population represents a rescheduling scheme. To more intuitively represent the feasible solution of the model and the task scheduling scheme, this invention proposes a matrix-based two-stage solution representation method, such as... Figure 5 As shown.

[0105] The first stage is the allocation of reprocessing lines. This stage uses elements... A one-dimensional matrix (indexed from 1) is used to represent the components. The reprocessing line allocation for each component is shown. Specifically, the reprocessing line allocation for each component is represented by a one-dimensional matrix. An element set to 1 in the matrix indicates that the component is being reprocessed on the corresponding reprocessing line; an element set to 0 indicates that the component is not being reprocessed on the corresponding reprocessing line. For example... This indicates that the component is on the reprocessing line. Further processing is carried out on top.

[0106] The second stage is reprocessing and sorting, represented by a two-dimensional matrix with three rows, as shown in the following encoding example: The matrix consists of three rows: the first row represents the product index, the second row represents the component index, and the third row represents the reprocessing line allocation index obtained from the first stage. The order of the index values ​​in the third row corresponds to the execution order of the reprocessing tasks. Each column in the two-dimensional matrix represents the reprocessing of a product's component on the corresponding reprocessing line. For example, the first column... Indicates product The first component On the reprocessing line The first position is reprocessed; the fifth column Indicates product The first component On the reprocessing line The second position is then further processed, and so on.

[0107] At the same time, define the framework and parameters of Q-learning:

[0108] (A) State definition: The state of an individual (i.e., a feasible solution) is defined as S, which includes changes in role and fitness value, as shown in formula (14).

[0109] (14)

[0110] in, This represents the current individual's role in the population and satisfies... , It is the change in the fitness value of the current individual in this iteration, as shown in formula (15). yes The discretized representation of is shown in Equation (16).

[0111] (15)

[0112] (16)

[0113] in, It is the current iteration number. Is the current individual in the th ? The fitness value at the next iteration.

[0114] (B) Action Definition: An individual's action (i.e., the choice of position update strategy) is defined as follows: The actions involved in this embodiment include selecting Levi flight, spiral search, and random search as update strategies.

[0115] (C) Definition of reward: Reward Based on the improved fitness function design, it is used to guide the learning process of the algorithm, and its calculation is shown in formula (17):

[0116] (17)

[0117] in, The current leader is in The fitness value at the next iteration.

[0118] (D) Q-value update: This represents an individual in a given state. Next action The expected value of the long-term cumulative reward is used to evaluate the merits of taking a certain action in a specific state, thereby helping individuals choose the optimal update strategy in different states. The calculation method is shown in formula (18), and is updated according to formula (19).

[0119] (18)

[0120] (19)

[0121] in, Represent the expected function; It is the offset of the future iteration number, that is, the current iteration number. After the nth iteration The next iteration; It is a discount factor used to balance current rewards with future rewards; Indicates starting from the current number The sum of discounted future rewards starting from the next iteration; Indicates the individual in the first... The state at the next iteration; Indicates the individual in the first... The action performed in the next iteration; It is the learning rate, used to control the degree to which new information affects the Q value; This represents the new state of an individual. This indicates a new action chosen by an individual.

[0122] (E) Exploration and Utilization: To avoid the algorithm getting trapped in local optima, the following methods are employed: - A greedy strategy is used to balance the probability of choosing an action with a high Q value and performing a random search, as shown in Equation (20).

[0123] (20)

[0124] in, Indicates the first The probability of performing a random search in the next iteration is calculated using formula (21).

[0125] (twenty one)

[0126] (2) Using the objective function as the fitness function, calculate the fitness value and... value.

[0127] (3) Role switching mechanism: Inspired by the artificial bee colony algorithm, the HWOA-QL algorithm proposed in this embodiment divides the whale swarm into leaders, followers, and scouts to balance global and local search, where each whale represents an individual. During initialization, the individual with the highest fitness value in the current population is first considered the leader. The fitness value is directly calculated using the comprehensive objective function value. The smaller the objective function value, the better the fitness. Subsequently, the remaining individuals are randomly assigned to either followers or scouts. Followers perform a local search around the leader, while scouts perform a global search within the solution space.

[0128] In each iteration, the role of an individual whale will be dynamically adjusted based on its fitness value and search history:

[0129] (A) If an individual has a fitness value that is better than the current leader, then set that individual as the new leader.

[0130] (B) If the optimal fitness value of a follower remains unchanged after multiple (10) iterations, then the individual is designated as a scout.

[0131] (C) If the fitness value of the current leader is surpassed by the fitness value of other individuals, then the leader is randomly assigned to either a follower or a scout.

[0132] (4) Execution Strategy Selection Mechanism: In this embodiment, the HWOA-QL algorithm uses Levy flight, spiral search of the basic whale optimization algorithm, and random search as individual update strategies. Subsequently, based on Q-learning, the optimal update strategy is dynamically selected for each individual, enabling the individual to continuously move closer to the leader, thereby improving the algorithm's global search capability. Specifically:

[0133] (A) For individual leaders, spiral search is used by default for position updates. When the leader gets stuck in a local optimum, the Levy flight strategy is used for position updates, as shown in formula (22).

[0134] (twenty two)

[0135] in, Indicates the updated leader position; Indicates the current leader's position; This indicates the position of a randomly selected non-leader individual; It is a random number in the range [-1, 1]; and Does it follow a standard normal distribution? Random numbers; The distance between the randomly selected individual and the current leader individual is represented by formula (23).

[0136] (twenty three)

[0137] (B) For individual followers, their position update strategy selection is entirely determined by Q-learning, as shown in Equation (24).

[0138] (twenty four)

[0139] in, Indicates the position of the individual after the update; Indicates the current individual in the th order. The position at the next iteration; It is a random number used to control the intensity of random disturbances, and is usually set to 0.1; Does it follow a standard normal distribution? Random numbers; The distance between the current individual and the current leader individual is represented by formula (25).

[0140] (25)

[0141] (C) For individual scouts, the Levy flight strategy is always used for position updates, as shown in formula (26).

[0142] (26)

[0143] (5) Discretize the solution and calculate the reward value and update it. value.

[0144] (6) Determine whether the iteration termination condition is met. If it is met, output the global optimal solution as the optimal rescheduling scheme; otherwise, return to step (2) to continue execution.

[0145] Steps 2-6: Send the rescheduling plan to the physical workshop for execution.

[0146] Step 2-7: Clear the first and second windows of the virtual workshop and return to step 2-1 to continue execution until there are no tasks that need to be rescheduled in the third window.

[0147] This invention includes an experimental design comprising a series of simulation experiments, comparative experiments, and statistical analysis experiments, aimed at evaluating the performance of the proposed HWOA-QL algorithm. All experiments were implemented using Python and run on a personal computer with a 64-bit Windows 11 operating system, an AMD Ryzen 7 8845H 8-core processor (3.80GHz), and 32GB of RAM.

[0148] (I) Experimental Design:

[0149] This experiment selected three baseline algorithms for performance comparison, including the basic Whale Optimization Algorithm (WOA) (from Mirjalili, S., & Lewis, A. (2016). The whale optimization algorithm. Advances in Engineering Software, 95, 51-67.), the improved Hybrid Particle Swarm Optimization Algorithm (IPSO) (from Liao, J., & Lin, C. (2019). Optimization and simulation of job-shop supply chain scheduling in manufacturing enterprises based on particle swarm optimization. International Journal of Simulation Modelling, 18(1), 187-196.), and the improved Ant Colony Algorithm (IACO) (from Zhu, YZ, Song, Q., & Li, M. (2025). Multi-AGV multitask collaborative scheduling based on an improvedant colony algorithm. International Journal of Advanced Robotic Systems, 22(1), 17298806241312784.). To ensure the reliability and stability of the experimental results, all experiments were run independently 16 times.

[0150] Due to the lack of relevant benchmark sets in the remanufacturing scheduling field, this experiment randomly generated several simulation problem instances within a given interval. All problem instances involved in the experiment have been uploaded to the Figshare database. Each problem instance name includes information on the number of products and the number of emergency tasks; for example, P35-U5 indicates that the instance contains 35 products and 5 randomly inserted emergency tasks.

[0151] (II) Parameter Design: To ensure that all algorithms are compared under optimal performance conditions, three sets of simulation experiments were conducted to determine the common parameters of all comparison algorithms. First, simulation experiments were carried out on instance P35-U5 to evaluate the performance of each algorithm under different population sizes. The maximum number of iterations of the algorithm was set to 1000, the initial population size was set to 10, and it increased to 150 in step sizes of 10. The algorithm used the comprehensive objective function value T as the evaluation index, and its change curve with the increase of population size is shown in the figure. Figure 6 As shown.

[0152] like Figure 6 As shown, the optimization performance of each algorithm improves with increasing population size. However, regardless of the population size, the solutions obtained by the HWOA-QL algorithm are superior to those of other baseline algorithms, indicating that the proposed HWOA-QL algorithm outperforms other baseline algorithms in solving the DT-RSS-DD-UT model (specifically, the digital twin-driven reprocessing scheduling optimization method of this embodiment). Since most baseline algorithms can obtain good solutions with a population size of 130, the population size for subsequent experiments was set to 130.

[0153] Furthermore, simulation experiments were conducted on instance P35-U5 to evaluate the performance of each algorithm at different iteration counts. The maximum number of iterations was set to 1000, and the population size was set to 130. All algorithms used a combined objective function value. As an evaluation metric, its curve changing with the number of iterations is as follows: Figure 7 As shown.

[0154] like Figure 7 As shown, the combined objective function value of each algorithm increases with the number of iterations. The decreasing trend indicates that the solutions obtained by the algorithms are continuously optimizing. Among them, the HWOA-QL algorithm can obtain the optimal solution in fewer iterations, demonstrating that the proposed algorithm has better solution space search capabilities than other baseline algorithms when solving the DT-RSS-DD-UT model. Since all algorithms converge within 980 iterations, the number of iterations for subsequent experiments was set to 980.

[0155] In addition, simulation experiments were conducted on instance P35-U5 to evaluate the HWOA-QL algorithm under different discount factors. and learning rate Performance settings: The maximum number of iterations is set to 980, the population size is set to 130, and a comprehensive objective function value is used. As an evaluation indicator, the experimental results are presented through a heatmap, such as... Figure 8As shown. Blue indicates that the HWOA-QL algorithm performs better at the current discount factor and learning rate (corresponding to a smaller average). (Value), red indicates that the HWOA-QL algorithm performs worse at the current discount factor and learning rate (corresponding to a larger average). value).

[0156] like Figure 8 As shown, when the discount factor The learning rate is 0.9. The HWOA-QL algorithm achieves the optimal solution when the discount factor is 0.7. Therefore, in subsequent experiments, the discount factor and learning rate of the HWOA-QL algorithm were set to 0.9 and 0.7, respectively.

[0157] (III) Performance Analysis:

[0158] To comprehensively evaluate the performance of the HWOA-QL algorithm in solving the DT-RSS-DD-UT model, this experiment generated 11 sets of problem instances and applied different combinations of objective function weight coefficients (i.e., different...). and Below, the performance of the HWOA-QL algorithm is compared with other baseline algorithms. The results of the comparative experiments are shown in Tables 1-6. Tables 1-3 present the optimal and average values ​​of the comprehensive objective function T obtained by each algorithm on each instance under different weight coefficient combinations, denoted as "optimal value" and "average value," respectively. Tables 4-6 show the CPU computation time and standard deviation of the comprehensive objective function T of each algorithm on each instance under different weight coefficient combinations, denoted as "time" and "standard deviation," respectively.

[0159] As shown in Tables 1-3, under different combinations of weight coefficients, the optimal and average values ​​of T obtained by the HWOA-QL algorithm are better than other baseline algorithms in most instances, indicating that the HWOA-QL algorithm has better optimization performance than other baseline algorithms when solving the DT-RSS-DD-UT model.

[0160] As shown in Tables 4-6, under different combinations of weight coefficients, the standard deviation of T obtained by the HWOA-QL algorithm on most instances is smaller than that of other baseline algorithms, indicating that the HWOA-QL algorithm has better stability than other baseline algorithms when solving the DT-RSS-DD-UT model.

[0161] However, the HWOA-QL algorithm takes longer than the basic whale optimization algorithm on most instances, but shorter than the improved ant colony optimization algorithm and the improved particle swarm optimization algorithm. Meanwhile, the HWOA-QL algorithm outperforms the basic whale optimization algorithm in terms of solution quality and stability. Therefore, the solution efficiency of the HWOA-QL algorithm meets the needs of practical applications, achieving a good balance between computation time and solution quality.

[0162] Table 1. Optimal results of each algorithm Values ​​and averages Value (weighted coefficient combination F1:) =0.3, =0.7)

[0163]

[0164] Note: The optimal value is indicated by bold, and "↓" indicates that the smaller the value, the better.

[0165] Table 2 Optimal results of each algorithm Values ​​and averages Value (weighted coefficient combination F2:) =0.5, =0.5)

[0166]

[0167] Note: The optimal value is indicated by bold, and "↓" indicates that the smaller the value, the better.

[0168] Table 3 Optimal results of each algorithm Values ​​and averages Value (weighted coefficient combination F3:) =0.7, =0.3)

[0169]

[0170] Note: The optimal value is indicated by bold, and "↓" indicates that the smaller the value, the better.

[0171] Table 4 CPU computation time for each algorithm Standard deviation of values ​​(weighted combination F1: =0.3, =0.7)

[0172]

[0173] Note: The optimal value is indicated by bold, and "↓" indicates that the smaller the value, the better.

[0174] Table 5. CPU computation time for each algorithm Standard deviation of values ​​(weighted combination F2: =0.5, =0.5)

[0175]

[0176] Note: The optimal value is indicated by bold, and "↓" indicates that the smaller the value, the better.

[0177] Table 6. CPU computation time for each algorithm Standard deviation of values ​​(weighted combination F3: =0.7, =0.3)

[0178]

[0179] Note: The optimal value is indicated by bold, and "↓" indicates that the smaller the value, the better.

[0180] (iv) Statistical analysis:

[0181] To further verify the solution stability of the HWOA-QL algorithm, this experiment conducted error analysis and statistical analysis. Specifically, by selecting a subset of instances (P10-U3, P21-U3, P68-U8, and P100-U24), and under the weighting coefficient combinations F1 and F3, the comprehensive objective function values ​​obtained from 16 independent runs were analyzed. Error analysis was performed. Experimental results are presented as box plots, such as... Figures 9-12 As shown.

[0182] First, in most instances, the HWOA-QL algorithm obtains... The median and mean values ​​are lower than other baseline algorithms, indicating that the HWOA-QL algorithm has superior solution space search capability. Secondly, in most instances, the bin length of the HWOA-QL algorithm is smaller than that of other baseline algorithms, indicating that the experimental results of the HWOA-QL algorithm have lower dispersion, i.e., higher stability. Thirdly, in terms of interquartile range, in most instances, the HWOA-QL algorithm obtains... The optimal and worst values ​​of this algorithm are both better than those of other baseline algorithms, indicating that the algorithm can obtain higher quality solutions.

[0183] To further analyze whether the experimental results obtained by the HWOA-QL algorithm differ significantly from other baseline algorithms, and to statistically verify that the HWOA-QL algorithm is superior to other baseline algorithms, a Wilcoxon test with a significance level of 0.05 was conducted in this experiment. This test was performed using SPSS statistical software, and the statistical results under different combinations of weighting coefficients are shown in Table 7.

[0184] As shown in Table 7, in most instances, the experimental results obtained by the HWOA-QL algorithm are significantly different from those of other baseline algorithms (i.e., p < 0.05), indicating that the HWOA-QL algorithm is statistically superior.

[0185] Table 7. P-values ​​of the experimental results obtained by the HWOA-QL algorithm under different weight coefficient combinations compared with other baseline algorithms.

[0186]

[0187] Note: * indicates that the p-value is below the significance level of 0.05.

[0188] (V) Case Studies:

[0189] This experiment, using a real-world case study from a company, verifies the applicability and superiority of the proposed DT-RSS-DD-UT model and HWOA-QL algorithm in a real-world scenario. Two common recycled products from this company—the front hydraulic cylinder protection frame and the water system—are selected as the experimental subjects.

[0190] In this real-world example, the reprocessing workshop includes 6 cleaning machines, 1 polishing machine, 3 grinding machines, 5 milling machines, 8 cold welding stations, 8 welding workstations, 8 repair stations, 2 laser cladding machines, and 3 milling machines, forming approximately 6-8 reprocessing lines. The daily remanufacturing volume is approximately 2-10 products, with 2-5 being emergency tasks. Each product contains approximately 10-100 parts. The reprocessing path for each part is: cleaning → replacement → secondary assembly on old equipment → spare parts refurbishment → debugging.

[0191] Based on the above actual production data, this experiment conducted a performance comparison experiment between the HWOA-QL algorithm and other baseline algorithms. The experimental results are shown in Table 8.

[0192] As shown in Table 8, the HWOA-QL algorithm can obtain better solutions in actual production scenarios, which verifies the effectiveness and superiority of the HWOA-QL algorithm in solving the DT-RSS-DD-UT model from a practical perspective.

[0193] Table 8. Comparison of Algorithms in Real-World Cases

[0194]

[0195] Note: The optimal value is indicated by bold, and "↓" indicates that the smaller the value, the better.

[0196] This invention proposes a novel digital twin-driven reprocessing scheduling model that considers differential delays and the insertion of urgent tasks, aiming to minimize the total reprocessing cost and time. Furthermore, addressing the dynamic scheduling problem of urgent task insertion, this experiment designs an improved hierarchical dynamic rescheduling strategy based on a dynamically adjusted rolling window mechanism to more efficiently handle dynamic scheduling issues considering urgent task insertion.

[0197] The reprocessing scheduling problem, as a subproblem of remanufacturing scheduling, is a typical NP-hard problem. Swarm intelligence optimization algorithms have shown strong adaptability to this problem and have been widely applied by researchers. The whale optimization algorithm, with its excellent global search capability and algorithmic simplicity, is often used to solve complex optimization problems. Therefore, to further improve the optimization performance of the algorithm, this invention proposes a hybrid whale optimization algorithm incorporating a Q-learning mechanism. This algorithm designs a matrix-based two-stage solution representation method; constructs a Q-learning-based policy selection mechanism to enhance the solution space search capability; and introduces a role-switching mechanism that integrates artificial bee colony optimization to balance the algorithm's global and local search capabilities. Finally, extensive simulation experiments and practical case studies verify the effectiveness and practicality of the proposed model and algorithm.

[0198] In another embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the digital twin-driven reprocessing scheduling optimization method of the present invention.

[0199] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0200] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0201] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A digital twin-driven reprocessing scheduling optimization method, characterized in that, The reprocessing scheduling optimization method driven by digital twin technology maps the physical workshop of reprocessing to generate a virtual workshop, and the virtual workshop has three windows. Step 1: Initialize and generate the initial reprocessing schedule in the physical workshop, and move the initial reprocessing schedule to the third window of the virtual workshop; Step 2: After the scheduled task begins execution, the virtual workshop executes a rescheduling strategy, including: Step 2-1: Based on the reprocessing data collected from the physical workshop, move the currently reprocessing tasks and the tasks about to be reprocessed from the third window to the second window using a scrolling window mechanism; Step 2-2: If no emergency task insertion is detected, proceed to step 2-5; otherwise, proceed to step 2-3. Steps 2-3: After the currently reprocessing task is completed, move all completed reprocessing tasks in the second window to the first window; Steps 2-4: Move the emergency task to the second window; Steps 2-5: Based on the objective function of minimizing total cost and total time, the whale optimization algorithm is used to solve the problem and obtain the rescheduling scheme for all tasks in the second window. Steps 2-6: Send the rescheduling plan to the physical workshop for execution; Step 2-7: Clear the first and second windows of the virtual workshop and return to step 2-1 to continue execution until there are no tasks that need to be rescheduled in the third window.

2. The digital twin-driven reprocessing scheduling optimization method according to claim 1, characterized in that, The scrolling window mechanism includes: When no urgent tasks are inserted, the current scrolling frequency is adjusted based on the baseline scrolling frequency and the current load of the second window. When an urgent task is inserted, scroll the window immediately.

3. The digital twin-driven reprocessing scheduling optimization method according to claim 1, characterized in that, The solution of the whale optimization algorithm is represented by a matrix-based two-stage solution. The first stage is the allocation of reprocessing lines. The allocation of reprocessing lines for each component is represented by a one-dimensional matrix. If the element at the corresponding position in the one-dimensional matrix is ​​set to 1, it means that the component is reprocessed on the reprocessing line with the corresponding sequence number; if the element at the corresponding position in the one-dimensional matrix is ​​set to 0, it means that the component is not reprocessed on the reprocessing line with the corresponding sequence number. The second stage is reprocessing sorting, represented by a two-dimensional matrix with three rows. The first row of the two-dimensional matrix represents the product index, the second row represents the component index of the product, and the third row represents the sequence index of the reprocessing line of the component in the product obtained from the first stage. The order of the index values ​​in the third row of the two-dimensional matrix corresponds to the execution order of reprocessing. Each column of the two-dimensional matrix represents the component of the product being reprocessed on the corresponding sequence reprocessing line.

4. The digital twin-driven reprocessing scheduling optimization method according to claim 1, characterized in that, In the whale optimization algorithm, the whale pod is divided into leaders, followers, and scouts. During initialization, the whale with the best fitness value is set as the leader, and the remaining whales are randomly assigned as followers or scouts. Followers perform local searches based on the leader, while scouts perform global searches in the solution space.

5. The digital twin-driven reprocessing scheduling optimization method according to claim 4, characterized in that, In each iteration, the dynamic adjustment strategy for the whale's role is as follows: If there is a whale among the followers or scouts with a fitness value higher than that of the current leader, then set that whale as the new leader and randomly assign the original leader to a follower or scout. If the optimal fitness value of a follower remains unchanged throughout multiple iterations, then the follower is designated as a scout.

6. The digital twin-driven reprocessing scheduling optimization method according to claim 4, characterized in that, The whale position update strategy in the whale optimization algorithm is as follows: For the leader, when it gets stuck in a local optimum, the Levy flight strategy is used for position update; otherwise, the spiral search strategy is used for position update. For followers, the position update strategy is determined by a reinforcement learning algorithm, which can be a spiral search strategy, a Levy flight strategy, or a random search strategy. For the scouts, the Levy flight strategy is used for position updates.

7. The digital twin-driven reprocessing scheduling optimization method according to claim 6, characterized in that, The location update strategy is determined by a reinforcement learning algorithm, including: State definition: The change in a whale's role and individual fitness value is defined as a state; Action definition: The position update strategy is defined as an action, which includes the spiral search strategy, the Levy flight strategy, and the random search strategy; Reward definition: If the current leader's fitness value is greater than the updated fitness value of the current individual, the reward value is twice the difference between the current leader's fitness value and the updated fitness value of the current individual; if the current individual's fitness value before the update is greater than the updated fitness value of the current individual, the reward value is the difference between the current individual's fitness value before the update and the updated fitness value of the current individual; otherwise, the reward value is the difference between the current individual's fitness value before the update and the updated fitness value of the current individual. Based on the defined state, action, and reward, generate the optimal position update strategy for each follower.

8. The digital twin-driven reprocessing scheduling optimization method according to claim 7, characterized in that, The reinforcement learning algorithm employs - Greedy strategy, based on probability Choose a random search strategy, with probability Select the action with the highest Q value.

Images