A digital twin driven rework scheduling optimization method

Abstract

The application belongs to the technical field of remanufacturing process, and discloses a kind of digital twin driven reprocessing scheduling optimization method, including in physical workshop initialization generation reprocessing initial scheduling scheme, and reprocessing initial scheduling scheme is moved to the third window of virtual workshop;Based on the task that is currently being reprocessed and the task that will be reprocessed from the third window to the second window based on the rolling window mechanism;If emergency task insertion is detected, move all completed reprocessing tasks in the second window to the first window;Emergency task is moved to the second window;Based on the objective function of minimizing total cost and total time, a whale optimization algorithm is used to solve, to obtain the rescheduling scheme of all tasks in the second window;Rescheduling scheme is sent to physical workshop execution until there is no task that needs rescheduling in the third window.The application considers the insertion of emergency task, and considers rescheduling and response efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remanufacturing process, and particularly relates to a digital twin driven remanufacturing scheduling optimization method. BACKGROUND

[0002] A remanufacturing process generally includes the following three core links: firstly, a recovered product is disassembled in a disassembly workshop to obtain a plurality of components that can be used for subsequent processing; secondly, the components are reprocessed in a reprocessing workshop; and finally, the reprocessed components are reassembled into a complete product in a reassembly workshop, and necessary performance optimization is performed. Among them, the reprocessing link plays an important role in the remanufacturing process, and through reasonable scheduling and optimization of production resources, it helps to improve resource utilization, enhance production flexibility and overall manufacturing efficiency. With the continuous development of the remanufacturing industry, the reprocessing scheduling problem has gradually become a research hotspot.

[0003] Current research on the reprocessing scheduling problem mainly focuses on two research directions: one is to explore the influence of different reprocessing production lines or workshop configurations on the scheduling process, such as parallel flow line type reprocessing production line, flexible job shop type reprocessing workshop, and parallel and dedicated mixed type reprocessing production line; the second is to explore the influence of various production disturbances on the scheduling process, such as random rework, new task insertion and outsourcing.

[0004] These studies usually assume that there are significant differences between the components in different recovered products, so the corresponding remanufactured products must wait for all the components in the same original product to complete reprocessing before they can be assembled, resulting in a large amount of waiting time. However, in actual remanufacturing scenarios, some components can be completely repaired after reprocessing, and there is no essential difference between components that only have cosmetic differences (such as labels, colors, etc.). Such components have a certain degree of universality and can be used across products for assembly without the need for separate processing before reassembly. This processing strategy is called "delayed differentiation". Through the delayed differentiation strategy, universal components after reprocessing can be stored in inventory for immediate use, thereby saving waiting time for reassembly.

[0005] In addition, the insertion of emergency tasks is often encountered in remanufacturing production processes. In order to effectively schedule emergency tasks, the reprocessing scheduling model needs to have high flexibility and dynamic response capability to achieve real-time scheduling updates. However, existing research on dynamic scheduling is mostly based on physical space, making it difficult to respond quickly to emergency tasks. Digital twin technology integrates physical and virtual spaces to achieve information interaction, improving response speed. Research shows that digital twins have been widely used to solve dynamic scheduling problems. However, existing dynamic scheduling methods based on digital twins mostly use global rescheduling strategies and rely on fixed rolling frequencies, making it difficult to balance rescheduling and system response efficiency.

[0006] As one of the sub-problems of remanufacturing scheduling, the reprocessing scheduling problem belongs to the typical NP-Hard Problem, and swarm intelligence optimization algorithms have been widely used in this problem. In addition to traditional swarm intelligence optimization algorithms (such as ant colony algorithm, artificial bee colony algorithm, etc.), whale optimization algorithm has attracted widespread attention from scholars due to its good solution space search ability and algorithm simplicity. At present, an improved whale optimization algorithm based on integrated method has been proposed to solve the dynamic job shop scheduling model. Whale optimization algorithm combined with genetic algorithm can more efficiently solve the production scheduling model. In addition, an improved whale optimization algorithm combining adaptive weight and quadratic interpolation technology is used to solve the cloud remanufacturing scheduling model. However, the basic whale optimization algorithm has a simple solution representation, limited adaptability to discrete problems, and is prone to local optimal solution, which is difficult to directly apply to complex solution representation and update model. SUMMARY

[0007] The purpose of the present application is to provide a digital twin driven reprocessing scheduling optimization method, which takes into account the insertion of urgent tasks and takes into account rescheduling and response efficiency.

[0008] To achieve the above purpose, the technical scheme adopted by the present application is:

[0009] A digital twin driven reprocessing scheduling optimization method, which maps the physical workshop of reprocessing to generate a virtual workshop based on digital twin technology, and the virtual workshop is provided with three windows, the digital twin driven reprocessing scheduling optimization method comprises:

[0010] Step 1: generate a reprocessing initial scheduling scheme in the physical workshop, and move the reprocessing initial scheduling scheme to the third window of the virtual workshop;

[0011] Step 2, after the start of the scheduling task, the virtual workshop executes the rescheduling strategy, comprising:

[0012] Step 2-1, according to the reprocessing data collected from the physical workshop, the tasks currently being reprocessed and the tasks to be reprocessed are moved from the third window to the second window based on the rolling window mechanism;

[0013] Step 2-2, if no urgent task insertion is detected, step 2-5 is executed; otherwise, step 2-3 is executed;

[0014] Step 2-3, after the reprocessing of the task currently being reprocessed is completed, all the completed reprocessing tasks in the second window are moved to the first window;

[0015] Step 2-4, move the urgent task to the second window;

[0016] Step 2-5, based on the objective function of minimizing the total cost and the total time, the whale optimization algorithm is used to solve, and the rescheduling scheme of all tasks in the second window is obtained;

[0017] Step 2-6, the rescheduling scheme is sent to the physical workshop for execution;

[0018] Step 2-7, the first window and the second window of the virtual workshop are emptied, and the execution returns to step 2-1 until there is no task that needs to be rescheduled in the third window.

[0019] The following also provides several optional modes, but not as an additional limitation of the above overall scheme, just a further supplement or preferred, without technical or logical contradiction, each optional mode can be combined alone for the above overall scheme, and can also be combined between multiple optional modes.

[0020] As a preferred, the rolling window mechanism comprises:

[0021] When there is no urgent task insertion, the current rolling frequency is adjusted on the basis of the reference rolling frequency according to the current load of the second window;

[0022] When there is an urgent task insertion, the window is immediately rolled.

[0023] As a preferred, the solution of the whale optimization algorithm is represented by a two-stage solution based on a matrix;

[0024] The first stage is the assignment of the reprocessing line, and the assignment of the reprocessing line of each component is represented by a one-dimensional matrix. The element at the corresponding position in the one-dimensional matrix is 1, indicating that the component is reprocessed on the reprocessing line corresponding to the serial number. The element at the corresponding position in the one-dimensional matrix is 0, indicating that the component is not reprocessed on the reprocessing line corresponding to the serial number.

[0025] The second stage is the reprocessing sequence, which is represented by a two-dimensional matrix with three rows. The first row of the two-dimensional matrix represents the product index, the second row of the two-dimensional matrix represents the component index in the product, and the third row of the two-dimensional matrix represents the serial number index of the reprocessing line of the component in the product obtained from the first stage. The order of the index values in the third row of the two-dimensional matrix corresponds to the execution order of the reprocessing. Each row in the two-dimensional matrix represents that the component of the product is reprocessed on the reprocessing line corresponding to the serial number.

[0026] As a preferred, the whale population in the whale optimization algorithm is divided into leaders, followers and scouts. When initialized, the whale with the optimal fitness value is set as the leader, and the remaining whales are randomly divided into followers or scouts. The followers perform local search according to the leader, and the scouts perform global search in the solution space.

[0027] As preferred, in each iteration, the dynamic adjustment strategy of the role of the whale is as follows:

[0028] If there is a whale with a fitness value better than the fitness value of the current leader in the follower or scout, the whale is set as the new leader, and the original leader is randomly set as a follower or a scout.

[0029] If the optimal fitness value of the follower remains unchanged for multiple iterations, the follower is set as a scout.

[0030] As preferred, the position update strategy of the whale in the whale optimization algorithm is as follows:

[0031] For the leader, when trapped in a local optimal solution, the position is updated by using the Levy flight strategy; otherwise, the position is updated by using the spiral search strategy.

[0032] For the follower, the position update strategy is determined by the reinforcement learning algorithm, which is the spiral search strategy, the Levy flight strategy, or the random search strategy.

[0033] For the scout, the position is updated by using the Levy flight strategy.

[0034] As preferred, the position update strategy is determined by the reinforcement learning algorithm, which includes:

[0035] State definition: the role of the whale and the change of the individual fitness value are defined as the state.

[0036] Action definition: the position update strategy is defined as the action, which includes the spiral search strategy, the Levy flight strategy, and the random search strategy.

[0037] Reward definition: if the fitness value of the current leader is greater than the updated fitness value of the current individual, the reward value is twice the difference between the fitness value of the current leader and the updated fitness value of the current individual; if the fitness value of the current individual before updating is greater than the updated fitness value of the current individual, the reward value is the difference between the fitness value of the current individual before updating and the updated fitness value of the current individual; otherwise, the reward value is the difference between the fitness value of the current individual before updating and the updated fitness value of the current individual.

[0038] Based on the defined state, action, and reward, the best position update strategy is generated for each follower.

[0039] As preferred, the reinforcement learning algorithm uses - the greedy strategy with a probability of the random search strategy with a probability of the action with the highest Q value.

[0040] This invention provides a digital twin-driven reprocessing scheduling optimization method that considers component differential delays and the insertion of urgent tasks. It introduces a general component differential delay strategy to reduce wasted waiting time and constructs a rescheduling framework based on a rolling window mechanism. Specifically, based on a rolling window mechanism with dynamic rolling frequency, a hierarchical dynamic rescheduling strategy is introduced to model the urgent task insertion problem, thus more efficiently addressing the dynamic scheduling problem of urgent task insertion. To achieve efficient solution, a hybrid whale optimization algorithm is proposed, which integrates solution representation methods, the role-switching mechanism of the artificial bee colony algorithm, and a Q-learning-based strategy selection mechanism to improve the algorithm's optimization performance. Finally, simulation experiments, comparative experiments, and real-world case studies verify the superiority of the proposed method in solving the reprocessing scheduling model. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the execution flow of the reprocessing scheduling model of the present invention;

[0042] Figure 2 This is a schematic diagram illustrating an example of the combination of physical and virtual workshops in this invention;

[0043] Figure 3 This is a flowchart of the digital twin-driven reprocessing scheduling optimization method of the present invention;

[0044] Figure 4 This is a flowchart illustrating the execution of the whale optimization algorithm of the present invention.

[0045] Figure 5 This is a schematic diagram of the matrix-based two-stage solution representation method of the present invention;

[0046] Figure 6 The comprehensive objective function value of each algorithm as the population size increases in the experiments of this invention. The change curve;

[0047] Figure 7 The combined objective function value of each algorithm increases with the number of iterations in the experiments of this invention. The change curve;

[0048] Figure 8 The average value obtained by the HWOA-QL algorithm under different discount factors and learning rates in the experiments of this invention is... Value diagram;

[0049] Figure 9 These are the results obtained by each algorithm on the P10-U3 instance under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values;

[0050] Figure 10 These are the results obtained by each algorithm on instance P21-U3 under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values;

[0051] Figure 11 These are the results obtained by each algorithm on the P68-U8 instance under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values;

[0052] Figure 12 These are the results obtained by each algorithm on instance P100-U24 under the weight coefficient combinations F1 and F3 in the experiments of this invention. Error analysis chart of the values. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0055] The remanufacturing process includes three key stages: disassembly, reprocessing, and reassembly. For example... Figure 1 As shown, this invention proposes a digital twin-driven reprocessing scheduling model (DT-RSS-DD-UT model) that considers differential delays and the insertion of urgent tasks, focusing on the scheduling optimization problem in the reprocessing stage. First, the DT-RSS-DD-UT model assumes that the common components for reprocessing are non-differentiated; the identification, appearance, and other characteristics of the components do not need to be differentiated before reassembly and can be applied across products in the reassembly process. Second, the DT-RSS-DD-UT model considers multiple equivalent parallel reprocessing production lines, allowing components to be reprocessed on any available line. Furthermore, the model defines a common component warehouse to store common components not immediately used for reassembly for later retrieval, while non-common components are directly transferred to the reassembly workshop after reprocessing to await assembly.

[0056] During the scheduling process, urgent tasks may be inserted at any time, and these tasks have a higher priority than other tasks. Once an urgent task is detected, it needs to be reprocessed immediately. This invention achieves this process through a digital twin-driven dynamic scheduling architecture. This architecture establishes a real-time connection between the physical workshop and the virtual workshop, thereby enabling dynamic response to the demand for urgent task insertion. The virtual workshop continuously monitors and collects scheduling data from the physical workshop in real time. When an urgent task is detected, the virtual workshop will reschedule the task based on existing data, using models and optimization algorithms. Subsequently, the optimized rescheduling scheme is transmitted back to the physical workshop, which then rearranges the reprocessing tasks accordingly, thus effectively addressing the dynamic scheduling challenges posed by the insertion of urgent tasks.

[0057] The costs incurred from reprocessing parts and storing common components directly affect the economic benefits of remanufacturing. Meanwhile, actual remanufacturing companies typically prioritize ensuring timely completion of production tasks to achieve on-time product delivery. Therefore, the DT-RSS-DD-UT model constructed in this invention is a bi-objective optimization model, aiming to simultaneously minimize total cost and total reprocessing time. The following assumptions were considered when constructing the DT-RSS-DD-UT model: (1) The reprocessing line will not be interrupted during operation. (2) The idle time cost of the reprocessing line is negligible. (3) When an urgent task is inserted, it will be prioritized through a rolling window mechanism. (4) There is no priority distinction between urgent tasks. (5) There is no priority distinction between non-urgent tasks. (6) All reprocessing lines are available at the start of reprocessing. (7) All components required for the product can immediately enter the reassembly stage after reprocessing is completed.

[0058] Specifically, the objective function is constructed as follows: The DT-RSS-DD-UT model aims to minimize the total cost and total time of part reprocessing. Minimizing the total cost... The objective function is shown in formula (1).

[0059] (1)

[0060] Minimize total time The objective function is shown in formula (2).

[0061] (2)

[0062] The comprehensive objective function of the DT-RSS-DD-UT model is shown in Equation (3).

[0063] (3)

[0064] in, It is the value of the comprehensive objective function. and are the weight coefficients of the two objective functions, respectively, whose value range is [0, 1] and are subject to constraint (4). is the cost required to rework a component. is the unit storage cost of a generic component. is the number of the nth generic component in the inventory. is the start time of reassembly of a product. is the start time of reassembly of a product. is the start time of reassembly of a product. is the start time of reassembly of a product. is the start time of reassembly of a product, where is the total number of products. is the nth component of a product. is the nth component of a product. is the nth component of a product, where is the total number of components of a product.

[0065] (4)

[0066] Meanwhile, the DT-RSS-DD-UT model satisfies the following constraints:

[0067] Equation (5) ensures that a component only needs to be reworked on one rework line.

[0068] (5)

[0069] Equation (6) ensures that a component can only be reworked on one rework line at a time.

[0070] (6)

[0071] where is the end time of reworking another component on the rework line , which is completed before the component is reworked. Equations (7) and (8) define the start and end times of reworking each component on the rework line.

[0072]

[0073] (7)

[0074] (8)

[0075] Set B is used to record the types of generic components whose inventory is not sufficient to support the reassembly of products , as shown in equation (9).

[0076] ​​ (9)

[0077] Since reassembly must start after all its required components are reprocessed, the start time of reassembly is the latest end time of reprocessing for all required components of the product, as shown in equation (10).

[0078] (10)

[0079] where, denotes the latest reprocessing end time of non-common components required by product denotes the latest reprocessing end time of common components required by product denotes the order statistics, which is used to calculate the end time of reprocessing for the common components in stock shortage when reassembling product , as shown in equation (11).

[0080] (11)

[0081] where, is the dynamic threshold selected from the set , and satisfies the following condition: among all possible values of , the smallest one is selected, so that at least values of are less than or equal to . is the th reprocessing line, where is the total number of reprocessing lines. is the th common component, where is the total number of common component types. is a binary decision variable, where 1 indicates that component is the th common component, and 0 otherwise. is a binary decision variable, where 1 indicates that component is reprocessed on reprocessing line , and 0 otherwise. is a binary decision variable, where 1 indicates that component is an urgent task, and 0 otherwise. is the number of the th common component required by reassembling product . is the disassembled product​​​ The end time. For components The time required for further processing. For components The start time for reprocessing. For components The end time for further processing.

[0082] For example, there are 4 types of general-purpose components: , , and The end time for reprocessing these components , , and The values ​​are {11, 17, 7, 9}. Reassembling a certain product requires 3 of these common parts, but currently only 1 of these common parts is in stock. Therefore, This indicates that at least two more general-purpose parts of this type need to be manufactured. To ensure timely completion... At least two reprocessing of this type of common parts must be completed within the specified time. The possible values ​​are {9, 11, 17}. The minimum value is 9, which means that the earliest end time for processing two more general-purpose parts of this type is 9, corresponding to the second order statistic of the set {11, 17, 7, 9}.

[0083] The digital twin system architecture consists of two interconnected parts: a physical space and a virtual space. Monitoring, analysis, and optimization of the physical space are achieved through information interaction between the two spaces. In this invention, the physical space refers to the physical workshop, and the virtual space refers to the virtual workshop generated by mapping the physical workshop. In the digital twin-driven dynamic scheduling architecture, data transmission and interaction between the physical and virtual workshops are achieved through advanced communication technologies and network protocols. This process mainly includes three stages: First, the virtual workshop acquires data from the physical workshop in real time through a data sensing and acquisition system (including sensors, IoT devices, and other monitoring devices); then, the scheduling model within the virtual workshop adjusts and optimizes the scheduling scheme based on the acquired data using dynamic scheduling strategies, responding to the insertion of urgent tasks through timely rescheduling; finally, the optimized scheduling scheme generated by the virtual workshop is transmitted back to the physical workshop to guide the implementation of actual reprocessing operations.

[0084] To solve the dynamic scheduling problem of urgent task insertion more efficiently, this invention proposes an improved digital twin-driven hierarchical dynamic rescheduling strategy based on a rolling window mechanism. First, to balance global and local optimization, the scheduling problem is processed in layers, with the physical workshop serving as the global layer L. globalis responsible for long-term planning, while the virtual workshop is responsible for local layer L local is responsible for real-time adjustment. Secondly, to avoid repeated global calculations and improve scheduling efficiency, a rolling window mechanism is applied and expanded in the rescheduling strategy. In the optimized rolling window mechanism, the reprocessing of a component represents a task. For example, the reprocessing of component is represented as task 3-2. The virtual workshop contains three task windows: the first window (W1) represents the window of completed tasks; the second window (W2) represents the window of tasks being scheduled, including tasks being reprocessed and tasks planned to be reprocessed; and the third window (W3) represents the window of tasks waiting to be scheduled. Figure 2 is a schematic diagram of the combination of physical and virtual windows, and the inserted emergency tasks are 5-1, 5-2, and 5-3.

[0085] Based on the foregoing preparations, the embodiment proposes a reprocessing scheduling optimization method driven by digital twinning, as shown in Figure 3 , which specifically includes the following steps:

[0086] Step 1: generate a reprocessing initial scheduling scheme in the physical workshop, and move the reprocessing initial scheduling scheme to the third window of the virtual workshop.

[0087] Step 2: after the scheduling task starts to execute, the virtual workshop executes the rescheduling strategy.

[0088] Step 2-1: according to the reprocessing data collected from the physical workshop, the tasks currently being reprocessed and the tasks about to be reprocessed are moved from the third window to the second window based on the rolling window mechanism.

[0089] Since there are differences in the time required for reprocessing each component, and the task load in each window also varies at different time periods. Therefore, the rolling frequency used in the existing rolling window mechanism is fixed and cannot be dynamically changed according to the window load. To make up for the above defects, the embodiment proposes a dynamic rolling frequency adjustment strategy.

[0090] When there is no emergency task insertion, the current rolling frequency will be adjusted based on the reference rolling frequency according to the current load of window W2, as shown in formulas (12) and (13).

[0091] (12)

[0092] (13)

[0093] wherein, is the time required to complete the tasks in the current window W2. is the time needed to complete the tasks in the initial window W2, i.e., taking the time needed to execute the tasks in the first move-in of window W2 as the reference. is an adjustment factor with a value range of [1, 2]. According to the current scroll frequency, step 2-1 of the next iteration is triggered for execution.

[0094] When an emergency task is inserted, the window will be immediately scrolled and the scheduling scheme will be updated, i.e., the next iteration will be immediately started, and step 2-1 will be executed.

[0095] Step 2-2, if no emergency task insertion is detected, step 2-5 is executed; otherwise, step 2-3 is executed.

[0096] Step 2-3, after the task currently being processed is reprocessed, all reprocessed tasks in the second window are moved to the first window.

[0097] Step 2-4, the emergency task is moved to the second window.

[0098] Step 2-5, based on the objective function of minimizing the total cost and the total time (i.e., the comprehensive objective function of the DT-RSS-DD-UT model), the whale optimization algorithm is used to solve, and the rescheduling scheme of all tasks in the second window is obtained.

[0099] The basic whale optimization algorithm solves mathematical optimization problems by simulating the hunting behavior of whale groups, mainly including three basic stages: surrounding prey, bubble net attack, and random search. The whale optimization algorithm has been widely used to solve scheduling optimization problems due to its superior global search ability and adaptability. However, the basic whale optimization algorithm is suitable for solving continuous optimization problems and needs to be improved to solve discrete optimization problems. In addition, the local search ability of the basic whale optimization algorithm is relatively weak, which easily leads to premature convergence to a local optimal solution. To effectively solve the DT-RSS-DD-UT model, this embodiment proposes a hybrid whale optimization algorithm (HWOA-QL algorithm), which mainly improves as follows:

[0100] (1) A two-stage solution representation method based on matrix is proposed to reasonably encode the feasible solutions of the DT-RSS-DD-UT model. In the position updating process, the discreteness of the solution is maintained through the rounding mechanism.

[0101] (2) The leader, follower and scout roles in the artificial bee colony algorithm are introduced into the whale group, and the whale individuals are divided into one of the three roles, and a role conversion mechanism is constructed to balance global and local search.

[0102] (3) A strategy selection mechanism based on Q-learning is designed to adaptively select strategies such as Levy flight, spiral search and random search, so as to enhance the search ability of the algorithm in the solution space.

[0103] As shown in Figure 4 , the execution process of the whale optimization algorithm of the embodiment is as follows:

[0104] (1) Population initialization is performed based on a random strategy, and each individual in the population represents a rescheduling scheme. In order to more intuitively represent the model feasible solution and task scheduling scheme, the present application proposes a two-stage solution representation method based on a matrix, as shown in Figure 5 .

[0105] The first stage is reprocessing line allocation. This stage uses a one-dimensional matrix (the index starts from 1) composed of elements to represent the reprocessing line allocation of the parts for reprocessing. Specifically, the reprocessing line allocation of each part is represented by a one-dimensional matrix, and the element at the corresponding position in the one-dimensional matrix is set to 1, indicating that the part is reprocessed on the reprocessing line corresponding to the sequence number; the element at the corresponding position in the one-dimensional matrix is set to 0, indicating that the part is not reprocessed on the reprocessing line corresponding to the sequence number; for example, indicates that the part is reprocessed on the reprocessing line .

[0106] The second stage is reprocessing sequencing, which is represented by a two-dimensional matrix with 3 rows, and an example of the encoding is as follows: . The first row represents the product index, the second row represents the part index, and the third row represents the reprocessing line allocation index obtained from the first stage, and the order of the third row index value corresponds to the execution order of the reprocessing task. Each column in the two-dimensional matrix represents that the part of the product is reprocessed on the reprocessing line corresponding to the sequence number, for example, the first column indicates that the first part of the product is reprocessed on the first position of the reprocessing line ; the fifth column indicates that the first part of the product is reprocessed on the second position of the reprocessing line , and so on.

[0107] Meanwhile, the framework and parameters of Q-learning are defined:

[0108] (A) State definition: the state of an individual (i.e., a feasible solution) is defined as S, which includes the change of the role and the fitness value, as shown in formula (14).

[0109] ​​(14)

[0110] wherein, represents the role of the current individual in the population and satisfies , is the change in the fitness value of the current individual in this iteration, as shown in equation (15). is a discretized representation of

[0111] (15)

[0112] (16)

[0113] wherein, is the current iteration number, is the fitness value of the current individual at the th iteration.

[0114] (B) Action Definition: The action (i.e., the selection of the position update strategy) of an individual is defined as . The actions involved in this embodiment include selecting Levy flight, spiral search, and random search as the update strategy.

[0115] (C) Reward Definition: The reward is designed based on the fitness function improvement, which is used to guide the learning process of the algorithm, and its calculation is shown in equation (17):

[0116] (17)

[0117] wherein, is the fitness value of the current leader at the th iteration.

[0118] (D) Update of Q-value: represents the expected value of the long-term cumulative reward obtained by an individual after performing action in a given state , which is used to evaluate the pros and cons of taking a certain action in a specific state, thereby helping the individual to choose the optimal update strategy in different states. The calculation method of is shown in equation (18), and it is updated according to equation (19).

[0119] (18)

[0120] (19)

[0121] wherein, represents the expectation function; is the offset of the future iteration number, i.e., the iteration number after the current iteration; is the discount factor to balance the current reward and the future reward; represents the discounted sum of the future rewards from the current iteration; represents the state of the individual at the iteration; represents the action performed by the individual at the iteration; is the learning rate to control the degree of influence of new information on the Q value; represents the new state of the individual, represents the new action selected by the individual.

[0122] (E) Exploration and exploitation: To avoid the algorithm falling into a local optimal solution, a greedy strategy is adopted to balance the selection of high Q value actions and the probability of random search, as shown in equation (20).

[0123] (20)

[0124] where, represents the probability of random search at the iteration, which is calculated by equation (21).

[0125] (21)

[0126] (2) Calculate the fitness value and the value by taking the objective function as the fitness function.

[0127] (3) Perform the role conversion mechanism: inspired by the artificial bee colony algorithm, the HWOA-QL algorithm proposed in this embodiment divides the whale swarm into leaders, followers and scouts to balance global search and local search, where each whale represents an individual. At initialization, first, the individual with the highest fitness value in the current population is regarded as the leader. The fitness value is directly measured by the comprehensive objective function value , and the smaller the objective function value, the better the fitness. Subsequently, the remaining individuals are randomly divided into followers or scouts. The followers perform local search around the leader, while the scouts perform global search in the solution space.

[0128] In each iteration, the role of the whale individual will be dynamically adjusted according to its fitness value and search history:

[0129] (A) If the fitness value of a certain individual is better than the current leader, the individual is set as the new leader.

[0130] (B) If the optimal fitness value of the follower remains unchanged after multiple (10) iterations, the individual is set as a scout.

[0131] (C) If the fitness value of the current leader is overtaken by the fitness value of other individuals, the leader is randomly set as a follower or a scout.

[0132] (4) Strategy selection mechanism: The HWOA-QL algorithm of the embodiment adopts Levy flight, spiral search of the basic whale optimization algorithm, and random search as the update strategy of the individual. Subsequently, the optimal update strategy is dynamically selected for each individual based on Q-learning, so that the individual continuously approaches the leader, thereby improving the global search ability of the algorithm. Specifically:

[0133] (A) For the leader individual, the spiral search is used by default for position update. When the leader falls into a local optimal solution, the Levy flight strategy is used for position update, as shown in formula (22).

[0134] (22)

[0135] wherein, represents the updated leader position; represents the position of the current leader; represents the position of a randomly selected non-leader individual; is a random number in the range [-1, 1]; and are random numbers subject to standard normal distribution ; and represents the distance between the randomly selected individual and the current leader individual, as shown in formula (23).

[0136] (23)

[0137] (B) For the follower individual, the position update strategy selection is completely determined by Q-learning, as shown in formula (24).

[0138] (24)

[0139] wherein, represents the updated individual position; represents the position of the current individual at the th iteration; is a random number for controlling the strength of random disturbance, which is usually set to 0.1; is a random number subject to standard normal distribution ; and The distance between the current individual and the current leader individual is represented by formula (25).

[0140] (25)

[0141] (C) For individual scouts, the Levy flight strategy is always used for position updates, as shown in formula (26).

[0142] (26)

[0143] (5) Discretize the solution and calculate the reward value and update it. value.

[0144] (6) Determine whether the iteration termination condition is met. If it is met, output the global optimal solution as the optimal rescheduling scheme; otherwise, return to step (2) to continue execution.

[0145] Steps 2-6: Send the rescheduling plan to the physical workshop for execution.

[0146] Step 2-7: Clear the first and second windows of the virtual workshop and return to step 2-1 to continue execution until there are no tasks that need to be rescheduled in the third window.

[0147] This invention includes an experimental design comprising a series of simulation experiments, comparative experiments, and statistical analysis experiments, aimed at evaluating the performance of the proposed HWOA-QL algorithm. All experiments were implemented using Python and run on a personal computer with a 64-bit Windows 11 operating system, an AMD Ryzen 7 8845H 8-core processor (3.80GHz), and 32GB of RAM.

[0148] (I) Experimental Design:

[0149] Three baseline algorithms were selected for algorithm performance comparison, including the basic whale optimization algorithm (WOA) (from the literature Mirjalili, S., & Lewis, A. (2016). The whale optimization algorithm. Advances in Engineering Software, 95, 51-67.), the improved hybrid particle swarm optimization algorithm (IPSO) (from the literature Liao, J., & Lin, C. (2019). Optimization and simulation of job-shop supply chain scheduling in manufacturing enterprises based on particle swarm optimization. International Journal of Simulation Modelling, 18(1), 187-196.), and the improved ant colony algorithm (IACO) (from the literature Zhu, Y. Z., Song, Q., & Li, M. (2025). Multi-AGV multitask collaborative scheduling based on an improved ant colony algorithm. International Journal of Advanced Robotic Systems, 22(1), 17298806241312784.). To ensure the reliability and stability of the experimental results, all experiments were run independently for 16 times.

[0150] Since there is a lack of relevant benchmark test sets in the field of remanufacturing scheduling, this experiment randomly generates a number of simulation problem instances within a given interval. All problem instances involved in the experiment have been uploaded to the Figshare database. Among them, the name of each problem instance contains the number of products and the number of urgent tasks, for example, P35-U5 indicates that the instance contains 35 products and 5 randomly inserted urgent tasks.

[0151] (II) Parameter design: To ensure that all algorithms are compared under the optimal performance condition, this experiment conducted three groups of simulation experiments to determine the common parameters of all comparative algorithms. First, simulation experiments were carried out on instance P35-U5 to evaluate the performance of each algorithm under different population sizes. The maximum number of iterations of the algorithm was set to 1000, the initial population size was set to 10, and was increased to 150 by 10 as a step. The algorithm used the comprehensive objective function value T as the evaluation index, and its change curve with the increase of population size is shown in Figure 6

[0152] As shown in Figure 6 , with the increase of population size, the optimization performance of each algorithm is improved. However, regardless of the change of population size, the solution obtained by HWOA-QL algorithm is better than that of other baseline algorithms, indicating that the optimization performance of the proposed HWOA-QL algorithm in solving the DT-RSS-DD-UT model (the specific implementation is the digital twin driven reprocessing scheduling optimization method of the embodiment) is better than that of other baseline algorithms. Since most of the baseline algorithms can obtain good solutions when the population size is 130, the population size of the subsequent experiment is set to 130.

[0153] Further, the experiment conducted simulation experiments on instance P35-U5 to evaluate the performance of each algorithm under different iteration numbers. The maximum number of iterations of the algorithm was set to 1000, and the population size was set to 130. All algorithms used the comprehensive objective function value as the evaluation index, and its change curve with the increase of iteration number is shown in Figure 7 .

[0154] As shown in Figure 7 , with the increase of iteration number, the comprehensive objective function value of each algorithm presents a downward trend, indicating that the solution obtained by the algorithm is continuously optimized. Among them, the HWOA-QL algorithm can obtain the optimal solution in a smaller number of iterations, indicating that the solution space search ability of the proposed algorithm in solving the DT-RSS-DD-UT model is better than that of other baseline algorithms. Since all algorithms can converge within 980 iterations, the number of algorithm iterations in the subsequent experiment is set to 980.

[0155] In addition, the experiment conducted simulation experiments on instance P35-U5 to evaluate the performance of HWOA-QL algorithm under different discount factors and learning rates . The maximum number of iterations of the algorithm was set to 980, the population size was set to 130, and the comprehensive objective function value was used as the evaluation index. The experimental results are presented by a heat map, as shown in Figure 8 ​The performance of the HWOA-QL algorithm under the current discount factor and learning rate is better (corresponding to a smaller average value) is represented by blue, and the performance of the HWOA-QL algorithm under the current discount factor and learning rate is worse (corresponding to a larger average value) is represented by red.

[0156] As Figure 8 shown, when the discount factor is 0.9 and the learning rate is 0.7, the HWOA-QL algorithm can obtain the optimal solution. Therefore, in the subsequent experiments, the discount factor and learning rate of the HWOA-QL algorithm are set to 0.9 and 0.7, respectively.

[0157] (Three) Performance analysis:

[0158] To comprehensively evaluate the performance of the HWOA-QL algorithm in solving the DT-RSS-DD-UT model, this experiment generates 11 groups of problem instances and compares the performance of the HWOA-QL algorithm with other baseline algorithms under different combinations of objective function weight coefficients (i.e., different and ). The comparison results are shown in Tables 1-6. Tables 1-3 present the optimal values and average values of the comprehensive objective function value T obtained by each algorithm on each instance under different weight coefficient combinations, represented by "optimal value" and "average value", respectively. Tables 4-6 show the CPU computation time and standard deviation of the comprehensive objective function value T of each algorithm on each instance under different weight coefficient combinations, represented by "time" and "standard deviation", respectively.

[0159] As shown in Tables 1-3, under different weight coefficient combinations, the optimal values and average values of T obtained by the HWOA-QL algorithm on most instances are better than those of other baseline algorithms, indicating that the HWOA-QL algorithm has better optimization performance than other baseline algorithms in solving the DT-RSS-DD-UT model.

[0160] As shown in Tables 4-6, under different weight coefficient combinations, the standard deviation of T obtained by the HWOA-QL algorithm on most instances is smaller than that of other baseline algorithms, indicating that the HWOA-QL algorithm has better stability than other baseline algorithms in solving the DT-RSS-DD-UT model.

[0161] However, the CPU computing time of the HWOA-QL algorithm on most instances is longer than that of the basic whale optimization algorithm, but shorter than that of the improved ant colony algorithm and the improved particle swarm optimization algorithm. At the same time, the HWOA-QL algorithm is superior to the basic whale optimization algorithm in terms of solution quality and stability, so the solution efficiency of the HWOA-QL algorithm can meet the actual application requirements, and a good balance between computing time and solution quality is achieved.

[0162] Table 1 Optimal and average values of each algorithm (weight combination F1: = 0.3, = 0.7)

[0163]

[0164] Note: The optimal value is indicated in bold, and "↓" indicates that the smaller the value is, the better.

[0165] Table 2 Optimal and average values of each algorithm (weight combination F2: = 0.5, = 0.5)

[0166]

[0167] Note: The optimal value is indicated in bold, and "↓" indicates that the smaller the value is, the better.

[0168] Table 3 Optimal and average values of each algorithm (weight combination F3: = 0.7, = 0.3)

[0169]

[0170] Note: The optimal value is indicated in bold, and "↓" indicates that the smaller the value is, the better.

[0171] Table 4 CPU computing time and standard deviation of each algorithm (weight combination F1: = 0.3, = 0.7)

[0172]

[0173] Note: The optimal value is indicated in bold, and "↓" indicates that the smaller the value is, the better.

[0174] Table 5 CPU computing time and standard deviation of each algorithm (weight combination F2: = 0.5, = 0.5)​​​​​​​​​​​​​​​ Standard deviation of values (combination of weight coefficients F2: = 0.5, = 0.5)

[0175]

[0176] Note: The optimal value is indicated in bold, and "↓" means the smaller the value, the better.

[0177] Table 6 CPU computation time and Standard deviation of values (combination of weight coefficients F3: = 0.7, = 0.3)

[0178]

[0179] Note: The optimal value is indicated in bold, and "↓" means the smaller the value, the better.

[0180] (IV) Statistical analysis:

[0181] To further verify the stability of the HWOA-QL algorithm, error analysis and statistical analysis were carried out. Specifically, by selecting some instances (P10-U3, P21-U3, P68-U8 and P100-U24), the error analysis of the comprehensive objective function values obtained from 16 independent running experiments was carried out under the condition of weight coefficient combination F1 and F3. The experimental results are presented in the box plot as shown in Figures 9-12 .

[0182] First, in most instances, the median and average of the values obtained by the HWOA-QL algorithm are lower than those of other baseline algorithms, indicating that the HWOA-QL algorithm has better solution space search ability. Second, in most instances, the box length of the HWOA-QL algorithm is smaller than that of other baseline algorithms, indicating that the HWOA-QL algorithm has lower dispersion of experimental results, i.e., higher stability. Third, from the quartile range, in most instances, the optimal value and the worst value of obtained by the HWOA-QL algorithm are better than those of other baseline algorithms, indicating that the algorithm can obtain higher quality solutions.

[0183] To further analyze whether there is a significant difference between the experimental results obtained by the HWOA-QL algorithm and other baseline algorithms, and statistically verify that the HWOA-QL algorithm is better than other baseline algorithms, Wilcoxon test with a significance level of 0.05 was carried out. The test was completed using SPSS statistical software, and the statistical results under different weight coefficient combinations are shown in Table 7.

[0184] As shown in Table 7, in most cases, the experimental results obtained by the HWOA-QL algorithm have significant differences compared with other baseline algorithms (i.e., p<0.05), indicating that the HWOA-QL algorithm is significantly superior in a statistical sense.

[0185] Table 7 p-value of experimental results obtained by HWOA-QL algorithm under different combinations of weight coefficients compared with other baseline algorithms

[0186]

[0187] Note: * indicates that the p-value is lower than the significance level of 0.05.

[0188] (V) Actual case study:

[0189] Through the actual case of a certain company, this experiment verifies the applicability and superiority of the proposed DT-RSS-DD-UT model and HWOA-QL algorithm in real scenarios. Two common recycled products of the company, front cylinder protection frame and water system, are taken as experimental objects.

[0190] In this actual case, the reprocessing workshop includes 6 cleaning machines, 1 polishing machine, 3 grinding machines, 5 milling machines, 8 cold welding stations, 8 welding workstations, 8 repair stations, 2 laser cladding machines, and 3 milling machines, etc. equipment can form about 6-8 reprocessing lines. The number of products that need to be remanufactured per day is about 2-10, of which the number of urgent tasks is about 2-5, and the number of components included in a single product is about 10-100. The reprocessing path of each component is: cleaning → replacement → old equipment secondary assembly → spare parts renovation → debugging.

[0191] Based on the above actual production data, this experiment carried out performance comparison experiments of HWOA-QL algorithm and other baseline algorithms, and the experimental results are shown in Table 8.

[0192] As shown in Table 8, the HWOA-QL algorithm can obtain better solutions in the actual production scene, which verifies the effectiveness and superiority of the HWOA-QL algorithm in solving the DT-RSS-DD-UT model from the practical level.

[0193] Table 8 Comparison experimental results of each algorithm on actual cases

[0194]

[0195] Note: The optimal value is indicated by bold, and “↓” indicates that the smaller the value, the better.

[0196] The application provides a novel digital twin driven reprocessing scheduling model considering differential delay and emergency task insertion, aiming at minimizing the total cost and total time of reprocessing.

[0197] As one of the sub-problems of remanufacturing scheduling, the reprocessing scheduling problem belongs to a typical NP-hard problem, and swarm intelligence optimization algorithms have been widely used by scholars due to their strong adaptability in this problem. Whale optimization algorithm is often used by scholars to solve complex optimization problems due to its excellent global search ability and algorithm simplicity. Therefore, in order to further improve the optimization performance of the algorithm, the application provides a hybrid whale optimization algorithm combined with a Q-learning mechanism. The algorithm designs a two-stage solution representation method based on a matrix; a strategy selection mechanism based on Q-learning is constructed to improve the solution space search ability; and a role conversion mechanism combined with an artificial bee colony algorithm is introduced to balance the global and local search ability of the algorithm. Finally, a large number of simulation experiments and actual case studies verify the effectiveness and practicability of the model and algorithm.

[0198] In another embodiment, the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the digital twin driven reprocessing scheduling optimization method of the application.

[0199] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0200] The technical features of the above-mentioned embodiments can be combined in any way. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0201] The above-mentioned embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A digital twin-driven reprocessing scheduling optimization method, characterized in that, The reprocessing scheduling optimization method driven by digital twin technology maps the physical workshop of reprocessing to generate a virtual workshop, and the virtual workshop has three windows. Step 1: Initialize and generate the initial reprocessing schedule in the physical workshop, and move the initial reprocessing schedule to the third window of the virtual workshop; Step 2: After the scheduled task begins execution, the virtual workshop executes a rescheduling strategy, including: Step 2-1: Based on the reprocessing data collected from the physical workshop, the tasks currently undergoing reprocessing and those about to undergo reprocessing are moved from the third window to the second window using a rolling window mechanism. The rolling window mechanism includes: when no urgent task is inserted, the current rolling frequency is adjusted based on the current load of the second window according to the baseline rolling frequency; when an urgent task is inserted, the window is rolled immediately. Step 2-2: If no emergency task insertion is detected, proceed to step 2-5; otherwise, proceed to step 2-3. Steps 2-3: After the currently reprocessing task is completed, move all completed reprocessing tasks in the second window to the first window; Steps 2-4: Move the emergency task to the second window; Steps 2-5: Based on the objective function of minimizing total cost and total time, the whale optimization algorithm is used to solve the problem and obtain the rescheduling scheme for all tasks in the second window. The solution of the whale optimization algorithm is represented by a matrix-based two-stage solution. The first stage is the allocation of reprocessing lines. The allocation of reprocessing lines for each component is represented by a one-dimensional matrix. If the element at the corresponding position in the one-dimensional matrix is ​​set to 1, it means that the component is reprocessed on the reprocessing line with the corresponding sequence number; if the element at the corresponding position in the one-dimensional matrix is ​​set to 0, it means that the component is not reprocessed on the reprocessing line with the corresponding sequence number. The second stage is reprocessing sorting, represented by a two-dimensional matrix with 3 rows. The first row of the two-dimensional matrix represents the product index, the second row represents the component index of the product, and the third row represents the sequence index of the reprocessing line of the component in the product obtained from the first stage. The order of the index values ​​in the third row of the two-dimensional matrix corresponds to the execution order of reprocessing. Each column of the two-dimensional matrix represents the component of the product being reprocessed on the reprocessing line with the corresponding sequence number. Steps 2-6: Send the rescheduling plan to the physical workshop for execution; Step 2-7: Clear the first and second windows of the virtual workshop and return to step 2-1 to continue execution until there are no tasks that need to be rescheduled in the third window.

2. The digital twin-driven reprocessing scheduling optimization method according to claim 1, characterized in that, In the whale optimization algorithm, the whale pod is divided into leaders, followers, and scouts. During initialization, the whale with the best fitness value is set as the leader, and the remaining whales are randomly assigned as followers or scouts. Followers perform local searches based on the leader, while scouts perform global searches in the solution space.

3. The digital twin-driven reprocessing scheduling optimization method according to claim 2, characterized in that, In each iteration, the dynamic adjustment strategy for the whale's role is as follows: If there is a whale among the followers or scouts with a fitness value higher than that of the current leader, then set that whale as the new leader and randomly assign the original leader to a follower or scout. If the optimal fitness value of a follower remains unchanged throughout multiple iterations, then the follower is designated as a scout.

4. The digital twin-driven reprocessing scheduling optimization method according to claim 2, characterized in that, The whale position update strategy in the whale optimization algorithm is as follows: For the leader, when it gets stuck in a local optimum, the Levy flight strategy is used for position update; otherwise, the spiral search strategy is used for position update. For followers, the position update strategy is determined by a reinforcement learning algorithm, which can be a spiral search strategy, a Levy flight strategy, or a random search strategy. For the scouts, the Levy flight strategy is used for position updates.

5. The digital twin-driven reprocessing scheduling optimization method according to claim 4, characterized in that, The location update strategy is determined by a reinforcement learning algorithm, including: State definition: The change in a whale's role and individual fitness value is defined as a state; Action definition: The position update strategy is defined as an action, which includes the spiral search strategy, the Levy flight strategy, and the random search strategy; Reward definition: If the current leader's fitness value is greater than the updated fitness value of the current individual, the reward value is twice the difference between the current leader's fitness value and the updated fitness value of the current individual; if the current individual's fitness value before the update is greater than the updated fitness value of the current individual, the reward value is the difference between the current individual's fitness value before the update and the updated fitness value of the current individual; otherwise, the reward value is the difference between the current individual's fitness value before the update and the updated fitness value of the current individual. Based on the defined state, action, and reward, generate the optimal position update strategy for each follower.

6. The digital twin-driven reprocessing scheduling optimization method according to claim 5, characterized in that, The reinforcement learning algorithm employs - Greedy strategy, based on probability Choose a random search strategy, with probability Select the action with the highest Q value. It is the current iteration number. Indicates the first The probability of performing a random search in the next iteration.

Images