A human-robot collaborative partial destructive disassembly line balancing design method

Abstract

This invention discloses a human-machine collaborative method for balancing locally destructive dismantling lines. The method includes: dividing decommissioned electromechanical products into dismantling tasks; determining the dismantling priority relationship between tasks, as well as the dismantling profit, dismantling direction, selectable operators, and selectable dismantling modes for each task; determining the dismantling time and cost for each task by different operators under different dismantling modes; establishing a bilateral human-machine collaborative dismantling line model for locally destructive dismantling guided by workstations, smoothing indicators, robot numbers, and profit, and converting it into a mixed-integer linear programming model, while setting dismantling constraints; constructing an encoding / decoding strategy based on the dismantling constraints, and solving it using an improved multi-objective artificial bee colony algorithm based on reinforcement learning to obtain the dismantling scheme. This invention can obtain a dismantling line balancing scheme with superior overall performance in a short time, significantly improving dismantling efficiency and economic benefits, while reducing energy consumption and environmental impact.

Description

Technical Field

[0001] This invention relates to the field of human-machine collaborative partial destructive dismantling line design for decommissioned electromechanical products, and specifically to a balanced design method for a human-machine collaborative partial destructive dismantling line. Background Technology

[0002] The rapid development of science and technology has accelerated the iteration of electromechanical products, shortening their service life and resulting in a large number of retired electromechanical products. These waste products not only contain a large amount of renewable resources but also environmentally harmful components and materials. Failure to recycle and dispose of these waste products in a timely and standardized manner will not only waste resources but also pollute the environment and further endanger human survival. The recycling of waste products has certain economic value, is an important part of building remanufacturing, ensures the smooth progress of the circular economy, and is also an effective measure to achieve green environmental protection. Many dismantling companies use dismantling lines to dismantle and recycle waste electromechanical products, especially waste products such as automobiles and construction machinery. The use of a human-machine collaborative, partially destructive dismantling model is more in line with the complex dismantling production situation and can significantly enhance dismantling efficiency and improve economic benefits.

[0003] However, the balancing problem of a bilateral human-machine collaborative dismantling line with partial destructive dismantling is an NP-hard combinatorial optimization problem. Current dismantling companies mostly plan the operators and tasks at each workstation based on production experience or mathematical models. This makes it difficult to efficiently and accurately plan large-scale dismantling tasks and allocate dismantling operators, leading to low efficiency, high costs, low profits, and high energy consumption in the dismantling production line. Furthermore, existing technologies do not consider the destructive dismantling mode of human-machine collaboration, making it difficult to meet actual production requirements and failing to comprehensively balance profit and cost, resulting in poor overall performance of the dismantling solution. Summary of the Invention

[0004] Existing dismantling line design methods suffer from low efficiency, high cost, low economic benefits, and high energy consumption when processing waste electromechanical products. Furthermore, the dismantling modes are not suitable for actual production conditions, posing a significant technical challenge in this field. To address this, this invention provides a human-machine collaborative partially destructive dismantling line balancing design method. This method involves constructing a mixed-integer linear programming model with optimization objectives for the number of workstations, smoothing index, number of robots, and profit. It also designs an improved multi-objective artificial bee colony algorithm based on reinforcement learning. This provides theoretical and technical support for balancing human-machine collaborative partially destructive dismantling lines for retired electromechanical products, thereby achieving a dismantling solution with superior overall performance by balancing the efficiency, profit, and cost of dismantling production lines for dismantling enterprises.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a human-machine collaborative method for balancing locally destructive disassembly lines, the method comprising the following steps: Step S1: Determine the priority of dismantling based on the spatial location of the parts in the decommissioned electromechanical products, the dismantling process, and the actual condition; Step S2: Establish a bilateral human-robot collaborative dismantling line model for local destructive dismantling guided by the number of workstations, smoothing index, number of robots, and profit, and transform it into a standard mixed integer programming model; Step S3: Based on the dismantling characteristics of retired electromechanical products, an improved multi-objective artificial bee colony algorithm based on reinforcement learning is proposed. An encoding and decoding strategy based on dismantling constraints is constructed, and a combination operation of 20 states and 12 domain search strategies is designed. Step S4: Based on the non-dominated solution set obtained from the optimization of the design algorithm, draw the corresponding Gantt chart and provide guidance for enterprise decision-makers when breaking down the production line layout.

[0006] Preferably, the expression for the mixed-integer linear programming model described in step S2 is: ; In the formula, f1 represents the target number of workstations in the dismantling production line, f2 represents the target smoothing index, f3 represents the target total number of robots, and f4 represents the enterprise's profit target for dismantled products.

[0007] Preferably, the target expression for the number of workstations on the disassembly production line is: ; In the formula, W represents the set of workstations, and when workstation w is started, s w =1, otherwise 0; The expression for the smoothing indicator target is: ; In the formula, I represents the set of decomposition tasks, and C represents the set of tasks to be decomposed. T Indicates the rhythm of the disassembly line; T i This represents the dismantling time of task i; K={1, 2} represents the set of left and right sides of the workstation, k=1 for the left side and k=2 for the right side; when task i is assigned to the s side of workstation w, x iws =1, otherwise 0; The expression for the target total number of robots is: ; In the formula, r wk =1 indicates that the operator on the k-side of workstation w is a robot; otherwise, it is 0. The expression for a company's product profit target breakdown is: ; In the formula, G 1i For the profit of non-destructive dismantling task i, G 2i For the profit of destructive dismantling task i; m i =1 indicates that disassembly task i uses a destructive disassembly mode; otherwise, it is 0. i This represents the cost of disassembling task i. P represents the unit time cost of robot standby; f This represents the unit time cost after a pair of workstations are started.

[0008] Among them, there are non-linear variables, which are linearized.

[0009] Preferably, the method for solving the problem using the improved multi-objective artificial bee colony algorithm based on reinforcement learning in step S3 includes the following steps: Step S3001: Initialize algorithm parameters, setting the population size, maximum number of algorithm iterations, and Limit value; Step S3002: Initialize the two populations based on the decomposition priority relation matrix, decomposition task type constraints, heuristic rules, and encoding strategy, and generate individual encoding sequences; Step S3003: Construct a decoding strategy based on individual encoding and decomposition constraints to complete the decoding of individual sequences in the population; Step S3004: Calculate the objective function value F of the population, filter non-dominated solutions, and update the external file Q; Step S3005: Based on the decomposition constraints, the response to the algorithm environment state s through reinforcement learning and greedy rules are... ε The search strategy is dynamically adjusted, and the stages of hiring bees, observing bees, and scouting bees are executed respectively; Step S3006: In the hiring peak stage, determine the domain search operator through Q-Learning and perform the domain search; Step S3007: During the bee observation phase, if rand i If the condition is met, then the local neighborhood search operation determined by Q-Learning is executed; otherwise, proceed to the next step. rand is a random number. Step S3008: In the scout bee phase, determine the number of times an individual has not been updated, v. i If the value is set to Limit, a random mapping swap operation is performed; otherwise, proceed to the next step. Step S3009: Calculate the objective function value F of each individual in the population, determine the state s of the algorithm, filter non-dominated solutions, and update the external file Q; Step S3010: If the number of iterations has not reached the maximum number of iterations Imax, proceed to step S3005; otherwise, proceed to the next step. ​Step S3011: The algorithm terminates, outputs the non-dominated solution in the external file Q, and obtains the decomposition scheme.

[0010] Preferably, in the initial stage of a population, heuristic rules include: 1) To minimize the number of workstations, all disassembly tasks should be performed with the minimum disassembly time. For disassembly mode sequences, tasks that can be disassembled by destructive means should be disassembled by destructive means, and for operator sequences, tasks that can be disassembled by robots should be disassembled by robots. This will minimize the disassembly time of the tasks. 2) To minimize the number of robots, disassembly tasks should avoid robot disassembly. For tasks that can be disassembled by workers, they should be disassembled by workers. 3) To minimize the smoothing index, all disassembly tasks should adopt the minimum disassembly time. For disassembly mode sequences, the disassembly tasks should adopt destructive disassembly as much as possible. For operator sequences, the disassembly tasks should adopt robotic disassembly as much as possible. This can minimize the disassembly time of the task. 4) To maximize profits, the dismantling tasks should be carried out using robotic dismantling and non-destructive dismantling as much as possible. For dismantling mode sequences, non-destructive dismantling should be used for tasks that can be dismantled using non-destructive dismantling, and for operator sequences, robotic dismantling should be used for tasks that can be dismantled using robots. The above heuristic rules apply to the decomposition pattern decision sequence S. M and dismantling task operator sequence S O There are separate heuristic rules for the decomposition task sequence.

[0011] Preferably, the encoding strategy includes decomposing the task execution sequence S. T Decomposition Mode Decision Sequence S M and dismantling task operator sequence S O ; Decomposition Mode Decision Sequence S M It consists of 1-2 variables, where 1 indicates that the task adopts a conventional decomposition mode, and 2 indicates that the task adopts a destructive decomposition mode, and satisfies the following conditions: 1) Hazardous dismantling tasks and high-value dismantling tasks are all handled using the conventional dismantling method. 2) For tasks that require destructive disassembly, adopt destructive disassembly mode 2; 3) The decision variables for other dismantling tasks are randomly selected from 1 and 2; Disassembly task operator sequence S O It consists of 1-3 variables, where 1 indicates that the disassembly task is performed manually, 2 indicates that the disassembly task is performed by a robot, and 3 indicates that the disassembly task is performed through human-machine collaboration, and satisfies the following conditions: 1) Hazardous dismantling tasks are performed by robots to reduce harm to humans; 2) When there are high-value tasks or immediate tasks that are dangerous, manual dismantling will be carried out when destructive dismantling is required; 3) When the disassembly task is a human-machine collaborative disassembly, the operator sequence is 3; 4) The disassembly operator variable for other disassembly tasks is randomly selected from 1 and 2; The task decomposition sequence uses positive integer encoding, with each task number corresponding to one decomposition task; the method for constructing the task decomposition sequence includes the following steps: Step S3101: Select the priority matrix TP ij Tasks with all columns equal to 0 are added to the candidate task set, and already assigned tasks are removed from the set. Step S3102: Position p o Task i is randomly selected from the candidate task set; Step S3103: Update the priority matrix TP ij Remove the priority constraint on disassembly task i, and let TP ij =0, TP ji =0, p o ={1, ..., P} O}, TP ij and TP ji P represents the priority relationship between tasks. O Represents a set of locations; Step S3104: Repeat the above steps until the dismantling tasks are assigned, resulting in the dismantling task sequence S. T .

[0012] Preferably, the decoding strategy includes the following steps: Step S3201: Decoding begins: Input individual encoding sequence, decompose task information matrix KB, decompose task number |I|, priority relation matrix TP, beat C T ; Step S3202: Starting from the first disassembly task in the disassembly task sequence, start a new workstation, let w=1, s w =1, remaining allocable time T at the workstation R =C T The task set in the workstation is an empty set Ø; Step S3203: From the disassembly pattern sequence S M and disassembly operator sequence S O To understand the disassembly mode and the disassembly operator of the disassembly task, the disassembly time T of disassembly task i can be obtained. i Tasks are assigned to workstations for dismantling based on task attribute constraints or the minimum idle time strategy. Workstation operators must meet the task attribute constraints; otherwise, w = w + 1, T R =C TIf disassembly task i requires human-machine collaboration, then disassembly should be performed simultaneously from both sides of the workstation, then x iw1 =x iw2 =1, and simultaneously, by analyzing the end times of the previous dismantling tasks on both sides of the workstation of dismantling task i. +T j , +T j’ The moment the mission ends +T l Determine the start time of disassembly task i. Proceed to step S3205 if the condition is not met; otherwise, proceed to the next step. Step S3204: From the disassembly pattern sequence S M and disassembly operator sequence S O To understand the disassembly mode and the disassembly operator of the disassembly task, the disassembly time T of disassembly task i can be obtained. i Tasks are assigned to workstations for dismantling based on task attribute constraints or the minimum idle time strategy. Workstation operators must meet the task attribute constraints; otherwise, w = w + 1, T R =C T Determine whether disassembly task i is disassembled on one side of a certain workstation. If disassembly task i is disassembled on side k of workstation w, then x iwk =1, the end time of previously dismantled tasks on the same side of the same workstation. +T j The moment of dismantling of the immediate task +T j’ Determine the start time of disassembly task i. ; Step S3205: Determine whether the current disassembly task can be assigned to the current workstation: If +T>w×C T Then, a new workstation will be started, with w = w + 1, T R =C T If yes, proceed to the next step; otherwise, proceed directly to the next step. Step S3206: Place task i on side k of workstation w, or, for collaborative decomposition, place task i on both sides of workstation w, and calculate the remaining time T of workstation w. R =w×C T –( +T); Step S3207: Repeat steps S3202-S3206; otherwise, proceed directly to the next step. Step S3208: Output the disassembly task sequence for all workstations, and the decoding ends.

[0013] Preferably, the domain search during the stages of hired bees, observation bees, and scout bees is as follows: The domain search method in the hired bee phase includes: the first mapping exchange operator finds poor information fragments from the three-level sequence of individuals, deletes these fragments, and absorbs the fragments from the excellent individuals; the second mapping exchange operator finds high-quality information fragments from the three-level sequence of excellent individuals, finds these fragments in the individual sequence to obtain high-quality information fragments; the destruction and recombination operator randomly deletes poor information sequences from individuals based on priority relation constraints and problem task attribute constraints, and then randomly generates them.

[0014] The domain search methods observed in the bee phase include: First, in the swarm, the first domain search identifies poorly performing workstations and swaps the tasks dismantling them at the start or end of the workstation with tasks within neighboring workstations to address the poor performance caused by external factors. Second, the second domain search identifies poorly performing workstations and swaps the dismantling tasks within them to address the poor performance caused by internal factors. Third, when switching from worker to robot operators, workstations with longer dismantling times are switched to robots; when switching from robot to worker operators, workstations with shorter dismantling times are switched to workers. Fourth, when changing from non-destructive to destructive dismantling tasks, tasks with greater time changes are selected; when changing from destructive to non-destructive tasks, tasks with greater increased rewards are selected. The second group involves four types of search operations: First, a domain search involves swapping task sequences. Task 1 is randomly selected, and its preceding and succeeding tasks are identified as having the largest and smallest positions. Task 2 is then selected within this range, and its preceding and succeeding tasks are identified. If both tasks are within the preceding and succeeding range of the other, they are swapped. To avoid issues with operator constraints and decomposition pattern constraints, the task decomposition pattern and operator are swapped accordingly. Second, a domain search involves inserting tasks into a sequence. The preceding and succeeding tasks are identified, and insertion is performed only if priority constraints are satisfied. Third, a domain search involves changing the operator in an operation sequence. The operator of a task at a specific workstation is randomly changed. Fourth, a domain search involves changing the operator in an operation sequence. The operator of a task at a specific workstation is randomly changed.

[0015] The domain search method in the scout bee phase is a random mapping exchange operator that randomly selects parent individuals from the elite files of two populations. The probability of obtaining a corresponding parent individual is determined by the optimization goals exhibited by the parent and mother individuals. The information source for generating an individual is derived from the probability, and an individual is generated accordingly.

[0016] Preferably, Q-learning employs a domain search strategy to respond to environmental states, including: selecting actions for operators in the hired bee and scout bee phases of the algorithm using Q-learning. The algorithm's focus is on continuously approximating the true Pareto front by obtaining the non-dominated solution set through multi-objective optimization of the problem. The algorithm's state is defined by the change in multi-objective evaluation metrics between the non-dominated solution set obtained through the hired bee and scout bee phases and the previous generation's non-dominated solution set. The multi-objective evaluation metrics are Hypervolume (HV), Spacing, and Diversity Measure (DM). HV, Spacing can be combined into four cases: 1) HV>0, Spacing>0; 2) HV>0, Spacing ≤ 0; 3) HV ≤ 0, Spacing>0; 4) HV ≤ 0, Spacing ≤ 0; HV and The change in spacing is represented by the change in the non-dominated solution set after the algorithm selects and performs actions. The optimal value of each objective in the multi-objective optimization in its first-generation population. There is also the theoretical set of optimal values ​​for each objective. And contemporary ,pass The algorithm is divided into 5 states (0, 1), each with a value of 0.2, representing the stage of the algorithm. Here, M represents the total number of objectives, m represents the objective number, and g represents the generation. These together form 20 states, with 20 rows (s) and 12 columns (a) in the Q-table. Q The update formula for the table values ​​is as follows: ; In the formula, α γ is the learning rate; γ is the discount rate; and s' is the new state.

[0017] exist Q In the table update expression r The calculation of reward and penalty values ​​is as follows: ; Greed rule ε A probability r is randomly generated for selecting actions in the deterministic algorithm.a ,when ε <r a Its actions are randomly selected at that time. ε ≥r a Then, the action is selected based on the maximum value in the Q-table. Its expression is: .

[0018] In a second aspect, the present invention provides a computer device / apparatus / system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0019] Thirdly, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0020] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: (1) A mixed integer programming model was constructed for the equilibrium problem of a bilateral human-machine collaborative dismantling line with partial destructive dismantling, guided by the number of workstations, smoothing index, number of robots and profit. This model can provide theoretical and technical support for the human-machine collaborative partial destructive dismantling line of retired electromechanical products. (2) The combination of reinforcement learning and artificial bee colony algorithm makes up for the limitations of fixed strategies in static optimization. The algorithm is applied to the bilateral human-machine collaborative decomposition line balancing problem with local destructive decomposition, which ensures the feasibility and efficiency of the algorithm and lays a theoretical foundation for the application of dynamic decision optimization in combinatorial optimization problems. (3) The introduction of partial destructive dismantling and human-machine collaborative dismantling mode in the double-sided dismantling line effectively improves the efficiency of the dismantling production line and reduces costs. Its human-machine collaborative partial destructive dismantling line design method can efficiently obtain high-quality solutions, making the dismantling process of retired electromechanical products more accurate, significantly improving dismantling efficiency, and further reducing the impact on the environment. Attached Figure Description

[0021] Figure 1 This is a schematic flowchart of a human-machine collaborative partial destructive disassembly line balancing design method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a partially destructive disassembly line in human-machine collaboration according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a population 1 heuristic rule according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a population 2 heuristic rule according to an embodiment of the present invention; Figure 5 This is a schematic diagram of Q-learning according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the first mapping exchange operator according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the second mapping exchange operator according to an embodiment of the present invention; Figure 8 This is a schematic diagram of destructive recombination according to an embodiment of the present invention; Figure 9 This is a schematic diagram of a first group first domain search according to an embodiment of the present invention; Figure 10 This is a schematic diagram of a first group second domain search according to an embodiment of the present invention; Figure 11 This is a schematic diagram of a first group third domain search according to an embodiment of the present invention; Figure 12 This is a schematic diagram of a first group fourth domain search according to an embodiment of the present invention; Figure 13 This is a schematic diagram of a second group first domain search according to an embodiment of the present invention; Figure 14 This is a schematic diagram of a second group second neighborhood search according to an embodiment of the present invention; Figure 15 This is a schematic diagram of a second group third-domain search according to an embodiment of the present invention; Figure 16 This is a schematic diagram of the fourth domain search of the second group according to an embodiment of the present invention; Figure 17 This is a schematic diagram of a random mapping exchange operator according to an embodiment of the present invention; Figure 18 This is a diagram showing the priority relationship between vehicle disassembly task attributes and tasks according to an embodiment of the present invention; Figure 19 The figure shows a comparison between an improved multi-objective artificial bee colony algorithm based on reinforcement learning and a classic algorithm according to an embodiment of the present invention. Figure 20 The image shows an iterative graph comparing the HV and IGD indices of an improved multi-objective artificial bee colony algorithm based on reinforcement learning according to an embodiment of the present invention with those of the classic algorithm. Figure 21 This is a Gantt chart showing the disassembly line balance for automobile disassembly according to an embodiment of the present invention. Figure 22 This is a graph showing how the optimization objective of an embodiment of the present invention changes as the destructive dismantling task increases. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0023] like Figure 1 As shown, this invention provides a human-machine collaborative partially destructive dismantling line balancing design method, relating to the field of human-machine collaborative partially destructive dismantling line design for retired electromechanical products such as scrapped automobiles and construction machinery. Addressing the problems of low efficiency, high cost, low profit, and high energy consumption in existing dismantling line balancing methods, this invention achieves this through the following steps: First, based on the three-dimensional structure and dismantling process of the retired electromechanical products, dismantling tasks are divided and the priority relationships between tasks are determined; second, based on the dismantling capability and hazardousness of components, dismantling modes are determined, including conventional dismantling and destructive dismantling; then, a mixed-integer linear programming model is established with the number of workstations, smoothing index, number of robots, and profit as optimization objectives; finally, an encoding and decoding strategy is designed in conjunction with dismantling constraints, and an improved multi-objective artificial bee colony algorithm based on reinforcement learning is used to solve the problem and generate a dismantling scheme. This method can obtain a dismantling line balancing scheme with superior overall performance in a short time, significantly improving dismantling efficiency and economic benefits, while reducing energy consumption and environmental impact. Verification through examples shows that the technology of the present invention can achieve a workstation load balancing rate increase of up to 62% and a profit increase of 1.8% to 2.9%, which is significantly better than existing methods.

[0024] Before describing the embodiments, the design of the bilateral human-machine collaborative disassembly line for partial destructive disassembly will be explained: For large, decommissioned machinery products, such as automobiles and construction machinery, a human-machine collaborative partial destructive dismantling method can be adopted. To improve the efficiency, safety, and revenue of the dismantling production line, a double-sided layout dismantling line is used. Therefore, this invention proposes a design method for a double-sided human-machine collaborative dismantling line with partial destructive dismantling. The core problem addressed by this method is the balancing problem of a double-sided human-machine collaborative dismantling line with partial destructive dismantling. In the balancing problem of a double-sided human-machine collaborative dismantling line with partial destructive dismantling, such as... Figure 2 As shown, the task involves both conventional and destructive dismantling modes, and can be dismantled manually, by robots, or through human-robot collaboration. Under constraints of task type, priority, and cycle time, the task and operators are assigned to workstations. The challenge lies in how to rationally allocate operators and tasks to workstations, decide on the dismantling mode, satisfy multiple dismantling constraints, and optimize dismantling evaluation metrics.

[0025] The disassembly tasks are categorized into six types based on the condition of each component: ordinary tasks, hazardous tasks, high-value tasks, tasks requiring destructive disassembly, human-robot collaborative disassembly tasks, and related tasks. Each type of task has corresponding constraints. Hazardous tasks may involve chemicals harmful to humans or the environment and must be disassembled by robots. Hazardous tasks, which prevent the leakage of hazardous substances and thus harm to humans or the environment, and high-value tasks, which are highly profitable for the company, must be disassembled non-destructively. Tasks requiring destructive disassembly, due to corrosion or the use of riveting processes, necessitate destructive disassembly. Human-robot collaborative tasks, due to their complexity or inherent hazards, are time-consuming, labor-intensive, or potentially dangerous for individual workers; therefore, human-robot collaborative disassembly saves time and reduces hazards. Human-robot collaborative disassembly must begin and end simultaneously to reduce worker workload. Related tasks refer to tasks where destructive disassembly may affect the leakage of hazardous substances from hazardous tasks or the sale of high-value tasks; therefore, destructive disassembly must be performed by workers.

[0026] This invention provides a human-machine collaborative method for balancing locally destructive disassembly lines, comprising the following steps: Step S1: Determine the priority of dismantling based on the spatial location of the parts in the decommissioned electromechanical products, the dismantling process, and the actual condition.

[0027] This step primarily involves dividing dismantling tasks based on the component attributes of retired electromechanical products. It determines the dismantling profit, dismantling direction, selectable operators, available dismantling modes, and the costs of manual and robotic dismantling under different modes, as well as the costs of setting up workstations and the dismantling time. Subsequently, based on all the information regarding the dismantling tasks identified for the retired electromechanical products, the type of dismantling task, constraints, and the cycle time of the dismantling workstations will be determined.

[0028] Specifically, based on the 3D information of the decommissioned electromechanical products, the information of all components is determined. Dismantling tasks are then divided according to the relationships and attributes between components. Based on the 3D spatial structure and dismantling process sequence of the decommissioned electromechanical products, the dismantling priority relationships between dismantling tasks are determined, and a priority relationship matrix and priority relationship diagram are constructed accordingly. Based on the dismantling task attributes, worker and robot attributes, the dismantling time, energy consumption, and revenue of dismantling tasks under different dismantling modes are determined.

[0029] Step S2: Establish a bilateral human-machine collaborative dismantling line model for local destructive dismantling guided by the number of workstations, smoothing index, number of robots, and profit, and transform it into a standard mixed integer programming model.

[0030] For ease of explanation, the symbols and decision variables used in the embodiments of this invention are as follows: i、 j Task index number; w : Paired workstation index number; M : Maximum number; W A collection of paired workstations, | W The number of workstations; I A collection of tasks, | I | represents the number of tasks; k : Left and right indexes of paired workstations; K The set of left and right sides of a pair of workstations; l Index of destructive and non-destructive disassembly; L : A combination of destructive and non-destructive disassembly; o Index of workers and robots o = 1 represents a worker. o = 2 represents a robot; O A combination of workers and robots; a Index of disassembly patterns a = 1 indicates non-destructive disassembly. a = 2 indicates destructive dismantling; A A collection of decomposition patterns; : Non-destructive dismantling tasks for workers i Time, The time allotted for the worker's destructive dismantling task i; : Non-destructive disassembly tasks for robots i Time, The time required for task i, which involves the destructive dismantling of the robot; : The unit time cost for workers' non-destructive dismantling tasks. The unit time cost of a worker's destructive dismantling task; : The unit time cost for non-destructive disassembly tasks performed by robots. The unit time cost of a robot's destructive dismantling task; : The unit time cost of robot standby; P f The unit time cost after a pair of workstations are turned on; P h Additional unit time cost when dismantling harmful tasks; G li : G 1i For the profit of non-destructive dismantling task i, G 2i For the profit of destructive dismantling task i; H T : A collection of harmful tasks; D T A collection of tasks with demand attributes; G DN A collection of destructive dismantling tasks; C O : A collection of collaborative task breakdowns; F G If the task within the set involves destructive dismantling, it cannot be performed by the robot alone; A L The task is located in the set of decomposed parts on the left. A R The task is located in the set that is broken down on the right. TP ij Task priority matrix TP ij = 1 indicates the task i Need to be in the task j Previous disassembly task; C T : The number of ticks per workstation; Decision variables, task i The moment when disassembly began; s w : Decision variable, the on / off status of the pair of workstations, 1 indicates that the pair of workstations w is on, otherwise it is 0; x iwk Decision variable, 1 represents the task i lie in w workstationk Side disassembly, otherwise 0; y ijwk Decision variable, 1 represents the task i , j lie in w workstation k Side disassembly and task i In the mission j If the previous step is disassembled, otherwise the value is 0; z io Decision variables z i1 = 1 is the task i Disassembled by workers z i2 = 1 is the task i Disassembled by robots; p i Decision variables, task i Cost of disassembly; m i Decision variable, 1 represents the task i The disassembly mode is destructive; otherwise, it is 0. r wk : Decision variable, 1 represents a pair of workstations w of k If the side represents a robot, then it is 0; T i Decision variables, task i Disassembly time.

[0031] Define the objective function for optimization: The overall objective expression is: .

[0032] Sub-objective 1: The number of paired workstations on the production line can be represented as: .

[0033] Sub-objective 2: Decompose the production line smoothing index, which can be expressed as: .

[0034] Sub-objective 3: The total number of robots can be represented as: .

[0035] Sub-objective 4: The company's product profit breakdown can be expressed as: .

[0036] Set constraints, including: Constraint 1: Task allocation, its expression is: This formula means that, apart from human-machine collaborative disassembly tasks, each task can only be assigned to one workstation on the left or right side for disassembly. ; The two equations indicate that the human-machine collaborative disassembly task requires the joint disassembly by operators at the left and right workstations of the same workstation; ; ; The formula indicates that tasks with a left-direction attribute are broken down by the workstation on the left. ; The formula indicates that tasks with a right-direction attribute are decomposed from the workstation on the right. .

[0037] Constraint 2: The dismantling task is assigned by workers or robots, and its expression is: This formula represents that, apart from human-machine assisted disassembly tasks, all other tasks are disassembled by a worker or robot. ; The formula indicates that the human-robot collaborative disassembly task requires the joint disassembly of workers and robots; ; The harmful task is dismantled by the robot; ; If the related tasks are displayed in a destructive manner, they must be disassembled by workers. .

[0038] Constraint 3: Robot allocation, its expression is: The two equations indicate that when breaking down a human-machine collaborative task, one side of the same workstation must be a worker and the other side must be a robot. ; ; The formula constrains the number of robots to ensure it does not exceed the number of disassembly tasks. ; The two formulas constrain who breaks down the task and which type of operator is assigned to the workstation. ; .

[0039] Constraint 4: Task decomposition pattern constraint, its expression is: The disassembly mode for tasks requiring destructive disassembly is destructive disassembly. ; The formula indicates that the dismantling mode for harmful tasks and high-value tasks is non-destructive dismantling. .

[0040] Constraint 5: The time required for task decomposition, expressed as: The three formulas represent the relationship constraints between the disassembly time of all tasks and the disassembly operator and disassembly mode.

[0041] ; ; .

[0042] Constraint 6: The cost of breaking down the task, expressed as: The two equations represent the relationship between the profit obtained after breaking down all tasks and the decomposition time, decomposition operator, and decomposition mode. ; .

[0043] Constraint 7: The order of tasks is determined by the following expression: The two formulas ensure that when two tasks are disassembled at the same workstation, one will always be disassembled before the other.

[0044] ; ; This method ensures that the task will only be broken down once; ; The formula indicates that the order in which two tasks are disassembled from the same workstation satisfy a priority relationship. .

[0045] Constraint 8: On / Off Status of Paired Workstations: This formula indicates that the workstation is started when there is task decomposition in the workstation; ; The formula indicates that the workstation is shut down when there are no tasks to be disposed of. ; The formula indicates that the workstations are turned on in sequence; ; The formula states that the number of workstations must be greater than or equal to the theoretical number of workstations; ; The formula stipulates that the number of workstations must be less than or equal to the total number of dismantling tasks; .

[0046] Constraint 9: Task priority constraints and cycle time constraints, their expressions are as follows: The formula indicates that the start time of a successor task that satisfies a priority relationship must be later than the end time of its predecessor task. ; The formula indicates that tasks on the same workstation satisfy a sequential relationship in terms of dismantling order, and a later task must be dismantled after the earlier task has been dismantled. ; The meaning of the four formulas is that all tasks must satisfy the temporal and sequential relationship between work and the rhythm constraints of the task; ; ; ; .

[0047] Linearization includes: The equation in objective 2 contains nonlinear elements. x iwk T i Introducing variables u iwk = x iwk T i Therefore, the variable constraints are: ; ; ; .

[0048] Its objective 2 is changed to: .

[0049] According to the target formula 4 r wk x iwk T i To linearize, introduce Boolean variables. rxiwk Replace it with r wk x iwk , and non - negative variables d iwk Replace rx iwk T i The linearization variable constraint for it is: ; ; ; ; ; .

[0050] Its objective 4 is changed to: .

[0051] There are z io (1 - m i ) and z io m i non - linear elements in the constraint 5 and constraint 6. Introduce Boolean variables c ioa The constraint for it is: ; ; ; .

[0052] Its constraint 5 and 6 are changed to: ; ; ; ; .

[0053] There are also c ioa T i in the last two equations of the formula, so variables b ioa = cioa T i Therefore, the constraints on the variables are: ; ; ; .

[0054] Its format is changed to: ; .

[0055] Step S3: Based on the priority relationship and decomposition mode, construct the encoding and decoding strategy, and solve it using the improved multi-objective artificial bee colony algorithm based on reinforcement learning to obtain the decomposition scheme.

[0056] Specifically, based on the dismantling characteristics of retired electromechanical products, an improved multi-objective artificial bee colony algorithm based on reinforcement learning is proposed. An encoding and decoding strategy based on dismantling constraints is constructed, and a combination operation of 20 states and 12 domain search strategies is designed.

[0057] The method for solving the problem using the improved multi-objective artificial bee colony algorithm based on reinforcement learning includes the following steps: Step S3001: Initialize algorithm parameters, setting the population size, maximum number of algorithm iterations, and Limit value; Step S3002: Initialize the two populations based on the decomposition priority relation matrix, decomposition task type constraints, heuristic rules, and encoding strategy, and generate individual encoding sequences; Step S3003: Construct a decoding strategy based on individual encoding and decomposition constraints to complete the decoding of individual sequences in the population; Step S3004: Calculate the objective function value F of the population, filter non-dominated solutions, and update the external file Q; Step S3005: Based on the decomposition constraints, the response to the algorithm environment state s through reinforcement learning and greedy rules are... ε The search strategy is dynamically adjusted, and the stages of hiring bees, observing bees, and scouting bees are executed respectively; Step S3006: In the hiring peak stage, determine the domain search operator through Q-Learning and perform the domain search; Step S3007: During the bee observation phase, if rand i If the condition is met, then the local neighborhood search operation determined by Q-Learning is executed; otherwise, proceed to the next step. rand is a random number. Step S3008: In the scout bee phase, determine the number of times an individual has not been updated, v.​i If the value is set to Limit, a random mapping swap operation is performed; otherwise, proceed to the next step. Step S3009: Calculate the objective function value F of each individual in the population, determine the state s of the algorithm, filter non-dominated solutions, and update the external file Q; Step S3010: If the number of iterations has not reached the maximum number of iterations Imax, proceed to step S3005; otherwise, proceed to the next step. Step S3011: The algorithm terminates, outputs the non-dominated solution in the external file Q, and obtains the decomposition scheme.

[0058] In heuristic rules, optimization-oriented design is used as follows: ① For disassembly task sequences, in order to reduce idle time in workstations, priority is given to disassembly tasks with many follow-up tasks and long disassembly times. If a workstation needs to be changed when the last task is assigned, the task with the shortest idle time is selected; ② For disassembly mode sequences and disassembly operator sequences, combinations of destructive disassembly modes and robot operations, combinations of destructive disassembly modes and manual operations, and combinations of non-destructive disassembly modes and robot operations are preferred.

[0059] Specifically, in the initial stage of the population, the heuristic rules include: 1) To minimize the number of workstations, all disassembly tasks should be performed with the minimum disassembly time. For disassembly mode sequences, tasks that can be disassembled by destructive means should be disassembled by destructive means, and for operator sequences, tasks that can be disassembled by robots should be disassembled by robots. This will minimize the disassembly time of the tasks. 2) To minimize the number of robots, disassembly tasks should avoid robot disassembly. For tasks that can be disassembled by workers, they should be disassembled by workers. 3) To minimize the smoothing index, all disassembly tasks should adopt the minimum disassembly time. For disassembly mode sequences, the disassembly tasks should adopt destructive disassembly as much as possible. For operator sequences, the disassembly tasks should adopt robotic disassembly as much as possible. This can minimize the disassembly time of the task. 4) To maximize profits, the dismantling tasks should be carried out using robotic dismantling and non-destructive dismantling as much as possible. For dismantling mode sequences, non-destructive dismantling should be used for tasks that can be dismantled using non-destructive dismantling, and for operator sequences, robotic dismantling should be used for tasks that can be dismantled using robots. The above decision sequence S for the dismantling mode M and dismantling task operator sequence S O The heuristic rules for decomposing the task execution sequence are in Figure 3 and Figure 4 middle.

[0060] In the heuristic rules, population 1 generates individuals guided by optimization objectives 1 and 3 of the research problem, ensuring a relatively high-quality population in the initial stage. To increase population diversity, randomly generated initial solutions are incorporated. Therefore, population 1 uses 25% of the rules from rules 1 and 2, and 50% from randomly generated initial solutions. The rules regarding the task decomposition execution sequence are further refined. Figure 3 In Population 2, individuals are generated with optimization objectives 2 and 4 as the guiding principle. Excellent initial solutions accelerate the algorithm's progress. Since heuristic rules cannot cover all problem characteristics, randomly generated initial solutions are added. In Population 2, rules 3 and 4 each account for 25%, and randomly generated initial solutions account for 50%. Rules regarding the execution sequence of the task decomposition are... Figure 4 middle.

[0061] Specifically, in the encoding strategy, the sequence is allocated in three layers, which is the decomposition task sequence S. T Disassembly mode sequence S M and disassembly operator sequence S O The disassembly task sequence uses integer encoding, and the order of disassembly tasks is determined based on their priority relationships. The disassembly mode sequence also uses integer encoding, which, through a one-to-one correspondence with the position in the disassembly task sequence, describes whether the disassembly task employs conventional or destructive disassembly. The disassembly operator sequence uses integer encoding, bound to the position in the disassembly task sequence, describing whether the disassembly task is performed manually, by a robot, or through human-robot collaboration.

[0062] The coding steps are as follows: ① Input the number of disassembly tasks | I |, Deconstructing the information matrix KB Priority Relationship Matrix TP Population size N Task breakdown time T i ② Encode individuals in the population to determine the dismantling task sequence, dismantling pattern sequence, and dismantling operator sequence; ③ Determine the dismantling tasks based on the dismantling task priority matrix and heuristic rule ①. i Location; ④ Determine the decomposition mode and operator of the decomposition task based on heuristic rule ②; ⑤ Update the priority relation matrix. TP ⑥ Obtain a complete and feasible encoding sequence. Pop n = { S T ; S M ; S O};⑦ All individuals in the population complete the encoding. Half of the individuals in the entire population are generated using heuristic rules, and the other half are generated randomly while satisfying constraints. The encoded sequence of the entire population is output. Pop = { Popn}

[0063] Specifically, the encoding steps are as follows: The dismantling mode decision sequence S M It consists of 1-2 variables, where 1 indicates that the task adopts a conventional decomposition mode, and 2 indicates that the task adopts a destructive decomposition mode, and satisfies the following conditions: 1) Hazardous dismantling tasks and high-value dismantling tasks are all handled using the conventional dismantling method. 2) For tasks that require destructive disassembly, adopt destructive disassembly mode 2; 3) The decision variables for other dismantling tasks are randomly selected from 1 and 2; The disassembly task operator sequence S O It consists of 1-3 variables, where 1 indicates that the disassembly task is performed manually, 2 indicates that the disassembly task is performed by a robot, and 3 indicates that the disassembly task is performed through human-machine collaboration, and satisfies the following conditions: 1) Hazardous dismantling tasks are performed by robots to reduce harm to humans; 2) When there are high-value tasks or immediate tasks that are dangerous, manual dismantling will be carried out when destructive dismantling is required; 3) When the disassembly task is a human-machine collaborative disassembly, the operator sequence is 3; 4) The disassembly operator variable for other disassembly tasks is randomly selected from 1 and 2; The disassembly task sequence uses positive integer encoding, with each task number corresponding to a disassembly task.

[0064] Preferably, the method for constructing the decomposition task sequence includes the following steps: Step S3101: Select the priority matrix TP ij Tasks with all columns equal to 0 are added to the candidate task set, and already assigned tasks are removed from the set. Step S3102: Position p o Task i is randomly selected from the candidate task set; Step S3103: Update the priority matrix TP ij Remove the priority constraint on disassembly task i, and let TP ij =0, TP ji =0, p o ={1,…,P O}, TP ij and TP ji P represents the priority relationship between tasks. O Represents a set of locations; Step S3104: Repeat the above steps until the dismantling tasks are assigned, resulting in the dismantling task sequence S. T .

[0065] During decoding, the operation time of the disassembly task needs to be determined according to the disassembly mode and the operator. When assigning tasks, priority constraints and the order in the workstation need to be considered. When assigning operators to workstations, the task operators need to meet the requirements, and the idle time in the workstation should be minimized as much as possible, so as to ensure fewer disassembly workstations, higher efficiency, lower cost and higher profit.

[0066] The specific decoding process is as follows: Step S3201: Decoding begins: Input individual encoding sequence, decompose task information matrix KB, decompose task number |I|, priority relation matrix TP, beat C T ; Step S3202: Starting from the first disassembly task in the disassembly task sequence, start a new workstation, let w=1, s w =1, remaining allocable time T at the workstation R =C T The task set in the workstation is an empty set Ø; Step S3203: From the disassembly pattern sequence S M and disassembly operator sequence S O To understand the disassembly mode and the disassembly operator of the disassembly task, the disassembly time T of disassembly task i can be obtained. i Tasks are assigned to workstations for dismantling based on task attribute constraints or the minimum idle time strategy. Workstation operators must meet the task attribute constraints; otherwise, w = w + 1, T R =C T If disassembly task i requires human-machine collaboration, then disassembly should be performed simultaneously from both sides of the workstation, then x iw1 =x iw2 =1, and simultaneously, by analyzing the end times of the previous dismantling tasks on both sides of the workstation of dismantling task i. +T j , +T j’ The moment the mission ends +T l Determine the start time of disassembly task i. Proceed to step S3205 if the condition is not met; otherwise, proceed to the next step. Step S3204: From the disassembly pattern sequence S M and disassembly operator sequence S O To understand the disassembly mode and the disassembly operator of the disassembly task, the disassembly time T of disassembly task i can be obtained. i Tasks are assigned to workstations for dismantling based on task attribute constraints or the minimum idle time strategy. Workstation operators must meet the task attribute constraints; otherwise, w = w + 1, T R =C TDetermine whether disassembly task i is disassembled on one side of a certain workstation. If disassembly task i is disassembled on side k of workstation w, then x iwk =1, the end time of previously dismantled tasks on the same side of the same workstation. +T j The moment of dismantling of the immediate task +T j’ Determine the start time of disassembly task i. ; Step S3205: Determine whether the current disassembly task can be assigned to the current workstation: If +T>w×C T Then, a new workstation will be started, with w = w + 1, T R =C T If yes, proceed to the next step; otherwise, proceed directly to the next step. Step S3206: Place task i on side k of workstation w, or, for collaborative decomposition, place task i on both sides of workstation w, and calculate the remaining time T of workstation w. R =w×C T –( +T); Step S3207: Repeat steps S3202-S3206; otherwise, proceed directly to the next step. Step S3208: Output the disassembly task sequence for all workstations, and the decoding ends.

[0067] Reinforcement learning (Q-learning) strategies include: Q-learning learns optimal policies through interaction with the environment. It selects actions for operators in the "mercenary bee" and "scout bee" phases of the algorithm. The algorithm's focus is on continuously approximating the true Pareto front by obtaining the non-dominated solution set through multi-objective optimization. The algorithm's state is defined by the changes in multi-objective evaluation metrics between the non-dominated solution set obtained in the "mercenary bee" and "scout bee" phases and the previous generation's non-dominated solution set. Its multi-objective evaluation metrics include Hypervolume (HV), Spacing, and Diversity Measure (DM). HV, Spacing can be combined into four cases: 1) HV>0, Spacing>0; 2) HV>0, Spacing ≤ 0; 3) HV ≤ 0, Spacing>0; 4) HV ≤ 0, Spacing ≤ 0; HV and The change in spacing is represented by the change in the non-dominated solution set after the algorithm selects and performs actions. The optimal value of each objective in the multi-objective optimization in its first-generation population. There is also the theoretical set of optimal values ​​for each objective. And contemporary ,pass The algorithm is divided into 5 states (0, 1), each with a value of 0.2, representing the stage of the algorithm. Here, M represents the total number of objectives, m represents the objective number, and g represents the generation. These together form 20 states, with 20 rows (s) and 12 columns (a) in the Q-table. Q The update formula for the table values ​​is as follows: ; In the formula, α The learning rate; γ s' represents the discount rate and s' represents the new state.

[0068] exist Q In the table update expression r The calculation of reward and penalty values ​​is as follows: ; Greed rule ε A probability is randomly generated for selecting actions in a deterministic algorithm. r a ,when ε < r a Its actions are randomly selected at that time. ε ≥ r a Then according to Q The action to select the maximum value in the table. Its formula is: ; The hired bee phase includes: Three domain search strategies were designed for the hired bee: the first mapping exchange operator, the second mapping exchange operator, and the destruction-recombination operator. The first mapping exchange operator identifies poor information segments from the three-layer sequence of an individual, deletes these segments, and then absorbs these segments from the encoding sequences of excellent individuals. The operation process is as follows: Figure 6 The second mapping exchange operator: It identifies high-quality information fragments from the three-level sequences of excellent individuals, finds corresponding fragments in the individual sequences for information exchange, and absorbs these high-quality information fragments. The operation process is as follows: Figure 7The disruptive recombination operator: This operator avoids the excessive similarity of information sequences between individuals caused by the mapping and exchange operator. Its purpose is to find potentially high-quality information by shuffling small segments of the sequence, thus escaping the state of being trapped in a local optimum. The disruptive recombination operator randomly deletes the poor information sequences from individuals based on priority relation constraints and problem task constraints, and then randomly generates new ones. Its operation process is as follows: Figure 8 When a hired bee obtains a high-quality nectar source through information exchange during the hired bee phase, it discards the previous nectar source and resets the unupdated count to zero. If the obtained nectar source is worse than the previous nectar source, the unupdated count is updated.

[0069] The specific steps for the peak hiring phase are as follows: ① Input the peak hiring individual ② Determine the algorithm state through Q-learning s Based on this, a greedy rule is used to select the domain search operator. a ③ Select high-quality individuals from the resulting alternative group. Perform a domain search operation to obtain ④ If Then use the non-dominated solution. Replacement of hired peak individuals ;like Then maintain the individual hired bee Unchanged; ⑤ Output the final peak individual for employment .

[0070] The observation phase of bees includes: In the observation bee phase, a territorial search is designed, allowing the observation bees to conduct small-scale searches near nectar sources learned from the hired bees, thereby identifying potential high-quality nectar sources. To address the problem of finding potential solutions, multiple territorial search strategies are designed. In population 1, four territorial search strategies based on the problem's optimization objective are designed. To prevent rule-based territorial searches from encompassing too few search directions, four rule-free territorial search strategies are designed in population 2 to uncover potentially feasible solutions. If a better individual is found during the territorial search, the previous individual is replaced, and the unupdated count is reset to zero. If no better solution is found, the unupdated count is updated. Nectar sources for foraging are selected probabilistically. The higher the quality of the obtained nectar source, the higher its probability of being selected. The expression for the probability is: ; In the formula, Q i For individuals i The probability of being selected Represents an individual i sub-targets k The value; Represents the contemporary neutron target of the population k The optimal value.

[0071] In populations 1 and 2, the domain search strategy designed during the observation bee phase is as follows: In population 1, to conduct targeted domain search, and given the high probability of finding a good solution in this search direction, four domain search strategies based on the optimization objective were designed. Target domain search 1: Identify poorly performing workstations and swap tasks dismantled at the start or end of these workstations with tasks in neighboring workstations to address situations where poor workstation performance is caused by external factors. The operation is as follows: Figure 9 ; Target domain search 2: Identify poorly performing workstations and swap their internal tasks to resolve the issue of poor workstation performance caused by internal factors. The operation is as follows: Figure 10 Based on the target domain search 3: When the selected workstation operator is changed from a worker to a robot, the workstation with the longer disassembly time will be changed from a worker to a robot; when the selected workstation operator is changed from a robot to a worker, the workstation with the shorter disassembly time will be changed from a robot to a worker. The operation is as follows: Figure 11 ; Target domain search 4: When changing from non-destructive to destructive disassembly, select the task with the largest time change; when changing from destructive to non-destructive, select the task with the largest increase in benefit. The operation is as follows: Figure 12 .

[0072] In population 2, to prevent the rule-based neighborhood search in population 1 from missing potentially feasible solutions, a ruleless neighborhood search is performed to explore high-quality feasible solutions from more search directions. Four ruleless neighborhood searches are designed for population 1. Random Neighborhood Search 1: For task sequence swapping, randomly select task 1 and find the position of the largest task in its immediate predecessor sequence and the position of the smallest task in its immediate successor sequence. Within these ranges, select task 2 and find its immediate predecessor and successor tasks. If both task positions are within the immediate predecessor and successor ranges of the other, they are swapped. To avoid unmet operator constraints and decomposition pattern constraints, the task decomposition pattern follows the operator's swap. The operation is as follows: Figure 13 Random Neighborhood Search 2: For task sequence insertion, find the positions of the immediate predecessor and successor tasks, and perform the insertion operation only if the priority relationship constraint is satisfied. The operation is as follows: Figure 14 Random Neighborhood Search 3: Change the operator in the operation sequence, randomly change the operator of a task at a workstation, such as... Figure 15 Random Neighborhood Search 4: Change the disassembly task mode of the disassembly pattern sequence, randomly change the disassembly mode of the task, and perform the operation as follows: Figure 16 .

[0073] The specific steps of the peak hiring phase are as follows: ① Input the observed peak individuals ② Determine the algorithm population state through Q-learnings Based on this, a greedy rule is used to select the domain search operator. a ③ Obtained through the selected domain search strategy. ④ If Then use the non-dominated solution. Replace the observed peak individual ;like Then keep observing individual bees. Unchanged; ⑤ Output the final observed peak individual .

[0074] The reconnaissance bee phase includes: In the scout bee phase of the artificial bee colony algorithm, when the quality of a nectar source fails to improve after the territory search by the hired bee and observer bee phases for a specified limit number of times, the nectar source is abandoned, and a new nectar source is selected by the scout bee to replace it. A common method is to randomly generate a new nectar source, but this cannot guarantee its quality. Therefore, a method is designed to generate a new nectar source based on the territory search operation of existing excellent nectar sources. This method uses a random mapping exchange operator to randomly select parent bodies from the elite profiles of two populations. The probability of acquiring a corresponding parent body is determined by the optimization goals exhibited by the parent and mother bodies. The probability is calculated as follows: ; In the formula, Q i For individuals i The probability of being selected Represents an individual i sub-targets k The value; Represents the target of the parent body k The optimal value.

[0075] Its random mapping exchange strategy operation is as follows: Figure 17 .

[0076] The specific steps of the reconnaissance bee phase are as follows: ① Input the reconnaissance bee individual ① The non-dominated solution sets of the two groups; ② Individuals accessed Number of times not improved ① Determine whether an individual is discarded and regenerate it; ② After determining whether to perform a random mapping and exchange operation, select individuals from the non-dominated solution sets of the two populations. and individuals ④ Determine the probability in the operation; ④ Perform random mapping and exchange operations to generate individuals. and its original individuals ⑤ Perform the replacement; ⑤ Output the final reconnaissance peak individual .

[0077] Based on the non-dominated solution set obtained through optimization using the design algorithm, a corresponding Gantt chart is drawn to provide guidance for enterprise decision-makers when breaking down production line layout.

[0078] This embodiment uses a classic automotive disassembly case to construct a balanced disassembly line problem involving partial destructive disassembly on both sides, employing human-machine collaboration. The performance of the method described in this invention in practical engineering applications is analyzed. The automotive engineering case includes 74 disassembly tasks, the specific disassembly names of which are shown in Table 1. Information such as disassembly time and benefits for each task is shown in Table 2. The priority relationships between priority tasks and the task classification are also shown in Table 2. Figure 18 Other auxiliary parameters for the engineering case of the problem are: = 0.01 yuan / s, = 0.025 yuan / s = 0.0015 yuan / s, = 0.003 yuan / s = 0.0005 yuan / s, P f = 0.0001 yuan / s, P h = 0.005 yuan / s.

[0079] Table 1 Disassembly Name Table

[0080] Table 2 Information Table

[0081] To ensure fairness, the population and number of iterations were set to be the same for both algorithms. For IMOABC-QL, the parameters were set as follows: population N=80, number of iterations G=200, limit for no update=20, learning rate α=0.75, discount rate γ=0.1; the maximum number of no updates for MOABC was set to 20; the crossover rate for NSGA-II was 0.9, and the mutation rate was 0.1; the number of sampling points for HypE was 8888. To ensure fairness in the comparison, each algorithm was run 20 times on each case, with the same population size and number of iterations. Three multi-objective evaluation metrics—hypervolume (HV), inverted generational distance (IGD), and diversity measure (DM)—were used for evaluation, and the results are recorded below. Figure 19 .

[0082] Analysis of the results of the three evaluation metrics for algorithm comparison shows that IMOABC-QL outperforms the other algorithms in all three metrics, and its data distribution range differs somewhat from that of the other algorithms. In the HV metric, IMOABC-QL's stability is significantly better than the other algorithms. In the IGD metric, while NSGA-II, SPEA2, and HypE have some outliers comparable to IMOABC-QL's optimal value, their stability is poor and their data is more scattered, whereas IMOABC-QL's data is more concentrated. In the DM metric, IMOABC-QL shows better diversity and better coverage of the Pareto front. The comparison results indicate that the IMOABC-QL algorithm outperforms other optimization algorithms in stability, convergence, and diversity. Since the optimal value obtained by the algorithm reflects its capability, the data from one iteration of the IGD metric is used to illustrate this. Figure 20 As shown.

[0083] The iteration processes of the five algorithms with optimal HV values ​​of 1, 0.7982, 0.8146, 0.7975, and 0.7993, and optimal IGD values ​​of 0, 0.0446, 0.0115, 0.0191, and 0.0217, are all normalized data. Numerically, the algorithm performance in terms of convergence and uniformity is IMOABC-QL > MOABC > HypE > NSGA-II > SPEA2. To ensure fairness, their initialization methods are the same. IMOABC-QL has a poorer starting point in the iteration graphs of HV and IGD due to the randomness of its initialization, and its initial performance is not as good as the other algorithms. However, IMOABC-QL's optimization ability is significantly stronger than the other algorithms, and it can escape local optima to find higher quality solution sets.

[0084] In the above performance comparison, the extreme values ​​of the four optimization objectives were obtained as (8, 229659, 6, -13883.6587), of which the optimal value obtained by IMOABC-QL was (8, 229659, 6, -13883.6587); the optimal value obtained by MOABC was (8, 388674, 7, -13883.3461); and the optimal value obtained by NSGA-II was (8, 308321, 7, -1 The optimal value obtained by SPEA2 is (8, 330993, 7, -13883.3461); the optimal value obtained by HypE is (8, 334875, 6, -13883.3461). It can be seen that all algorithms can achieve the optimal value for objective 1, only IMOABC-QL and HypE can achieve the optimal value for objective 3, and only IMOABC-QL can achieve the optimal value for objectives 2 and 4. A Gantt chart is drawn with the three optimal solutions from the IMOABC-QL solution set, and the objective values ​​of each solution are... Figure 21 Option (a) 1 (8, 229659, 10, -13733.4747); Figure 21 Option (b) 2 (9, 598771, 6, -13491.7041); Figure 21 Scheme (c) 3 (8, 353328, 12, -13883.6587). Figure 21 In the diagram, the tasks highlighted in red represent destructive dismantling. Option 1 offers the most balanced workload for the workstations. The matrix scatter plot analysis shows a positive correlation between objective 1 and objective 2, with objective 1 being the minimum number of workstations. Compared to Option 2, it has fewer workstations, increasing the load balance between workstations by 62% and reducing the initial investment in setting up the factory by 11%. However, the number of robots increases because robot dismantling tasks require less time and cost compared to human workers, leading to a 66% increase in initial robot procurement costs. However, compared to Option 2, it has fewer destructive dismantling tasks, increasing profits by 1.8%. Compared to Option 3, the load balance between workstations increases by 35%, resulting in fewer robots and a corresponding 17% reduction in initial robot procurement costs. Option 2 aims to reduce the initial investment in robots by placing tasks that require disassembly (especially those involving hazards to humans) or human-robot collaboration within the same workstation, thus reducing the company's upfront investment in robots by 50% compared to Option 3. However, this increases the number of destructive disassembly tasks and workstations, resulting in a 13% increase in initial investment for workstation setup compared to Option 3, and consequently reducing profits. Option 3, to maximize profits, uses conventional disassembly for all tasks except those requiring destructive disassembly. Therefore, compared to Option 2, which has the most destructive disassembly tasks, its profit is 2.9% higher, and compared to Option 1, its profit is 1.1% higher.

[0085] by Figure 21 Taking the maximum profit scheme in (c) as an example, five tasks require destructive disassembly. The disassembly order and the disassembly operator remain unchanged, and the number of destructive disassembly tasks is continuously increased to observe the impact of destructive disassembly on the four optimization objectives. Figure 22 The analysis shows that during the period when the number of destructive dismantling tasks increased from 6 to 29, there was no change in the task operators. This is because the tasks involved in destructive dismantling were carried out by workers, indicating that the number of workstations and robots did not increase. The destructive dismantling did not lead to the initial investment in the establishment of workstations and the purchase of robots. However, the smoothing index increased slowly, and the load balance rate decreased, resulting in a 17% reduction in the corresponding workstation operating costs. But the profit decreased at a certain rate, down by 15% compared to the original plan, indicating that destructive dismantling reduces the profit from parts recycling.

[0086] In summary, existing dismantling line design methods for decommissioned electromechanical products suffer from problems such as incomplete dismantling modes, low efficiency, high cost, and low profit. This invention proposes a human-machine collaborative partially destructive dismantling line design method for decommissioned electromechanical products. It constructs a mixed-integer programming model for balancing the two-sided human-machine collaborative dismantling line problem using partially destructive dismantling, guided by the number of workstations, smoothing index, number of robots, and profit. Furthermore, it proposes an improved multi-objective artificial bee colony algorithm based on reinforcement learning, incorporating problem characteristics. This provides theoretical and technical support for human-machine collaborative partially destructive dismantling lines for decommissioned electromechanical products, balancing the efficiency, profit, and cost of dismantling production lines for dismantling enterprises, and achieving a dismantling solution with superior overall performance.

[0087] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; only preferred embodiments of the present invention are illustrated. The descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. As long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0088] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A human-machine collaborative method for balancing locally destructive disassembly lines, characterized in that, The method includes: S1. Divide retired electromechanical products into dismantling tasks, determine the dismantling priority relationship between dismantling tasks, as well as the dismantling profit, dismantling direction, selectable dismantling operators and selectable dismantling modes for each dismantling task, and determine the dismantling time and cost for each dismantling task by different dismantling operators under different dismantling modes. S2. Establish a bilateral human-machine collaborative dismantling line model for local destructive dismantling oriented by workstations, smoothing index, number of robots and profit, and transform it into a standard mixed integer linear programming model, while setting dismantling constraints. S3. Construct an encoding / decoding strategy by combining decomposition constraints, and solve the model using an improved multi-objective artificial bee colony algorithm based on reinforcement learning to obtain the decomposition scheme.

2. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 1, characterized in that, The method also includes: S4. Draw a Gantt chart based on the dismantling plan to guide the layout of the dismantling production line.

3. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 1, characterized in that, The expression for the mixed-integer linear programming model is: ； In the formula, F represents the total target, f1 represents the target number of workstations in the dismantling production line, f2 represents the smoothing index target, f3 represents the target number of robots, and f4 represents the profit target of the dismantled products. The target expression for the number of workstations in the dismantling production line is as follows: ； In the formula, W represents the set of workstations, and when workstation w is started, s w =1, otherwise 0; The expression for the smoothing indicator target is: ； In the formula, I represents the set of decomposition tasks, and C represents the set of tasks to be decomposed. T Indicates the rhythm of the disassembly line; T i This indicates the disassembly time of disassembly task i; x represents the time when disassembly task i is assigned to the s side of workstation w. iws =1, otherwise 0, s=1 represents the left side, s=2 represents the right side; The expression for the target total number of robots is: ； In the formula, r wk =1 indicates that the operator on side k in workstation w is a robot, otherwise it is 0; K={1,2} represents the set of left and right sides of the workstation, k=1 represents the left side, k=2 represents the right side; The expression for breaking down the product profit target is: ； In the formula, G 1i For the profit of non-destructive dismantling task i, G 2i For the profit of destructive dismantling task i; m i =1 indicates that disassembly task i uses a destructive disassembly mode; otherwise, it is 0. i This represents the cost of disassembling task i. P represents the unit time cost of robot standby; f This represents the unit time cost after a pair of workstations are started. The expressions for the smoothing index target and the decomposed product profit target, as well as the nonlinear variables in the decomposition constraints, are linearized.

4. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 1 or 3, characterized in that, The disassembly constraints include: Task allocation constraints: Except for human-machine collaborative disassembly tasks, each disassembly task can only be assigned to one workstation on the left or right side for disassembly; human-machine collaborative disassembly tasks must be disassembled collaboratively by the left and right sides of the same workstation; disassembly tasks with directional attributes are disassembled by the corresponding side workstation. Operator allocation constraints: Except for human-robot collaborative disassembly tasks, each disassembly task shall be disassembled by one worker or one robot; human-robot collaborative disassembly tasks shall be disassembled by a worker and a robot working together; hazardous tasks shall be disassembled by a robot; related tasks that involve destructive disassembly shall be disassembled by a worker. Robot allocation constraints: When performing human-robot collaborative disassembly tasks, each workstation must have a worker on one side and a robot on the other; the number of robots cannot exceed the number of disassembly tasks; there are corresponding constraints on the allocation of disassembly operators and workstations for each disassembly task. Disassembly mode constraints: Disassembly mode for tasks that must be destructive is destructive disassembly; disassembly mode for hazardous and high-value tasks is non-destructive disassembly. Disassembly time constraints: Constraints on the relationship between the disassembly time of all disassembly tasks and the disassembly operators and disassembly modes; Dismantling cost constraint: The relationship between the profit obtained after dismantling all dismantling tasks and the dismantling time, dismantling operators and dismantling modes. Disassembly task priority constraints: When two disassembly tasks are disassembled at the same workstation, one will always be disassembled before the other; a disassembly task will only be disassembled once; the disassembly order of two disassembly tasks disassembled at the same workstation satisfies the disassembly priority relationship; Workstation activation constraints: A workstation is activated when there are tasks to be disassembled, and is deactivated when there are no tasks to be disassembled; workstations are activated sequentially; the number of workstations is between the theoretical number of workstations and the total number of disassembly tasks. Task priority constraints and takt constraints: The start time of a successor task that satisfies the priority relationship must be later than the end time of the predecessor task; for tasks on the same workstation that satisfy the priority relationship in terms of dismantling order, the task that follows must be dismantled after the task that precedes it has been dismantled; all tasks must satisfy the temporal priority relationship with the work and the takt constraints of the tasks.

5. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 1, characterized in that, The model is solved using an improved multi-objective artificial bee colony algorithm based on reinforcement learning, including: Step S3001: Initialize algorithm parameters, setting the population size, maximum number of algorithm iterations, and Limit value; Step S3002: Initialize the two populations based on the decomposition priority relation matrix, decomposition task type constraints, heuristic rules, and encoding strategy, and generate individual encoding sequences; Step S3003: Construct a decoding strategy based on individual encoding and decomposition constraints to complete the decoding of individual sequences in the population; Step S3004: Calculate the objective function value F of the population, filter out non-dominated solutions, and update the external file Q; Step S3005: Based on the decomposition constraints, the response to the algorithm environment state s through reinforcement learning and greedy rules are... ε The search strategy is dynamically adjusted, and the stages of hiring bees, observing bees, and scouting bees are executed respectively; Step S3006: In the hiring peak stage, determine the domain search operator through Q-Learning and perform the domain search; Step S3007: During the bee observation phase, if rand i If the condition is met, then the local neighborhood search operation determined by Q-Learning is executed; otherwise, proceed to the next step. `rand` is a random number, and `Q` is the local neighborhood search operation. i For individuals i The probability of being selected;​ Step S3008: In the scout bee phase, determine the number of times an individual has not been updated, v. i If the value is set to Limit, a random mapping swap operation is performed; otherwise, proceed to the next step. Step S3009: Calculate the objective function value F of each individual in the population, determine the state s of the algorithm, filter non-dominated solutions, and update the external file Q; Step S3010: If the number of iterations has not reached the maximum number of iterations Imax, proceed to step S3005; otherwise, proceed to the next step. Step S3011: The algorithm terminates, outputs the non-dominated solution in the external file Q, and obtains the decomposition scheme.

6. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 5, characterized in that, The encoding strategy consists of a three-layer sequence allocation, namely, the task sequence S is decomposed. T Disassembly mode sequence S M and disassembly operator sequence S O ; Disassembly mode sequence S M It consists of 1-2 variables, where 1 indicates that the disassembly task adopts the conventional disassembly mode, and 2 indicates that the disassembly task adopts the destructive disassembly mode, and satisfies the following conditions: 1) Hazardous dismantling tasks and high-value dismantling tasks are all handled using the conventional dismantling method. 2) For tasks that require destructive disassembly, adopt destructive disassembly mode 2; 3) The decision variables for other dismantling tasks are randomly selected from 1 and 2; Disassembly operator sequence S O It consists of 1-3 variables, where 1 indicates that the disassembly task is performed manually, 2 indicates that the disassembly task is performed by a robot, and 3 indicates that the disassembly task is performed through human-robot collaboration, and satisfies the following conditions: 1) Hazardous dismantling tasks are handled by robots; 2) 2) When there are high-value tasks or immediate tasks that are dangerous, manual dismantling will be carried out when destructive dismantling is required; 3) When the disassembly task is a human-machine collaborative disassembly, the operator sequence is 3; 4) The disassembly operator variable for other disassembly tasks is randomly selected from 1 and 2; Each task number in the task decomposition sequence corresponds to one decomposition task. The construction method includes the following steps: Step S3101: Select the priority relation matrix TP ij Tasks with all columns equal to 0 are added to the candidate task set, and already assigned tasks are removed from the set. Step S3102: Position p o Task i is randomly selected from the candidate task set; Step S3103: Update the priority matrix TP ij Remove the priority constraint on disassembly task i, and let TP ij =0, TP ji =0, p o ={1, ..., P} O }, TP ij and TP ji P represents the priority relationship between tasks. O Represents a set of locations; Step S3104: Repeat the above steps until the dismantling tasks are assigned, resulting in the dismantling task sequence S. T ; The decoding strategy includes the following steps: Step S3201: Input individual encoding sequence, decompose task information matrix KB, decompose task number |I|, priority relation matrix TP, and beat C. T ; Step S3202: Starting from the first disassembly task in the disassembly task sequence, start a new workstation, let w=1, s w =1, remaining allocable time T at the workstation R =C T The task set in the workstation is an empty set Ø; where, when workstation w is started, s w =1, otherwise 0; Step S3203: From the disassembly pattern sequence S M and disassembly operator sequence S O Obtain the disassembly time T for disassembly task i. i Tasks are assigned to workstations for dismantling based on task attribute constraints or the minimum idle time strategy. Workstation operators must meet the task attribute constraints; otherwise, w = w + 1, T R =C T If disassembly task i requires human-machine collaboration, then disassembly will be performed simultaneously on both sides of the workstation. The start time of disassembly task i will be determined by using the end times of the previous disassembly tasks and the immediately preceding tasks on both sides of the workstation. Proceed to step S3205 if the condition is not met; otherwise, proceed to the next step. Step S3204: From the disassembly pattern sequence S M and disassembly operator sequence S O Obtain the disassembly time T for disassembly task i. i Tasks are assigned to workstations for dismantling based on task attribute constraints or the minimum idle time strategy. Workstation operators must meet the task attribute constraints; otherwise, w = w + 1, T R =C T Determine whether disassembly task i is disassembled on one side of a certain workstation. If disassembly task i is disassembled on side k of workstation w, then x iwk =1, determine the start time of dismantling task i by comparing the end time of the previously dismantled task on the same side of the same workstation with the end time of the immediately preceding task. ; Step S3205: Determine whether the current disassembly task can be assigned to the current workstation: If +T i >w×C T Then, a new workstation will be started, with w = w + 1, T R =C T Proceed to the next step if T is selected; otherwise, proceed directly to the next step. i Indicates the disassembly time of disassembly task i; Step S3206: Place task i on side k of workstation w, or, for collaborative decomposition, place task i on both sides of workstation w, and calculate the remaining time T of workstation w. R =w×C T –( +T i ); Step S3207: Repeat steps S3202-S3206; otherwise, proceed directly to the next step. Step S3208: Output the disassembly task sequence for all workstations, and the decoding ends.

7. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 6, characterized in that, Heuristic rules include: 1) To minimize the number of workstations, all tasks in the disassembly mode sequence that can be disassembled by destructive means will be disassembled by destructive means, and all tasks in the disassembly operator sequence that can be disassembled by robots will be disassembled by robots. 2) To minimize the number of robots, all tasks in the disassembly operator sequence that can be disassembled by workers will be disassembled by workers. 3) To minimize the smoothing index, destructive disassembly is adopted for all disassembly tasks in the disassembly pattern sequence, and robotic disassembly is adopted for all disassembly tasks in the disassembly operator sequence. 4) To maximize profits, disassembly tasks should be carried out using both robotic and non-destructive methods. For tasks in the disassembly mode sequence that can be disassembled using non-destructive methods, non-destructive methods should be used. For tasks in the disassembly operator sequence that can be disassembled using robots, robotic methods should be used.

8. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 5, characterized in that, The domain search strategies for the mercenary bee, observation bee, and scout bee phases are as follows: The domain search method in the bee-hiring stage includes: the first mapping exchange operator finds poor information fragments from the three-level sequence of individuals, deletes the fragments, and absorbs the sorting of the fragments from excellent individuals; the second mapping exchange operator finds high-quality information fragments from the three-level sequence of excellent individuals, finds the fragments in the individual sequence to obtain high-quality information fragments; the destruction and recombination operator randomly deletes poor information sequences from individuals based on priority relation constraints and problem task attribute constraints, and then randomly generates them; The domain search methods observed in the bee phase include: In the first type of swarm, the first domain search identifies poorly performing workstations and swaps the tasks dismantling at the start or end of these workstations with tasks within neighboring workstations; the second domain search identifies poorly performing workstations and swaps the dismantling tasks within them; the third domain search, when switching from worker to robot operator, switches workstations with longer dismantling times from worker to robot; when switching from robot to worker operator, switches workstations with shorter dismantling times from robot to worker; the fourth domain search, when changing from non-destructive to destructive dismantling tasks, selects tasks with larger time variations; when changing from destructive to non-destructive dismantling tasks, selects tasks with larger time variations. The first domain search involves swapping task sequences. A first task is randomly selected, and its preceding and succeeding tasks are identified as having the largest and smallest positions. A second task is then selected within this range, and its preceding and succeeding tasks are identified. If both tasks are within the preceding and succeeding ranges of the other, they are swapped. The task decomposition pattern and the decomposition operator are also swapped. The second domain search involves inserting tasks into a sequence. The preceding and succeeding task positions are identified, and insertion is performed only if priority constraints are met. The third domain search involves changing the operator in an operation sequence, randomly changing the operator of a task at a specific workstation. The fourth domain search involves changing the operator in an operation sequence, randomly changing the operator of a task at a specific workstation. The domain search method in the reconnaissance bee stage includes: the random mapping exchange operator randomly selects parent bodies from the elite files of two populations, determines the probability of acquiring the corresponding parent body based on the optimization goals shown by the parent and mother bodies, and derives the information source of the generated individual based on the probability, thereby generating the individual.

9. The human-machine collaborative locally destructive disassembly line balancing design method according to claim 8, characterized in that, Q-Learning addresses environmental conditions and responds to domain search strategies, including: The improved multi-objective artificial bee colony algorithm employs Q-learning to select actions for the operators in the mercenary bee and scout bee phases. The algorithm's state is defined by the changes in the non-dominated solution set obtained through neighborhood search in the mercenary and scout bee phases compared to the previous generation's non-dominated solution set in terms of multi-objective evaluation metrics. These multi-objective evaluation metrics include hypervolume (HV), spacing (Spacing), and diversity (DM). HV, Spacing can be combined into four cases: 1) HV > 0, Spacing > 0; 2) HV > 0, Spacing ≤ 0; 3) HV ≤ 0, Spacing > 0; 4) HV ≤0, Spacing ≤ 0; HV and The change in spacing is represented by the change in the non-dominated solution set after the algorithm selects and performs the action; the optimal value of each objective in the multi-objective optimization in the first generation population. Set of optimal values ​​for each objective And contemporary ,pass The (0, 1) group is divided into 5 states, where M is the total number of objectives, m is the nth objective, and g represents the generation. These states are combined into 20 states, with 20 rows of states s and 12 columns of actions a in the Q table. Q The update formula for the table values ​​is as follows: ； In the formula, α γ is the learning rate; γ is the discount rate; s' is the new state; exist Q In the table update expression r The calculation of reward and penalty values ​​is as follows: ； in, HV and Spacing represents the change in HV and Spacing, respectively.

10. A computer device / equipment / system, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

Images