Human-machine cooperation disassembly task dynamic planning method under uncertain operation time

Abstract

The invention discloses a man-machine cooperation disassembly task dynamic planning method under uncertain operation time. The method comprises the following steps: 1) constructing a man-machine cooperation disassembly information model under the uncertain operation time; 1.1) constructing an uncertain operation time description model of the product parts; 1.2) constructing a disassembling tool, direction and station position switching time model by considering the operation characteristics of the robot and the disassembling personnel; 1.3) constructing a product part disassembly constraint relation model; 2) taking the man-machine cooperation disassembly information model under the uncertain operation time constructed in the step 1) as an environment of a reinforcement learning algorithm, and establishing a man-machine cooperation disassembly task dynamic planning model; and 3) using the trained man-machine cooperation disassembly task dynamic planning model to generate a man-machine cooperation disassembly task planning scheme under the uncertain operation time. The uncertainty of the structural adhesive softening time is quantified by constructing the uncertain operation time man-machine cooperation disassembly information model, and the characterization problem of the disassembly operation time is solved.

Description

Technical Field

[0001] This invention relates to artificial intelligence technology, and more particularly to a dynamic planning method for human-machine collaborative task decomposition under uncertain operation time. Background Technology

[0002] Waste products may contain reusable parts, recyclable materials, and materials that may cause environmental pollution. Therefore, recycling waste products has become an important means of alleviating resource shortages, saving production costs, reducing environmental pollution, and promoting the development of a circular economy. Dismantling, as the primary step in waste product recycling, directly affects the effectiveness of subsequent resource recovery due to its operational quality and efficiency. Because waste products often have complex structures, diverse parts, and varying degrees of wear, industrial robots are often unable to independently complete complex dismantling tasks, while manual dismantling poses many potential safety and health problems. Combining the advantages of both manual and industrial robot dismantling, human-robot collaborative dismantling can balance dismantling efficiency and flexibility.

[0003] By rationally allocating dismantling tasks to dismantling personnel and industrial robots through dismantling task planning methods, the efficiency and flexibility of dismantling waste products can be improved. Most existing human-robot collaborative dismantling task planning methods generate optimal solutions based on a fixed dismantling time. However, in actual human-robot collaborative dismantling processes, factors such as the wear level, connection integrity, and connection status (e.g., the aging state of structural adhesive at the connection points) of waste product parts vary significantly, leading to a high degree of uncertainty in the dismantling time. This makes dismantling task planning methods based on fixed dismantling times difficult to apply to human-robot collaborative dismantling task planning problems in environments with uncertain operation times. For waste product dismantling scenarios involving structural adhesive bonding, this paper considers the uncertainty in dismantling time caused by differences in adhesive bonding performance and dynamically generates human-robot collaborative dismantling task planning schemes suitable for the current environment. This helps improve the applicability of dismantling task planning methods under uncertain operation times. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a dynamic planning method for human-machine collaborative task decomposition under uncertain operation time, which addresses the deficiencies in the prior art.

[0005] The technical solution adopted by this invention to solve its technical problem is: a dynamic programming method for human-machine collaborative task decomposition under uncertain operation time, comprising the following steps: 1) Construct a human-machine collaborative decomposition information model under uncertain operation time; 1.1) Construct a descriptive model for the uncertain operation time of product parts; 1.1.1) Taking the softening time of the structural adhesive as the object, it is set that when the degree of softening α of the structural adhesive reaches p, the structural adhesive softens to a disintegratable state, generally p=0.9; combined with temperature T, reaction surface area A, volume fraction of solvent in the reaction layer φ1, molar volumes of solvent and adhesive V1 and V2, and activation energy of the reaction. With diffusion activation energy And the solvent diffusion coefficient D0, the uncertain operation time of the product parts is calculated as follows;

[0006] 1.1.2) The boundary values ​​of the part disassembly operation time are determined by Monte Carlo simulation, and a descriptive model of the uncertain operation time of product parts is established; 1.2) Considering the operational characteristics of robots and dismantling personnel, construct a time model for switching dismantling tools, directions, and workstation positions; 1.2.1) Tool Switching: Define a set of disassembly tools and quantify the switching time between different tools using a tool switching matrix; 1.2.2) Direction switching: Define the set of decomposition directions, and describe the direction switching time through the direction switching matrix; 1.2.3) Workstation switching: Calculate the workstation transfer time for dismantling personnel based on the workstation index spacing; The time required for disassembly personnel to switch workstations when handling different colloids is calculated using the following formula:

[0007] Where LS(i, j) represents the time required for the disassembly personnel to switch positions from the current workstation of colloid i to the workstation of the target colloid j, and k ls Here, i represents the workstation switching time coefficient, i represents the index number of the target colloid, and j represents the index number of the currently operating colloid. 1.3) Construct a product parts disassembly constraint relationship model; Design a three-dimensional Boolean multi-directional disassembly constraint matrix M to quantify spatial constraints from two dimensions: part pairs and disassembly directions. The matrix element M(i,j,k) represents the constraint state of part i on part j in the k-th disassembly direction (1=constrained, 0=unconstrained). The disassemblyability of product parts is analyzed by disassembling the constraint matrix from multiple directions. 2) Using the human-machine collaborative decomposition information model under uncertain operation time constructed in step 1) as the environment for reinforcement learning algorithm, establish a dynamic programming model for human-machine collaborative decomposition task; 2.1) Define the elements of deep reinforcement learning, including: state, action, and reward function; With the optimization objective of human-machine collaborative dismantling time, the state space, action space, and reward function designed in this invention are used to solve the dynamic programming problem of human-machine collaborative dismantling task under uncertain operation time.

[0008] 2.1.1) State s t The description is as follows:

[0009] Where the array PS = [ps1, …, ps N [] represents the disassembly status of N parts, where ps represents the disassembly status. i ∈{0,1} represents the disassembly state of part i; where ps i =0 indicates that the part is in an undisassembled state; ps i =1 indicates that the part is in a disassembled state; array AdS=[as1,… as… adn ] represents the softening / disassembly state of adn structural adhesives; array AdT=[t s,1 ,…, t s,adn [ ] represents the reaction time t of adn structural adhesives. s ; where t s，i The reaction time t of structural adhesive i is indicated. s ; array AdP=[t s,1 / t h,1 ,…,t s,adn / t h,adn [] represents the softening progress of adn structural adhesives; array ES=[es1, es2] represents the busy status of the disassembly personnel and the robot, es1∈{0,1} represents the busy status of the disassembly personnel; es2∈{0, 1} represents the busy status of the robot; array ATS=[ap1,ap2] represents the disassembly task type of the disassembly personnel and the robot, ap1 represents the disassembly task type of the disassembly personnel; ap2 represents the disassembly task type of the robot; ETS=[O1(i), O2(j)] represents the unique thermal encoding of the index of the part / structural adhesive currently being disassembled by the disassembly personnel and the robot; ETL=[O1(t i ), O2(t j [)] represents the one-hot encoding of the index of the tools currently used by the disassembly personnel and the robot in Tools; EDR=[ O1(ds i ), O2(ds j [] represents the one-hot encoding of the index of the current disassembly direction of the disassembly personnel and robot in DS; array TLT=[tl1, tl2] represents the remaining time of the current disassembly task of the disassembly personnel and robot; tl1 represents the remaining time of the current disassembly task of the disassembly personnel; tl2 represents the remaining time of the current disassembly task of the robot. The disassembly task time is calculated by the following formula;

[0010] Among them, t e Indicates the time required for disassembly personnel / robots to disassemble parts / battery cells or apply chemical solvents / remove residual adhesives; TS ij This indicates that the disassembly personnel / robot have moved from the previous state s. t-1 The tool used is t i Switch to the current state s t Required tools t j Time; DM kl This indicates that the disassembly personnel / robot have moved from the previous state s. t-1 Switch the disassembly direction to the current state s t The time for disassembly direction; if the previous state s t-1 With the current state s t All actions are related to the colloid, and LS(m, n) represents the disassembly personnel's movement from the previous state s. t-1 The workstation handling colloid m is switched to the current state s. t The station position switching time required for processing colloid n; otherwise, LS(m,n)=0. 2.1.2) Action a t , This means that the intelligent agent assigns different dismantling tasks to the dismantling personnel, which is represented by the quadruple shown in the following formula;

[0011] Where etp∈{1, 2}, etp=1 indicates that the action is performed by the dismantling personnel; etp=2 indicates that the action is performed by the robot; ap etp For the disassembly task type of the disassembly personnel / robot; i is the part / structural adhesive index, if ap etp ∈{1, 2}, where i represents the index of the current chemical solvent application / removal of residual colloid; if ap etp ∈{3, 4}, where i represents the index of the currently disassembled part; if ap etp =0, then i is missing; j represents the disassembly direction ds j The index in DS, if ap etp If ∈{0,1,2}, then j is empty; 2.1.3) Reward function r t ; Reward r t Calculated by the following formula:

[0012] Where, d t This is the end marker;

[0013] Once all parts have been disassembled, Or, the number of steps exceeds the maximum value (maxstep), i.e., t > maxstep, d t If d is 1, otherwise d t =0; Where, r f To terminate the reward, calculate using the following formula;

[0014] If the number of disassembly steps exceeds the maximum value and there are still parts that have not been disassembled ( ), r f This is a penalty item, and its value is related to the completeness of the product disassembly ( Related to this, the lower the completion rate, the higher the penalty; if all parts are disassembled and the number of disassembly steps does not exceed the maximum value ( ), r f This is a reward item, and its value is related to the time t required to disassemble all parts. total Related, ω e The time reward constant is the time t required to disassemble all parts. total The less, the greater the reward; t total Based on the completion step time of the disassembly task / glue softening and all step times Perform a summation calculation; r s To address the problem of sparse rewards during training, rewards are assigned to the agent's task allocation at each step, thus resolving the issue of sparse rewards. s Values ​​and Disassembly Task Types (ap) etp The relevant conditions satisfy the following formula;

[0015] r w The purpose of this penalty is to punish agents for choosing to be assigned long-waiting tasks, r w The value of is related to the waiting time (wt) and satisfies the following formula;

[0016] 2.2) Establish a DQN network architecture; Using Dueling Double DQN, the Q value is decomposed into the value stream (state value) and the advantage stream (action advantage). The state-action Q-value is calculated based on the outputs of the value stream and the advantage stream, as shown in the following formula;

[0017] in, Indicates that in state s t Choose action a t Q value at that time; The output of the value stream represents a given state s. t The value of , that is, the expected return in that state; The output of the dominant flow represents the state s. t Choose action a t Advantages compared to other actions; is the mean normalization term of the dominance function; |A| is the size of the action space; 2.3) Model training; 3) Use the trained human-machine collaborative task decomposition dynamic programming model to generate a human-machine collaborative task decomposition planning scheme under uncertain operation time.

[0018] According to the above scheme, in step 1.2.1), the set of disassembly tools is Tools={t1, t2, …, t6}, where t1 to t6 correspond to pry bar, wire stripper, electric drill, screwdriver, clamp and adhesive removal tool, respectively; The switching time between different disassembly tools is quantitatively described by the switching matrix TS, as shown below: Among them, matrix element TS ij Indicates the use of the currently used tool t i Switch to target tool t j The required time (i, j∈{1,2,3,4,5,6}); when i=j (i.e., no tool switching is required, and the same tool is used continuously), the switching time TS ii =0.

[0019] According to the above scheme, in step 1.2.2), the set of disassembly directions is defined as DS=[+x, -x, +y, -y, +z], and the direction switching time is described by the disassembly direction switching matrix DM, as shown below:

[0020] Among them, matrix element DM ij Indicates the direction of disassembly from the current direction (ds) i Switch to target disassembly direction ds j Time required (ds) i ds j ∈DS), ds1 to ds5 respectively correspond to the [+x, ] in the decomposition direction set DS. x, +y, y, +z].

[0021] According to the above scheme, in step 2.2), the network architecture adopts a Dueling network architecture that integrates residual blocks and noisy linear layers; The output of the feature extraction layer is fed into the cascaded residual block sequence for higher-level feature representation. The residual block output features are input into the value stream and the advantage stream, respectively. Both branches introduce a noisy linear layer, which adds learnable random perturbations to the weights and biases. By integrating the outputs of value stream and advantage stream, the state-action Q-value is obtained, enabling the Dueling architecture to accurately decompose the Q-value; .

[0022] According to the above scheme, in step 2.2), the target Q value is calculated as follows: The action is selected through the policy network, and then the target network calculates the Q-value corresponding to that action. This model uses an n-step empirical approach, therefore the calculated target Q-value y... t The formula for calculating the cumulative discount return after n steps is as follows:

[0023] in, γ represents the cumulative discounted return over n consecutive steps starting from step t; n s is the discount factor for step n; t+n This represents the state at step t+n. The Q-value function representing the policy network estimate is used in state s. t+n Choose the optimal action; The target network is used to calculate the Q-value corresponding to the optimal action selected by the policy network.

[0024] According to the above scheme, in step 2.3), an action selection based on an ε-greedy strategy is adopted; The agent in state s t At that time, there is a probability of ε of randomly selecting a feasible action a. t The strategy network predicts the action 'a' with the highest Q-value with a probability of 1-ε. t As shown below:

[0025] in, The Q-value function estimated by the policy network. Represents state s t All feasible actions; To ensure that the agent can converge to the optimal policy after fully exploring the environment, the value of ε gradually decreases as training progresses. In the early stages of training, the value of ε is 1, and the agent tends to choose actions randomly. As training progresses, the value of ε gradually decreases to 0.01, and the agent will be more inclined to use the learned optimal policy.

[0026] According to the above scheme, in step 2.3), when calculating the loss, it is necessary to use importance sampling weight (ISW) to weight and correct the loss of each sample; .

[0027] According to the above scheme, in step 2.3), the loss function used for model training is as follows;

[0028] Where B is the batch size; ISW j The importance sampling weight for the j-th sample in the batch; y is the Q-value predicted by the policy network; t The target Q value; The smoothing L1 loss function is defined as follows: .

[0029] The beneficial effects of this invention are: 1. This invention quantifies the uncertainty of structural adhesive softening time by constructing an information model for human-machine collaborative dismantling with uncertain operation time, thus solving the problem of characterizing dismantling operation time.

[0030] 2. This invention proposes a dynamic programming model for human-machine collaborative disassembly tasks based on an improved DQN. Combining the advantages of the flexibility of manual disassembly during the chemical softening process of structural adhesives and the efficiency of robot disassembly of rigid parts, the state, action, and reward functions of a deep reinforcement learning algorithm are designed. Combined with the uncertain operation time, the optimal scheme for human-machine collaborative disassembly task planning is dynamically generated, which has good generalization performance. Attached Figure Description

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a diagram of the deep learning network structure according to an embodiment of the present invention; Figure 3 This is a Gantt chart of the human-machine collaborative disassembly planning scheme in an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0033] like Figure 1 As shown, a dynamic programming method for human-machine collaborative task decomposition under uncertain task time includes the following steps: (1) Constructing a human-machine collaborative disassembly information model under uncertain operation time. Considering the uncertainty of the softening time of the structural adhesive during chemical softening, a mathematical model of operation time under multiple factors is constructed; the boundary values ​​of the disassembly operation time of the parts are determined by Monte Carlo simulation, and a descriptive model of the uncertain operation time of the product parts is established; considering the operation characteristics of the robot and the disassembly personnel, a model of the switching time of disassembly tools, directions and workstation positions is constructed. Combined with the disassembly constraint matrix based on multiple directions, a model of the disassembly constraint relationship of the product parts is constructed.

[0034] In step (1), a human-machine collaborative disassembly information model under uncertain operation time is constructed. First, taking the softening time of structural adhesive as the object, a mathematical model of operation time under multiple factors is constructed. Combining the Monte Carlo simulation method, the boundary values ​​of the part disassembly operation time are determined, and a description model of the uncertain operation time of product parts is constructed; considering the operation characteristics of robots and disassembly personnel, a model of the switching time of disassembly tools, directions, and workstation positions is constructed. Combining the disassembly constraint matrix based on multiple directions, a model of the disassembly constraint relationship of product parts is constructed. The specific process is as follows: 1.1) Taking the softening time of structural adhesive as the object, its working time t h The softening rate v of the adhesive is influenced by the combined effects of multiple factors, mainly including temperature T, reaction surface area A, solvent volume fraction φ1 in the reaction layer, molar volumes V1 and V2 of solvent and adhesive, and activation energy of the reaction. With diffusion activation energy And the solvent diffusion coefficient D0.

[0035] Wherein, the reaction interface area A satisfies formula (1), L is the reaction interface length, and W is the reaction interface width.

[0036]

[0037] The softening rate v of the adhesive satisfies formula (2).

[0038]

[0039] Where α is the amount of reaction (m t The ratio of -m0 to the initial amount m0 ​​describes the degree of reaction; the reaction time t sThe time interval from the start of the reaction to the current time; the gel softening rate v can be expressed as the degree of reaction α versus the reaction time t. s Differential R is the molar gas constant. This invention assumes that when the softening reaction degree α reaches 90%, the structural adhesive softens to a removable state. The time from the start of the reaction to when the reaction degree α reaches 90% is defined as the adhesive softening time t. h The time is affected by the aforementioned multiple factors, and its value has a high degree of uncertainty, as shown in formula (3).

[0040]

[0041] Where C is an empirical constant.

[0042] To simplify model calculations, this invention treats temperature T, reaction surface area A, solvent volume fraction φ1 in the reaction layer, and the molar volumes V1 and V2 of solvent and adhesive as constants. Due to experimental measurement errors, [the following is omitted as the original text is incomplete and requires further context]. (mean value is 68 kJ / mol) - ¹) and (mean value is 11 kJ mol) - ¹) Set it as a normally distributed random variable, and quantify its effect on uncertain operation time t. h The impact of the above t. h Based on the uncertainty distribution characteristics, the boundary values ​​of the part disassembly operation time are determined by Monte Carlo simulation, and a descriptive model of the uncertain operation time of product parts is established. 1.2) Except for the uncertainty of the glue softening time, the operation time of disassembly tool switching, direction switching and workstation transfer in this invention is described by a deterministic time model. The specific operation time of disassembly tool switching and direction switching is shown in Tables 1 and 2.

[0043] The set of disassembly tools is Tools={t1, t2, …, t6}, where t1 to t6 correspond to pry bar, wire stripper, electric drill, screwdriver, clamp, and adhesive removal tool, respectively. The switching time of different disassembly tools is quantitatively described by the switching matrix TS, as shown in formula (4).

[0044] Among them, matrix element TS ij Indicates the use of the currently used tool t i Switch to target tool t j The required time (i, j∈{1,2,3,4,5,6}); when i=j (i.e., no tool switching is required, and the same tool is used continuously), the switching time TS ii =0.

[0045] The set of disassembly directions is defined as DS=[+x, -x, +y, -y, +z]. The direction switching time is described by the disassembly direction switching matrix DM, as shown in formula (5).

[0046]

[0047] Among them, matrix element DM ij Indicates the direction of disassembly from the current direction (ds) i Switch to target disassembly direction ds j Time required (ds) i ds j ∈DS), ds1 to ds5 respectively correspond to the [+x, ] in the decomposition direction set DS. x, +y, y, +z].

[0048] The time for dismantling personnel to switch positions at different workstations to handle different colloids is calculated by formula (6).

[0049]

[0050] Where LS(i, j) represents the time required for the disassembly personnel to switch positions from the current workstation of colloid i to the workstation of the target colloid j, and k ls The workstation switching time coefficient (its value can be calibrated through statistical analysis of actual disassembly operation data, reflecting the average switching time corresponding to the unit index spacing), i is the index number of the target colloid, and j is the index number of the currently operated colloid.

[0051] 1.3) Based on the disassembly direction set DS, a three-dimensional Boolean multi-directional disassembly constraint matrix M is constructed to quantify the disassembly constraint relationships between parts: the first dimension (i) and the second dimension (j) represent the two parts to be analyzed (i.e., "part i" and "part j"), respectively, and the third dimension (k) corresponds to the 5 directions in the disassembly direction set DS (k=1 to 5 represent disassembly directions +x, -x, +y, -y, and +z, respectively). The matrix elements are uniformly Boolean M(i, j, k) ∈ {0, 1}. When M(i,j, k)=1, it indicates that part i has a constraint on part j in the k-th disassembly direction (i.e., disassembly movement in that direction is restricted); when M(i, j, k)=0, it indicates that part i has no constraint on part j in the k-th disassembly direction (i.e., disassembly movement in that direction is allowed). Based on the product structure model, the spatial constraint relationships of different parts are analyzed to determine the constraint state of each part in different disassembly directions. If part i and part j in the 3D structure have spatial interference or connection restrictions in a certain direction in DS, then the matrix element M (i, j, k) in the corresponding direction is set to 1; if the two have no spatial overlap or connection in a certain direction, then the matrix element M (i, j, k) is set to 0.

[0052] The disassemblyability of product parts is analyzed by using a multi-directional disassembly constraint matrix. Specifically, for a part j to be disassembled, the set I = {i | part i is not disassembled} of all currently undisassembled parts is determined. All directions k (k = 1~5) in the disassembly direction set DS are traversed. For each direction k, the element M(i, j, k) corresponding to all undisassembled parts i ∈ I in the matrix is ​​queried. Then, the disassemblyability of the part is determined: if there exists a certain direction k... * Such that for all i∈I, M(i, j, k) is satisfied. * If )=0, then it is determined that part j is in the direction k. * If a component j is disassembled, it is considered non-disassembled. Conversely, if for all directions k, there exists at least one undisassembled component i∈I satisfying M(i, j, k)=1, then component j is determined to be non-disassembled. Finally, after component j is disassembled, it is removed from the set of undisassembled components I, and the constraint effect of the component on other components is no longer considered in subsequent disassembleability determinations, realizing dynamic updating of constraint relationships. For a certain battery cell in the product components, its disassembly is additionally restricted by adhesive bonding constraints: there are 3 adhesive bonding areas under a single battery cell, and the battery cell can only be disassembled after all 3 adhesive bonding areas are completely disassembled; based on the distribution characteristics of the adhesive bonding areas, the disassembly order of the adhesive bonding areas must follow the rule of "from the outside to the inside". For adhesive bonding areas 0~23, adhesive bonding area 0 must be disassembled before area 1 can be disassembled; adhesive bonding area 23 must be disassembled before area 22 can be disassembled.

[0053] (2) A dynamic programming model for human-machine collaborative disassembly tasks based on an improved DQN is proposed. Using the human-machine collaborative disassembly information model under uncertain operation times as the environment for the reinforcement learning algorithm, the state, action, and reward functions of the deep reinforcement learning algorithm are constructed by combining the disassembly state of parts, the state of glue softening disassembly, and the states of the disassembly personnel and robot. A deep learning network model is established by combining a noisy linear layer and residual blocks, and the Dueling Double DQN architecture, n-step experience and Priority Experience Replay (PER), and soft target update methods are integrated to construct a dynamic programming model for human-machine collaborative disassembly tasks. Based on this, various uncertain operation times are generated using Monte Carlo simulation, and the improved DQN deep reinforcement learning model is trained, verifying the effectiveness and generalization of the proposed method.

[0054] In step (2), the human-machine collaborative disassembly information model under uncertain operation time constructed in step (1) is used as the environment for the reinforcement learning algorithm. Combining the disassembly state of parts, the disassembly state of softened adhesive, and the states of the disassembly personnel and robot, the state space, action space, and reward function of the deep reinforcement learning algorithm are defined. A deep learning network architecture is built by combining a noisy linear layer and residual blocks. The Dueling Double DQN architecture, Priority Experience Replay (PER), and soft target update method are incorporated to construct a dynamic programming model for the human-machine collaborative disassembly task. Finally, various uncertain operation times are randomly generated using Monte Carlo simulation to train the improved DQN deep reinforcement learning model, verifying the effectiveness and generalization of the proposed method. The specific process is as follows: In the proposed human-computer collaborative task decomposition dynamic programming method based on deep reinforcement learning algorithm, the agent determines the task based on the current state s. t In performing action a t Then, the environment of the deep reinforcement learning algorithm is determined based on action a. t The execution result generates a new state s t+1 and provide feedback rewards r to the agent. t The state s of the deep reinforcement learning algorithm proposed in this invention t Described by formula (7):

[0055] Where the array PS = [ps1, …, ps N [] represents the disassembly status of N parts. The number of parts in the product is N=24. Binary discrete encoding is used, and the disassembly status is ps. i ∈{0,1} represents the disassembly state of part i. Where, ps i =0 indicates that the part is in an undisassembled state; ps i =1 indicates that the part is in a disassembled state. Array AdS=[as1,… as adn[Indicates the softening / disassembly state of adn structural adhesives, where adn = 24 structural adhesives in the product.] i ∈{0, 1, 2, 3} represents the four states of structural adhesive i. Where, as i =0 indicates that structural adhesive i is in an unsoftened state; as i =1 indicates that structural adhesive i is in a softening state; as i =2 indicates that structural adhesive i is in a fully softened (disassembled) state; as i =3 indicates that structural adhesive i is in a disassembled state. Array AdT=[t s,1 ,…, t s,adn [ ] represents the reaction time t of adn structural adhesives. s ; where t s，i The reaction time t of structural adhesive i is indicated. s Array AdP = [t s,1 / t h,1 ,…, t s,adn / t h,adn [] represents the softening progress of adn structural adhesives, where the softening progress of structural adhesive i is determined by its reaction time t. s Rather than softening time t h The ratio t s,i / t h,i Quantization is performed; where t h,i The softening time t of structural adhesive i hThe array ES=[es1,es2] represents the busy status of the disassembly personnel and the robot. es1∈{0, 1} represents the busy status of the disassembly personnel, es1=0 indicates the disassembly personnel are idle, and es1=1 indicates the disassembly personnel are busy. es2∈{0, 1} represents the busy status of the robot, es2=0 indicates the robot is idle, and es2=1 indicates the robot is busy. The array ATS=[ap1,ap2] represents the disassembly task type of the disassembly personnel and the robot. ap1∈{0, 1, 2, 3} represents the disassembly task type of the disassembly personnel. ap1=0 indicates the disassembly personnel are waiting; ap1=1 indicates the disassembly personnel are applying chemical solvents; ap1=2 indicates the disassembly personnel are removing residual adhesive; and ap1=3 indicates the disassembly personnel are disassembling parts. ap2∈{0, 3, 4} represents the robot's disassembly task type, where ap2=0 indicates the robot is waiting; ap2=3 indicates the robot is disassembling parts; and ap2=4 indicates the robot is disassembling battery cells. ETS=[O1(i), O2(j)] represents the unique thermal encoding of the disassembly personnel and the robot's currently disassembled part / structural adhesive index. If ap1=3, O1(i) is the unique thermal encoding of the disassembly personnel's currently disassembled part i; if ap1∈{1, 2}, O1(i) is the unique thermal encoding of the disassembly personnel's currently applied chemical solvent / removed residual adhesive i; if ap1=0, O1(i) is encoded as all 0s. If ap2∈{3, 4}, O2(j) is the unique thermal encoding of the robot's currently disassembled part j; if ap2=0, O2(j) is encoded as all 0s. ETL=[O1(t i ), O2(t j [)] represents the one-hot encoding of the index of the tool currently used by the disassembly personnel and the robot in Tools; where O1(t) represents the index of the tool currently used by the disassembly personnel and the robot in Tools. i ) indicates the disassembly tools currently being used by the disassembly personnel. i One-hot encoding of the index in Tools; O2(t) j ) indicates the disassembly tool t currently used by the robot. j One-hot encoding of the index in Tools, t j ∈Tools. EDR=[ O1(ds i ), O2(ds j O1(ds) represents the one-hot encoding of the index of the current disassembly direction of the disassembly personnel and robot in DS; where O1(ds) represents the index of the current disassembly direction of the disassembly personnel and robot in DS. i ) indicates the current disassembly direction of the disassembly personnel. i One-hot encoding of the index in DS; O2(ds) j ) indicates the robot's current disassembly direction ds jThe one-hot encoding of the index in DS. The array TLT=[tl1, tl2] represents the remaining time for the disassembly task of the disassembly personnel and the robot. tl1 represents the remaining time for the disassembly task of the disassembly personnel; tl2 represents the remaining time for the disassembly task of the robot. The disassembly task time is calculated by formula (8).

[0056]

[0057] Among them, t e Indicates the time required for disassembly personnel / robots to disassemble parts / battery cells or apply chemical solvents / remove residual adhesives; TS ij This indicates that the disassembly personnel / robot have moved from the previous state s. t-1 The tool used is t i Switch to the current state s t Required tools t j Time; DM kl This indicates that the disassembly personnel / robot have moved from the previous state s. t-1 Switch the disassembly direction to the current state s t The time for disassembly direction; if the previous state s t-1 With the current state s t All actions are related to the colloid, and LS(m, n) represents the disassembly personnel's movement from the previous state s. t-1 The workstation handling colloid m is switched to the current state s. t The station position switching time required for processing colloid n, otherwise LS(m,n)=0.

[0058] Action a of deep reinforcement learning algorithm t The meaning is that the intelligent agent assigns different dismantling tasks to the dismantling personnel, which is represented by a quadruple as shown in formula (9).

[0059]

[0060] Where etp∈{1, 2}, if etp=1, the action is performed by the disassembly personnel; if etp=2, the action is performed by the robot. etp This represents the disassembly task type for the disassembly personnel / robot. 'i' is the part / structural adhesive index; if ap etp ∈{1, 2}, where i represents the index of the current chemical solvent application / removal of residual colloid; if ap etp ∈{3, 4}, where i represents the index of the currently disassembled part; if ap etp =0, then i is empty. j represents the disassembly direction ds j The index in DS, if ap etp If ∈{0,1,2}, then j is empty.

[0061] The reward r of deep reinforcement learning algorithms t Calculated by formula (10).

[0062]

[0063] Where, d t This is the termination flag, and its value is shown below.

[0064]

[0065] Once all parts have been disassembled, Or, the number of steps exceeds the maximum value (maxstep), i.e., t > maxstep, d t If d is 1, otherwise d t =0; Where, r f To terminate the reward, calculate using formula (11).

[0066]

[0067] If the number of disassembly steps exceeds the maximum value and there are still parts that have not been disassembled ( ), r f This is a penalty item, and its value is related to the completeness of the product disassembly ( Related to this, the lower the completion rate, the higher the penalty; if all parts are disassembled and the number of disassembly steps does not exceed the maximum value ( ), r f This is a reward item, and its value is related to the time t required to disassemble all parts. total Related, ω e The time reward constant is the time t required to disassemble all parts. total The less, the greater the reward.

[0068] t total Based on the completion step time of the disassembly task / glue softening and all step times The summation calculation and step-time description process are as follows: First, both the dismantling personnel and the robot are idle. The agent assigns dismantling tasks to them respectively. At this time, the dismantling personnel and the robot change from an idle state to a busy state, and the remaining time (tl1 and tl2) of their respective dismantling tasks is the time of their respective dismantling tasks (t). task，1 With t task，2 Because both the dismantling personnel and the robots were busy, the step time was... The remaining time (tl1 and tl2) for dismantling personnel and robots, and the time required for all softening structural adhesives to complete softening. The minimum value of ) is calculated using formula (12).

[0069]

[0070] After that, update the status s t For s t+1 ; for step The dismantling personnel completed their assigned dismantling task ① (which involved applying chemical solvents to soften the structural adhesive) and became idle; tl1 was updated to 0. The robot remained busy; tl2 was updated to t. task，2 - The structural adhesive begins to react simultaneously. Then, the step time is considered. The dismantling personnel perform dismantling task ③. After the step time, the remaining time tl1 for dismantling task ③ is updated to t. task，3 - The robot completes disassembly task ② and enters an idle state; tl2 is updated to 0. Afterwards, regarding the step time... Since the robot has no disassembly task to perform at the moment, it chooses to wait (disassembly task ④). Its waiting time (wt) is the same as the step time. The formula for calculating the waiting time is as follows.

[0071]

[0072] The process continues until the disassembly personnel complete their current disassembly task or the adhesive softens, at which point new tasks are assigned. Then, the step time is considered. The dismantling personnel and the robot respectively perform dismantling tasks ⑤ and ⑥. After the step time, the remaining time tl1 for the dismantling personnel's dismantling task ⑤ is t. task,5 - The remaining time for the robot disassembly task ⑥ is tl2. task，6 - After that, regarding the stepping... The structural adhesive has softened, but the disassembly personnel and robot remain busy. The environment will recursively advance the time according to the above method until one of the personnel or robot becomes idle. Finally, the time will be continuously advanced according to the above process until the last part is disassembled. In the example, t... total This is the sum of all step times.

[0073] The time required to complete the disassembly of all parts t total Calculated using formula (14).

[0074]

[0075] r t r in s To address the issue of sparse rewards during training, rewards are assigned to the agent's task allocation at each step. sValues ​​and task types (ap) etp ) is related and satisfies formula (15).

[0076]

[0077] It is worth noting that the total reward for each step after all parts have been disassembled is a fixed value.

[0078] r w The purpose of this penalty is to punish agents for choosing to be assigned long-waiting tasks, r w The value of is related to the waiting time (wt), which satisfies formula (16).

[0079]

[0080] When an agent interacts with the environment of a deep reinforcement learning algorithm to collect training data, the model uses a policy network to guide actions. t The choice of action. To select a new action or utilize the existing optimal action in the balancing model, this model adopts an ε-greedy strategy.

[0081] The ε-greedy policy relies on the policy network's estimation of the state-action Q-value. The agent in state s... t At that time, there is a probability of ε of randomly selecting a feasible action a. t The strategy network predicts the action 'a' with the highest Q-value with a probability of 1-ε. t As shown in formula (17).

[0082]

[0083] in, The Q-value function estimated by the policy network. Represents state s t Take all feasible actions.

[0084] To ensure that the agent converges to the optimal policy after fully exploring the environment, the value of ε gradually decreases as training progresses. In the early stages of training, ε is 1, and the agent tends to choose actions randomly; as training progresses, the value of ε gradually decreases to 0.01, and the agent will be more inclined to utilize the learned optimal policy.

[0085] In deep reinforcement learning algorithms, in order to use the experience collected during the interaction between the agent and the environment to train the deep learning model, this invention integrates n-step experience and the PER method to complete experience sampling.

[0086] First, the agent will generate raw experience (s) from its interaction with the disassembly environment. t , a t , rt , s t+1 , d t The experience is cached in the n-step experience queue. When the experience stored in the n-step experience pool reaches the preset n-step length or a termination state is encountered, the consecutive experiences in the n-step experience queue are integrated to obtain a new n-step experience. .in, The cumulative discount return for n consecutive steps starting from step t is calculated using formula (18).

[0087]

[0088] Where γ∈[0,1] is the discount factor; finally, the n-step experience... Stored in the PER experience pool.

[0089] During the sampling phase, the model combines the temporal difference error (TD error) of the n-step experience with its temporal position in the PER experience pool to determine the probability of the n-step experience being extracted. The PER experience pool has a length of L, and the n-step experience samples are stored in a unidirectional queue. The probability of the n-step experience sample at index i being sampled is... i The calculation formula is shown in (19).

[0090]

[0091] Where L is the size of the PER experience pool; dec∈(0, 1) is the temporal decay factor, used to increase the weight of new experiences; μ∈[0, 1] is the mixing coefficient with uniform sampling, used to balance uniform sampling and priority-driven sampling; p i p is the priority of the n-step experience at index i. i The calculation formula is shown in (20).

[0092]

[0093] Where, δ i For the n-step empirical TD error at index i, It is a very small constant.

[0094] In the Priority Experience Replay (PER) strategy, Importance Sampling Weights (ISWs) are used to adjust the relative importance of sampled experiences, ensuring that the model does not suffer from poor learning performance due to sampling bias during training. Therefore, for each batch sampled from the experience pool, the Importance Sampling Weight (ISW) for each experience within that batch needs to be calculated. The normalized Importance Sampling Weight (ISW) for each experience in the batch is... i The calculation formula is shown in (21).

[0095]

[0096] Where L is the size of the PER experience pool; probs i β is the sampling probability of empirical i in the sampling batch; β is the bias compensation coefficient; The maximum importance sampling weight (ISW) for all experiences in the batch is used to normalize the importance sampling weights.

[0097] This invention employs a deep learning network structure based on the Dueling DQN algorithm. By separating the value stream and the advantage stream, the state-action Q-value is calculated, thus decomposing the Q-value calculation into two independent parts: the value stream and the advantage stream. The deep learning network structure is as follows: Figure 2 As shown.

[0098] The feature extraction layer is the first part of a deep reinforcement learning network. It is responsible for initially processing the input n-step experience and extracting useful features. The feature extraction layer includes a fully connected layer (Linear), layer normalization (LayerNorm), and the ReLU activation function.

[0099] The output of the feature extraction layer will be used as the input to the residual block sequence to further perform higher-level feature representation on the input data and extract more abstract patterns. The residual block sequence is composed of multiple residual blocks concatenated. Each residual block consists of a fully connected layer (Linear), a layer normalization function (LayerNorm), a ReLU activation function, and a skip connection. Among them, the skip connection allows information to flow directly through the network by directly adding the input and output, avoiding the gradient vanishing problem common in deep networks. The forward propagation formula of the residual block is shown in Equation (22).

[0100]

[0101] Where H(x) is the output of the residual block, x is the input of the residual block, and F(x) is the output of the input x after processing by a fully connected layer, normalization, and ReLU activation function.

[0102] The output of the residual block sequence will be input into the value stream and the dominance stream, respectively. The value stream and the dominance stream have similar network structures, both consisting of a noisy linear layer and a ReLU activation function. The noisy linear layer adds learnable noise to the fully connected layer, and independent random perturbations are added to the weights ω and biases b of the linear layer, as shown in Equation (23).

[0103]

[0104] Where, μ ω σ ω Let μ be the mean and standard deviation of the weights ω of the linear layer with noise, respectively. b σ b Let ε be the mean and standard deviation of the bias b of the linear noise layer, respectively. ω ε b For each noise vector, ⊙ represents element-wise multiplication.

[0105] The state-action Q value is calculated using the outputs of the value stream and the advantage stream, and the calculation formula is shown in (24).

[0106]

[0107] in, Indicates that in state s t Choose action a t Q value at that time; The output of the value stream represents a given state s. t The value of , that is, the expected return in that state; The output of the dominant flow represents the state s. t Choose action a t Advantages compared to other actions; is the mean normalization term of the dominance function; |A| is the size of the action space.

[0108] In the DQN deep reinforcement learning algorithm, the loss function measures the difference between the Q-value predicted by the policy network and the target Q-value. Backpropagation is used to adjust the model's weights and biases, gradually making the model's Q-value prediction more accurate. Combining the idea of ​​the Double DQN algorithm, the calculation of the target Q-value is divided into two stages: first, the policy network selects an action, and then the target network calculates the Q-value corresponding to that action. Because this model uses n-step empirical methods, the calculated target Q-value y... t The cumulative discount return after n steps is calculated using the formula shown in equation (25).

[0109]

[0110] in, γ represents the cumulative discounted return over n consecutive steps starting from step t; n s is the discount factor for step n; t+n This represents the state at step t+n. The Q-value function representing the policy network estimate is used in state s. t+n Choose the optimal action; The target network is used to calculate the Q-value corresponding to the optimal action selected by the policy network.

[0111] As mentioned earlier when introducing the Priority Experience Replay (PER) strategy, the priority p of n-step experience samples... i Based on the time difference error (TD error) δ i Calculate the TD error δ. i δ reflects the deviation between the Q-value predicted by the policy network and the target Q-value in the empirical samples. i The calculation formula is shown in (26).

[0112]

[0113] Among them, y t The target Q value; This represents the Q-value predicted by the policy network. During model training, a new TD error δ is obtained each time. i This will be fed back into the corresponding n-step experience sample to update its priority p. i .

[0114] Since this model employs a priority experience replay strategy, the importance of different samples varies. Therefore, when calculating the loss, importance sampling weights (ISW) are needed to weight and correct the loss for each sample. The calculation of the loss function consists of two steps: First, the smoothed L1 loss function (Huber loss function) is used to measure the Q-value predicted by the current policy network relative to the target Q-value y. t The difference between them is then combined with ISW to perform a weighted average of the sample loss, and the final loss function loss is calculated as shown in (27).

[0115]

[0116] Where B is the batch size; ISW j The importance sampling weight for the j-th sample in the batch; y is the Q-value predicted by the policy network; t The target Q value; Let L1 be the smoothing loss function, defined as follows.

[0117]

[0118] The weighted loss of all samples within a batch is averaged to obtain the final loss, which is then used for backpropagation to update the parameters of the policy network. This weighting method ensures that important samples with large TD errors receive higher weights during training, thereby accelerating model convergence and improving learning efficiency.

[0119] Correspondingly, the target network does not participate in backpropagation, but instead stabilizes the target value by performing an exponential smoothing average (a soft update method) on the policy network. The update formula for the target network is shown in formula (29).

[0120]

[0121] Where ← represents the assignment update (overwriting the old θ with the linear combination to the right of ←). target ); θ policy With θ target These are the complete learnable parameter vectors of the policy network and the target network (including weights / biases of each layer); τ∈(0,1] are the soft update coefficients.

[0122] During the training phase, based on uncertain operation time data for product parts, the generalization ability of the intelligent agent to dynamically plan human-machine collaborative disassembly tasks under uncertain operation times is trained. The activation energy of the adhesive softening reaction is then determined. Let the mean be μ er =68 (kJ mol) - ¹) The standard deviation is σ er Normally distributed random variable; diffusion activation energy Let the mean be μ df =11 (kJ mol) - ¹) The standard deviation is σ df The normally distributed random variable. Combined with the uncertain operation time t shown in equation (3). h The calculation formula can be adjusted. and Standard deviation σ er With σ df Controlling the uncertain operation time t h The randomization intensity is adjusted. Simultaneously, multi-stage randomization training is employed, dividing the total number of iterations of agent training into multiple stages, each with a different randomization intensity. This is achieved by progressively increasing the standard deviation σ. er With σ df The value of t makes the uncertain operation time t h The fluctuation range gradually increases. With each iteration, Monte Carlo simulation is used to generate the corresponding uncertain operation time t for each structural adhesive (totaling adn) in the product. h A diverse training environment was constructed. During training, the model's reward curve first rose, then oscillated, and finally converged. Initially, under the dominance of exploration, the curve fluctuated significantly. In the middle stage, as the policy adapted to the randomized environment, the reward continuously increased and the variance gradually converged. The standard deviation σ... er With σ dfThe increased switching point resulted in slight oscillations in the curve, but it remained stable, indicating that the model maintains good performance even under greater uncertainty. In the offline evaluation, the t-values ​​of all structural adhesives (24 in total) were calculated. h Values ​​are independently and randomly selected within the interval (2.3, 8.5), and the trained model is used to generate a human-machine collaborative task decomposition planning scheme under uncertain task time. The Gantt curve of the human-machine collaborative decomposition planning scheme is then used. Figure 3 As shown.

[0123] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A dynamic programming method for human-machine collaborative dismantling tasks under uncertain operation time, characterized in that, Includes the following steps: 1) Construct a human-machine collaborative decomposition information model under uncertain operation time; 1.1) Construct a descriptive model for the uncertain operation time of product parts; 1.1.1) Taking the softening time of the structural adhesive as the object, it is set that when the softening degree α of the structural adhesive reaches p, the structural adhesive softens to a disintegratable state; combined with temperature T, reaction surface area A, volume fraction of solvent in the reaction layer φ1, molar volumes of solvent and adhesive V1 and V2, and activation energy of the reaction. With diffusion activation energy And the solvent diffusion coefficient D0, to calculate the uncertain operation time of the product parts; 1.1.2) The boundary values ​​of the part disassembly operation time are determined by Monte Carlo simulation, and a descriptive model of the uncertain operation time of product parts is established; 1.2) Considering the operational characteristics of robots and dismantling personnel, construct a time model for switching dismantling tools, directions, and workstation positions; 1.3) Construct a product parts disassembly constraint relationship model; Design a three-dimensional Boolean multi-directional disassembly constraint matrix M to quantify spatial constraints from two dimensions: part pairs and disassembly directions. The matrix element M(i,j,k) represents the constraint state of part i on part j in the k-th disassembly direction: 1 indicates that there is a constraint, and 0 indicates that there is no constraint. The disassemblyability of product parts is analyzed by disassembling the constraint matrix from multiple directions. 2) Using the human-machine collaborative decomposition information model under uncertain operation time constructed in step 1) as the environment for reinforcement learning algorithm, establish a dynamic programming model for human-machine collaborative decomposition task; 2.1) Define the elements of deep reinforcement learning, including: state, action, and reward function; With the optimization objective of human-machine collaborative disassembly time, a state space, action space, and reward function are designed to solve the dynamic programming problem of human-machine collaborative disassembly task under uncertain operation time. 2.2) Establish a DQN network architecture; Using Dueling Double DQN, the Q-value is decomposed into the value stream and the advantage stream; Calculate the state-action Q-value using the outputs of the value stream and the advantage stream; 2.3) Model training; 3) Use the trained human-machine collaborative task decomposition dynamic programming model to generate a human-machine collaborative task decomposition planning scheme under uncertain operation time.

2. The dynamic planning method for human-machine collaborative dismantling tasks under uncertain operation time as described in claim 1, characterized in that, In step 1.1), the uncertain operation time of the product parts is calculated as follows; 。 3. The dynamic planning method for human-machine collaborative dismantling tasks under uncertain operation time as described in claim 1, characterized in that, In step 1.2), the time model for switching disassembly tools, directions, and workstation positions is constructed, including: 1.2.1) Tool Switching: Define a set of disassembly tools and quantify the switching time between different tools using a tool switching matrix; 1.2.2) Direction switching: Define the set of decomposition directions, and describe the direction switching time through the direction switching matrix; 1.2.3) Workstation switching: Calculate the workstation transfer time for dismantling personnel based on the workstation index spacing; The time required for disassembly personnel to switch workstations at different workstations when handling different colloids is calculated using the following formula; Where LS(i, j) represents the time required for the disassembly personnel to switch positions from the current workstation of colloid i to the workstation of the target colloid j, and k ls is the station switching time coefficient, i is the index number of the target colloid, and j is the index number of the currently operating colloid.

4. The dynamic planning method for human-machine collaborative dismantling tasks under uncertain operation time as described in claim 1, characterized in that, In step 2.1), the specific details are as follows: 2.1.1) State s t The description is as follows: Where the array PS = [ps1, …, ps N [] represents the disassembly status of N parts, where ps represents the disassembly status. i ∈{0,1} represents the disassembly state of part i; where ps i =0 indicates that the part is in an undisassembled state; ps i =1 indicates that the part is in a disassembled state; array AdS=[as1,… as… adn ] represents the softening / disassembly state of adn structural adhesives; array AdT=[t s,1 ,…, t s,adn [ ] represents the reaction time t of adn structural adhesives. s ; array AdP=[t s,1 / t h,1 ,…, t s,adn / t h,adn [] represents the softening progress of adn structural adhesives; array ES=[es1, es2] represents the busy status of the disassembly personnel and the robot, es1∈{0,1} represents the busy status of the disassembly personnel; es2∈{0, 1} represents the busy status of the robot; array ATS=[ap1,ap2] represents the disassembly task type of the disassembly personnel and the robot, ap1 represents the disassembly task type of the disassembly personnel; ap2 represents the disassembly task type of the robot; ETS=[O1(i), O2(j)] represents the unique thermal encoding of the index of the part / structural adhesive currently being disassembled by the disassembly personnel and the robot; ETL=[O1(t i ), O2(t j [)] represents the one-hot encoding of the index of the tools currently used by the disassembly personnel and the robot in Tools; EDR=[ O1 (ds i ), O2 (ds j ] represents the one-hot encoding of the index of the current disassembly direction of the disassembly personnel and robot in DS; array TLT=[tl1, tl2] represents the remaining time of the current disassembly task of the disassembly personnel and robot; tl1 represents the remaining time of the current disassembly task of the disassembly personnel; tl2 represents the remaining time of the current disassembly task of the robot; the disassembly task time is calculated by the following formula; Among them, t e Indicates the time required for disassembly personnel / robots to disassemble parts / battery cells or apply chemical solvents / remove residual adhesives; TS ij This indicates that the disassembly personnel / robot have moved from the previous state s. t-1 The tool used is t i Switch to the current state s t Required tools t j Time; DM kl This indicates that the disassembly personnel / robot have moved from the previous state s. t-1 Switch the disassembly direction to the current state s t The time for disassembly direction; if the previous state s t-1 With the current state s t All actions are related to the colloid, and LS(m, n) represents the disassembly personnel's movement from the previous state s. t-1 The workstation handling colloid m is switched to the current state s. t The station position switching time required for processing colloid n; otherwise, LS(m,n)=0. 2.1.2) Action a t , This means that the intelligent agent assigns different dismantling tasks to the dismantling personnel, which is represented by the quadruple shown in the following formula; Where etp∈{1, 2}, etp=1 indicates that the action is performed by the dismantling personnel; etp=2 indicates that the action is performed by the robot; ap etp For disassembly personnel / robots, i represents the disassembly task type; i is the part / structural adhesive index, if ap etp ∈{1, 2}, where i represents the index of the current chemical solvent application / removal of residual colloid; if ap etp ∈{3, 4}, where i represents the index of the currently disassembled part; if ap etp =0, then i is missing; j represents the disassembly direction ds j The index in DS, if ap etp If ∈{0,1,2}, then j is empty; 2.1.3) Reward function r t ; Reward r t Calculated using the following formula; Where, d t This is a termination marker; Once all parts have been disassembled, that is Or, the number of steps exceeds the maximum value maxstep, i.e., t>maxstep, d t If d is 1, otherwise d t =0; Where, r f To terminate the reward, it is calculated using the following formula: If the number of disassembly steps exceeds the maximum value and there are still parts that have not been disassembled, r f This is a penalty item, and its value is related to the completeness of the product disassembly ( Related to this, the lower the completion rate, the higher the penalty; if all parts are disassembled and the number of disassembly steps does not exceed the maximum value, r f This is a reward item, and its value is related to the time t required to disassemble all parts. total Related, ω e The time reward constant is the time t required to disassemble all parts. total The less, the greater the reward; t total Based on the completion step time of the disassembly task / glue softening and all step times Perform a summation calculation; r s To address the problem of sparse rewards during training, rewards are assigned to the agent's task allocation at each step, thus resolving the issue of sparse rewards. s Values ​​and Disassembly Task Types (ap) etp The relevant conditions satisfy the following formula; r w The purpose of this penalty is to punish agents for choosing to be assigned long-waiting tasks, r w The value of is related to the waiting time wt, and satisfies the following formula; 。 5. The dynamic programming method for human-machine collaborative dismantling tasks under uncertain operation time as described in claim 1, characterized in that, In step 2.1), the formula for calculating the Q value is as follows: in, Indicates that in state s t Choose action a t Q value at that time; The output of the value stream represents a given state s. t The value of , that is, the expected return in that state; The output of the dominant flow represents the state s. t Choose action a t Advantages compared to other actions; is the mean normalization term of the dominance function; |A| is the size of the action space.

6. The dynamic programming method for human-machine collaborative dismantling tasks under uncertain operation time as described in claim 1, characterized in that, In step 2.3), the loss function used for model training; Where B is the batch size; ISW j The importance sampling weight for the j-th sample in the batch; y is the Q-value predicted by the policy network; t The target Q value; The smoothing L1 loss function is defined as follows: 。 7. The dynamic programming method for human-machine collaborative dismantling tasks under uncertain operation time as described in claim 1, characterized in that, When calculating the loss, importance sampling weights (ISW) are used to weight and correct the loss for each sample. 。 8. The dynamic programming method for human-machine collaborative dismantling tasks under uncertain operation time as described in claim 1, characterized in that, In step 2.3), an action selection based on an ε-greedy strategy is adopted; The agent in state s t At that time, there is a probability of ε of randomly selecting a feasible action a. t The strategy network predicts the action 'a' with the highest Q-value with a probability of 1-ε. t ; in, The Q-value function estimated by the policy network. Represents state s t Take all feasible actions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 8.

Images