Task real-time allocation method and device for man-machine cooperation disassembly

By identifying the images of the products to be disassembled and optimizing the human-machine task allocation using genetic algorithms, the problem of unreasonable task allocation of robotic arms and operators in complex disassembly scenarios is solved, the disassembly efficiency and safety are improved, and efficient product disassembly is achieved and recycling.

CN118941930APending Publication Date: 2024-11-12ZHEJIANG UNIV
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202410950019.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

During the disassembly of complex products, robotic arms are difficult to deal with changing scenarios. Human operators are limited by fatigue and safety risks. The existing human-machine collaboration model fails to reasonably formulate task allocation strategies and fails to fully consider the comprehensive impact of task allocation on sequence planning. .

Method used

By obtaining the image of the product to be disassembled, using a two-level neural network classifier to identify the quality status of the fastener, and combining genetic algorithms to generate a human-machine task allocation scheme, optimize the disassembly task allocation, and utilize the accuracy of the robotic arm and the operator's intellectual resources. , reduces operator fatigue, improves disassembly flexibility and safety.

Benefits of technology

It realizes efficient disassembly of complex products, improves the quality, safety and efficiency of disassembly operations, enhances the detachability and recyclability of products, reasonably allocates human-machine tasks, and alleviates the fatigue of operators.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941930A_ABST
    Figure CN118941930A_ABST
Patent Text Reader

Abstract

The invention discloses a task real-time allocation method and device for man-machine cooperation disassembly. According to the method, the image of the to-be-disassembled product is acquired, then the two-stage neural network classifier is constructed to detect the quality state of the to-be-disassembled fastener, and finally, the disassembly man-machine task allocation scheme is solved by using the genetic algorithm, so that detection and state classification of the to-be-disassembled fastener of the product are realized; real-time man-machine cooperation task allocation gives full play to the accuracy and endurance advantages of the mechanical arm, meanwhile, intelligence resources of operators are utilized, fatigue of the operators is greatly relieved, the ability of people to flexibly handle complex conditions and the characteristic of accurate and repeated operation execution of the mechanical arm are given full play, disassembly flexibility is enhanced, uncertain faults of products are solved, and the working efficiency is improved. The detachability and recoverability of products of different degrees are improved, and the quality, safety and efficiency of detaching operation are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for allocating human-machine collaborative disassembly tasks in a human-cyber-physical system of a new generation of intelligent manufacturing, and specifically to a method and device for real-time allocation of human-machine collaborative disassembly tasks for products. Background Art

[0002] Human-cyber-physical systems are the cornerstone of human-centered manufacturing. Remanufacturing has received widespread attention in promoting sustainability. Disassembly is a crucial but challenging task in remanufacturing. However, the process faces many complex problems: robotic arms have difficulty coping with complex and changing disassembly scenarios, and human operators are limited by fatigue and safety hazards in disassembly operations. Although human-machine collaboration technology has shown initial results in other industrial fields, there are still significant difficulties in dealing with the disassembly of complex products, such as how to formulate reasonable human-machine task allocation strategies in the face of highly uncertain situations. In view of the uncertainty of products, the limitations of pure robotic arm disassembly, and the fatigue and safety risks of manual operation, the existing human-machine collaboration model presupposes unreasonable human-machine task division and fails to fully consider the comprehensive impact of task allocation on sequence planning. Summary of the invention

[0003] In order to solve the problems existing in the background technology, the present invention provides a method and device for real-time task allocation for human-machine collaborative disassembly. The present invention obtains the image of the product to be disassembled, then constructs a two-level neural network classifier to identify the quality status of the fasteners to be disassembled, and finally uses the genetic algorithm to solve the disassembly human-machine task allocation scheme, thereby realizing the detection and status classification of the fasteners to be disassembled of the product. The real-time human-machine collaborative task allocation fully utilizes the precision and endurance advantages of the robot arm, while utilizing the intellectual resources of the operator, greatly alleviating the fatigue of the operator, and fully utilizing the ability of people to flexibly handle complex situations and the characteristics of the robot arm to accurately and repeatedly perform operations, thereby enhancing the flexibility of disassembly, solving product uncertainty failures, improving the disassembly and recyclability of products of different degrees, and improving the quality, safety and efficiency of disassembly operations.

[0004] The technical solution of the present invention is as follows:

[0005] 1. A real-time task allocation method for human-machine collaborative disassembly

[0006] 1) Capturing images of the product to be disassembled to obtain images of the product to be disassembled;

[0007] 2) Detecting the quality status of the fasteners to be disassembled in the image of the product to be disassembled;

[0008] 3) Calculate the disassembly complexity and human disassembly fatigue of the fastener to be disassembled according to the quality status of the fastener to be disassembled;

[0009] 4) According to the disassembly complexity of each fastener to be disassembled and the disassembly fatigue of the human body, a human-machine task allocation plan for the product to be disassembled is calculated and generated.

[0010] In the above 2), a two-stage neural network classifier is used to detect the quality status of the fasteners to be disassembled in the image of the product to be disassembled, and the two-stage neural network classifier includes a fastener status classification neural network and a fastener quality classifier neural network connected in sequence.

[0011] The fastener state classification neural network divides the state of the fasteners to be removed into a normal state, a stripped state and a rusted state; the fastener quality classifier neural network outputs the quality state of the fasteners to be removed, and the quality state of the fasteners to be removed includes normal, moderately stripped, moderately rusted, severely stripped and severely rusted.

[0012] In the above 3), the disassembly complexity of the fastener to be disassembled is calculated according to the quality status of the fastener to be disassembled, specifically:

[0013] According to the quality status of the fasteners to be disassembled, combined with the disassembly weight, disassembly tools and disassembly operation force, the disassembly complexity of the fasteners to be disassembled is fuzzy processed to obtain the disassembly complexity of the fasteners to be disassembled. The disassembly complexity includes the disassembly complexity of the operator and the disassembly complexity of the robotic arm.

[0014] In the above 3), the human disassembly fatigue degree of the fastener to be disassembled is calculated according to the quality state of the fastener to be disassembled, specifically:

[0015] According to the quality status of the fasteners to be disassembled, combined with the disassembly work intensity, disassembly work time and disassembly operation speed, the human disassembly fatigue degree of the fasteners to be disassembled is fuzzy processed to obtain the human disassembly fatigue degree of the fasteners to be disassembled.

[0016] The specific aspects of 4) are:

[0017] The disassembly task allocation optimization target is constructed. According to the disassembly complexity of each fastener to be disassembled and the disassembly fatigue of the human body, the genetic algorithm is used to solve the disassembly task allocation optimization target, and then the human-machine task allocation scheme for the product to be disassembled is obtained.

[0018] The formula for the optimization objective of the disassembly task allocation is as follows:

[0019]

[0020] Among them, f is the comprehensive complexity of disassembly, x i Assign parameters to the disassembly task of the i-th fastener to be disassembled, x i =1 means the operator disassembles; x i =0 means the robot is disassembled, α and β are the first weight and the second weight respectively, The complexity of disassembly for the operator; is the disassembly complexity of the robot arm, c E,i is the disassembly fatigue of human body, and m is the number of fasteners to be disassembled.

[0021] 2. A real-time task allocation device for human-machine collaborative disassembly

[0022] include:

[0023] The image acquisition module is used to acquire images of products to be disassembled;

[0024] The storage module is used to store images of products to be disassembled, store recognition results of the two-stage neural network classifier, and store a human-machine task allocation plan for the products to be disassembled;

[0025] The calculation module is used to calculate and generate a human-machine task allocation plan for the product to be disassembled based on the image of the product to be disassembled;

[0026] The display module is used to display the quality status of the fasteners to be disassembled and the human-machine task allocation scheme.

[0027] 3. A computer device

[0028] The device comprises a memory and a processor, wherein the memory stores a computer program, and the steps of the method are implemented when the processor executes the computer program.

[0029] 4. A computer-readable storage medium

[0030] The medium stores a computer program, which implements the steps of the method when executed by a processor.

[0031] The beneficial effects of the present invention are:

[0032] 1. The present invention takes photos of the products to be disassembled and uses a two-stage neural network classifier to detect the quality status of the fasteners to be disassembled, thereby solving the problem of uncertainty in the disassembly object during disassembly.

[0033] 2. The present invention uses a task allocation algorithm and combines the quality status of the fasteners to be disassembled to reasonably allocate human-machine disassembly tasks, improve the efficiency of disassembly operations, and optimize the collaborative operation mode of human resources and automated equipment.

[0034] 3. Compared with the traditional human-machine collaborative method, the present invention adopts a two-level neural network classifier and an efficient algorithm to achieve a flexible, reasonable and efficient real-time allocation method for product human-machine collaborative disassembly tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to further illustrate the contents described in the present invention, the specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. It should be understood that these drawings are only used as typical examples and should not be regarded as limiting the scope of the present invention.

[0036] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0037] Figure 2 It is a schematic diagram of the main functional modules of the overall device of the method of the present invention.

[0038] Figure 3 Result diagram for identifying the quality status of fasteners to be removed.

[0039] Figure 4 This is a state classification diagram. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0041] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0042] like Figure 1 As shown, the method comprises the following steps:

[0043] 1) Using a camera to collect images of the product to be disassembled, and obtaining images of the product to be disassembled;

[0044] 2) Using a two-stage neural network classifier, the quality status of the fasteners to be disassembled in the image of the product to be disassembled is detected;

[0045] 2), a two-stage neural network classifier is used to detect the quality status of the fasteners to be disassembled in the image of the product to be disassembled, and the two-stage neural network classifier includes a fastener status classification neural network and a fastener quality classifier neural network connected in sequence.

[0046] The fastener status classification neural network classifies the status of the fasteners to be removed into normal status, thread stripping status and rusty status; the fastener quality classifier neural network outputs the quality status of the fasteners to be removed, such as Figure 3 As shown in Figure 1, the quality status of the fasteners to be disassembled includes normal, moderately stripped, moderately rusted, severely stripped, and severely rusted. Figure 4 shown.

[0047] In this example, the quality status of the fasteners to be disassembled is shown in Table 1:

[0048] Table 1 shows the quality status of each fastener in the product to be disassembled

[0049] Fasteners to be removed 1 2 3 4 5 6 Quality status normal Medium Slip Severe rust normal Severe thread slip Moderate rust

[0050] 3) Calculate the disassembly complexity and human disassembly fatigue of the fastener to be disassembled according to the quality status of the fastener to be disassembled;

[0051] 3), the disassembly complexity of the fastener to be disassembled is calculated according to the quality status of the fastener to be disassembled, specifically:

[0052] According to the quality status of the fastener to be disassembled, combined with the disassembly weight, disassembly tools and disassembly operation force, the disassembly complexity of the fastener to be disassembled is fuzzy processed to obtain the disassembly complexity of the fastener to be disassembled, which includes the disassembly complexity of the operator and the disassembly complexity of the robot arm. Specifically, the following steps are included:

[0053] First, according to the natural semantic information table of the quality status and disassembly complexity attribute of the fastener to be disassembled, the fuzzy attribute processing value of the disassembly weight of the fastener to be disassembled is calculated. Disassembly tool fuzzy attribute processing value and disassembly operation force fuzzy attribute processing value Then, the disassembly weight fuzzy attribute processing value is calculated based on the mass state of the fastener to be disassembled. Disassembly tool fuzzy attribute processing value and disassembly operation force fuzzy attribute processing value Calculating the fuzzy disassembly complexity of the fastener to be disassembled satisfy Using triangular fuzzy number defuzzification operator to defuzzify the complexity of fuzzy Defuzzification is converted to disassembly complexity c C,i Among them, the natural semantic information table of disassembly complexity attributes is shown in Table 2.

[0054] Table 2 is the natural semantic information table of disassembly complexity attributes

[0055]

[0056] Fuzzy attribute processing value of disassembly weight of fasteners to be disassembled The calculation function is as follows:

[0057]

[0058] Fuzzy attribute processing value of the disassembly tool of the fastener to be disassembled The calculation function is as follows:

[0059]

[0060] Fuzzy attribute processing value of disassembly operation force of fasteners to be disassembled The calculation function is as follows:

[0061]

[0062] Operator disassembly complexity of fasteners to be disassembled in different situations and robot disassembly complexity As shown in Table 3.

[0063] Table 3 shows the operator disassembly complexity of the fasteners to be disassembled in different situations and robot disassembly complexity

[0064]

[0065] 3), according to the quality status of the fastener to be disassembled, the human disassembly fatigue of the fastener to be disassembled is calculated, specifically:

[0066] According to the quality status of the fastener to be disassembled, combined with the disassembly work intensity, disassembly work time and disassembly operation speed, the human disassembly fatigue degree of the fastener to be disassembled is fuzzy processed to obtain the human disassembly fatigue degree of the fastener to be disassembled. Specifically, the following steps are included:

[0067] First, according to the quality status of the fastener to be disassembled and the natural semantic information table of the human body disassembly fatigue attribute, the disassembly work intensity attribute processing value {c I,i}、Disassembly work time attribute processing value {c H,i}, disassemble the body posture attribute processing value {c P,i} and the teardown operation speed attribute processing value {c V,i Then, the disassembly work intensity attribute processing value c is calculated according to the quality state of the fastener to be disassembled. I,i , Disassembly work time attribute processing value c H,i , disassemble the body posture attribute processing value c P,i and the disassembly operation speed attribute processing value c v,i , calculate the human disassembly fatigue of the fastener to be disassembled: c E,i =0.2×c I,i +0.2×c H,i +0.2×c P,i +0.2×c V,i Among them, the natural semantic information table of human body disassembly fatigue attribute is shown in Table 4.

[0068] Table 4 is the natural semantic information table of human disassembly fatigue attributes

[0069]

[0070] Disassembly work strength attribute processing value c of the fastener to be disassembled I,i The calculation function is as follows:

[0071]

[0072] Disassembly working time attribute processing value c of the fastener to be disassembled H,i The calculation function is as follows:

[0073]

[0074] Disassembly body posture attribute processing value c of the fastener to be disassembled P,i The calculation function is as follows:

[0075]

[0076] Disassembly operation speed attribute processing value c of the fastener to be disassembled V,i The calculation function is as follows:

[0077]

[0078] Human disassembly fatigue score c of the fastener to be disassembled E,i As shown in Table 5.

[0079] Table 5 shows the fatigue scores of human disassembly of fasteners to be disassembled c E, i

[0080] <![CDATA[Human disassembly fatigue degree c E, i]]> Medium Slip 0.2×6.0+0.2×1.0+0.2×1.5+0.2×1.0=1.9 Severe thread slip 0.2×13.0+0.2×3.0+0.2×3.0+0.2×1.0=4.0 Moderate rust 0.2×6.0+0.2×1.0+0.2×1.5+0.2×1.0=1.9 Severe rust 0.2×9.0+0.2×2.0+0.2×2.0+0.2×1.0=2.8 Normal screw 0.2×1.0+0.2×1.0+0.2×1.0+0.2×1.0=0.8

[0081] The calculated removal complexity c of each fastener to be removed C,i and human disassembly fatigue c E,i As shown in Table 6.

[0082] Table 6 shows the disassembly complexity and human disassembly fatigue of each fastener to be disassembled

[0083]

[0084] 4) According to the disassembly complexity of each fastener to be disassembled and the disassembly fatigue of the human body, a human-machine task allocation plan for the product to be disassembled is calculated and generated.

[0085] 4) Specifically:

[0086] The disassembly task allocation optimization target is constructed. According to the disassembly complexity of each fastener to be disassembled and the disassembly fatigue of the human body, the disassembly task allocation scheme of the product to be disassembled is obtained after solving the disassembly task allocation optimization target using the genetic algorithm. Among them, the formula of the disassembly task allocation optimization target is as follows:

[0087]

[0088] Among them, f is the comprehensive complexity of disassembly, x i Assign parameters to the disassembly task of the i-th fastener to be disassembled, x i =1 means the operator disassembles; x i =0 means the robot is disassembled, α and β are the first weight and the second weight respectively, α=0.3, β=0.7, The complexity of disassembly for the operator; is the disassembly complexity of the robot arm, c E,i is the disassembly fatigue of human body, and m is the number of fasteners to be disassembled.

[0089] The final calculated human-machine task allocation scheme is shown in Table 7.

[0090] Table 7 Human-machine task allocation scheme

[0091] Fasteners to be removed 1 2 3 4 5 6 <![CDATA[Human-machine task allocation x i > 0 1 1 0 1 1

[0092] like Figure 2 As shown, a real-time task allocation device for human-machine collaborative disassembly includes an image acquisition module, a storage module, a calculation module, and a display module;

[0093] An image acquisition module, used for acquiring images of products to be disassembled;

[0094] A storage module, used for storing images of products to be disassembled, storing recognition results of a two-stage neural network classifier, and storing a human-machine task allocation plan for products to be disassembled;

[0095] A calculation module for solving the human-machine task allocation scheme for calculating and generating fasteners;

[0096] The display module is used to display the quality status of the fasteners to be disassembled and the human-machine task allocation plan.

[0097] The above specific embodiments are only exemplary of the principles and effects of the present invention and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A real-time task allocation method for human-machine collaborative disassembly, characterized in that: The following steps are involved: 1) Capturing images of the product to be disassembled to obtain images of the product to be disassembled; 2) Detecting the quality status of the fasteners to be disassembled in the image of the product to be disassembled; 3) Calculate the disassembly complexity and human disassembly fatigue of the fastener to be disassembled according to the quality status of the fastener to be disassembled; 4) According to the disassembly complexity of each fastener to be disassembled and the disassembly fatigue of the human body, a human-machine task allocation plan for the product to be disassembled is calculated and generated.

2. A real-time task allocation method for human-machine collaborative disassembly according to claim 1, characterized in that: In the above 2), a two-stage neural network classifier is used to detect the quality status of the fasteners to be disassembled in the image of the product to be disassembled, and the two-stage neural network classifier includes a fastener status classification neural network and a fastener quality classifier neural network connected in sequence.

3. The real-time task allocation method for human-machine collaborative disassembly according to claim 2 is characterized in that: The fastener state classification neural network divides the state of the fasteners to be removed into a normal state, a stripped state and a rusted state; the fastener quality classifier neural network outputs the quality state of the fasteners to be removed, and the quality state of the fasteners to be removed includes normal, moderately stripped, moderately rusted, severely stripped and severely rusted.

4. The method for real-time task allocation for human-machine collaborative disassembly according to claim 1, characterized in that: In the above 3), the disassembly complexity of the fastener to be disassembled is calculated according to the quality status of the fastener to be disassembled, specifically: According to the quality status of the fasteners to be disassembled, combined with the disassembly weight, disassembly tools and disassembly operation force, the disassembly complexity of the fasteners to be disassembled is fuzzy processed to obtain the disassembly complexity of the fasteners to be disassembled. The disassembly complexity includes the disassembly complexity of the operator and the disassembly complexity of the robotic arm.

5. The method for real-time task allocation for human-machine collaborative disassembly according to claim 1, characterized in that: In the above 3), the human disassembly fatigue degree of the fastener to be disassembled is calculated according to the quality state of the fastener to be disassembled, specifically: According to the quality status of the fasteners to be disassembled, combined with the disassembly work intensity, disassembly work time and disassembly operation speed, the human disassembly fatigue degree of the fasteners to be disassembled is fuzzy processed to obtain the human disassembly fatigue degree of the fasteners to be disassembled.

6. The method for real-time task allocation for human-machine collaborative disassembly according to claim 1, characterized in that: The specific aspects of 4) are: The disassembly task allocation optimization target is constructed. According to the disassembly complexity of each fastener to be disassembled and the disassembly fatigue of the human body, the genetic algorithm is used to solve the disassembly task allocation optimization target, and then the human-machine task allocation scheme for the product to be disassembled is obtained.

7. The method for real-time task allocation for human-machine collaborative disassembly according to claim 6, characterized in that: The formula for the optimization objective of the disassembly task allocation is as follows: min{f} Among them, f is the comprehensive complexity of disassembly, x i Assign parameters to the disassembly task of the i-th fastener to be disassembled, x i =1 means the operator disassembles; x i =0 means the robot is disassembled, α and β are the first weight and the second weight respectively, The complexity of disassembly for the operator; is the disassembly complexity of the robot arm, c E,i is the disassembly fatigue of human body, and m is the number of fasteners to be disassembled.

8. A real-time task allocation device for human-machine collaborative disassembly for implementing any of the methods described in claims 1-7, characterized in that: include: The image acquisition module is used to acquire images of products to be disassembled; The storage module is used to store images of products to be disassembled, store recognition results of the two-stage neural network classifier, and store a human-machine task allocation plan for the products to be disassembled; The calculation module is used to calculate and generate a human-machine task allocation plan for the product to be disassembled based on the image of the product to be disassembled; The display module is used to display the quality status of the fasteners to be disassembled and the human-machine task allocation scheme.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Bending machine selection method, carried out on a control system of a bending machine

    CN110446987A

  • Man-machine cooperation dismounting sequence planning method based on personnel fatigue

    CN110968950A

  • Image capturing apparatus, control method, and program

    CN112543256A

  • Detection method and device of communication equipment and electronic equipment

    CN114022404A

  • Robot U-shaped disassembly line dynamic balance method based on deep reinforcement learning

    CN116690589A