Real-time task allocation method and apparatus for human-robot collaborative disassembly

Abstract

To provide a real-time task distribution method and device for disassembly work by human and robot cooperation.SOLUTION: The present invention obtains an image of a fastening part to be disassembled, then constructs a two stage neural network classifier to detect the quality state of the fastening part to be disassembled, and finally uses a genetic algorithm to solve a disassembly human and robot task distribution plan, thereby realizing the detection and state classification of the fastening part to be disassembled. The real-time human-robot collaborative task distribution can make full use of the advantages of the accuracy and durability of the robotic arm, and at the same time, use the intelligent resources of the operator to greatly reduce the fatigue of the operator. In addition, the flexibility of the disassembly work is improved, the problem of uncertain failure of the product is solved, and the quality, safety and efficiency of the disassembly work are improved by making the best use of the flexible response ability of the human and the accurate repetitive operation ability of the robotic arm.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a method for allocating disassembly tasks through collaboration between humans and robots, which belongs to a new generation of human-information-physical systems (CPS) in smart manufacturing, and in particular to a method and apparatus for real-time allocation of disassembly tasks between humans and robots in collaboration with products. [Background technology]

[0002] "Industry 5.0," which aims for human-centered and sustainable development, is ushering in a new industrial revolution. Human-Networked-Physical Systems (CPS) are the foundation of human-centered manufacturing, and remanufacturing has received widespread attention in promoting sustainability. Disassembly is a crucial and challenging task in remanufacturing. However, this process faces many complex issues. For example, robotic arms have difficulty adapting to complex and diverse disassembly scenarios, and human workers are constrained by fatigue and safety risks in the disassembly process.

[0003] Although human-robot collaboration has achieved some success in other industrial fields, significant challenges remain in the disassembly of complex products. For example, how to formulate a rational human-robot task allocation strategy when faced with highly uncertain situations is an issue. When product uncertainty exists, the limitations of purely robotic arm disassembly and the fatigue and safety risks to human workers mean that current human-robot collaboration models often result in irrational pre-defined human-robot task allocation and fail to fully consider the overall impact of task allocation on sequence planning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication JP2024-018151 Summary of the Invention [Problem to be solved by the invention]

[0005] To address the problems in the background art, the present invention provides a real-time task allocation method and apparatus for human-robot collaborative disassembly. The method acquires images of fasteners to be disassembled and builds a two-stage neural network classifier to identify the quality status of the fasteners. A genetic algorithm is then used to solve a human-robot disassembly task allocation plan, achieving the detection and status classification of the fasteners to be disassembled, and real-time human-robot collaborative task allocation. This fully utilizes the precision and durability of the robot arm and utilizes the intellectual resources of the operator. This significantly reduces operator fatigue and maximizes the human's ability to flexibly respond to complex situations and the robot arm's ability to perform precise and repetitive operations. This improves the flexibility of disassembly, eliminates product failures due to product uncertainty, and improves the disassembly and recyclability of products in different states, thereby improving the quality, safety, and efficiency of disassembly. [Means for solving the problem]

[0006] First, a real-time task allocation method for human-robot collaborative disassembly tasks includes the following steps:

[0007] (1) An image of the decomposition object is acquired, and an image of the decomposition object is obtained. (2) Detect the quality state of the fastening parts to be disassembled contained in the image of the disassembled object. (3) Based on the quality condition of the fastening parts to be disassembled, the complexity of the disassembly and the fatigue level of the worker in the disassembly work are calculated. (4) Based on the disassembly complexity and human fatigue level of each disassembly target fastener, calculate and generate a human-robot task allocation plan for the disassembly target fastener.

[0008] In (2) above, A two-stage neural network classifier is used to detect the quality state of the fasteners to be disassembled in the image of the disassembly object, and the two-stage neural network classifier is composed of a neural network for classifying the state of the fasteners to be disassembled and a neural network for classifying the quality of the fasteners to be disassembled, which are connected in series.

[0009] The neural network for classifying the state of fasteners to be disassembled classifies the state of fasteners to be disassembled into "normal state," "slippage state," and "rust state." The neural network for classifying the quality of fasteners to be disassembled outputs the quality state of fasteners to be disassembled, which includes "normal," "moderate slippage state," "moderate rust," "severe slippage state," and "severe rust."

[0010] In (3) above, The disassembly complexity of the fastening parts to be disassembled is calculated based on the quality state of the fastening parts to be disassembled. Specifically, the disassembly complexity of the fastening parts to be disassembled is obtained by fuzzy processing of the disassembly complexity of the fastening parts to be disassembled based on the quality state of the fastening parts to be disassembled, combining the weight required for disassembly, the tools used, and the operating force during the disassembly operation. Note that the disassembly complexity includes the disassembly complexity by a worker and the disassembly complexity by a robot arm.

[0011] Furthermore, in (3) above, The degree of fatigue during disassembly work by a person is calculated based on the quality state of the fastening parts to be disassembled. Specifically, the fatigue degree of the person disassembling the fastening parts to be disassembled is obtained by fuzzy processing of the quality state of the fastening parts to be disassembled, combining the intensity, work time, and work speed of the disassembly work.

[0012] Details of (4) above An optimization goal for disassembly task allocation is established, and the optimization goal for disassembly task allocation is solved using a genetic algorithm based on the disassembly complexity of each fastener to be disassembled and the disassembly fatigue level of the worker, resulting in a human and robot task allocation plan for the fastener to be disassembled.

[0013] The mathematical expression of the optimization goal of the decomposition task allocation is as follows: JPEG2026013337000002.jpg36117

[0014] Second, a real-time task allocation system for human-robot collaborative disassembly work. It includes the following components: (a) Image acquisition module: A module for acquiring images of fasteners to be disassembled. (b) Memory module: A module for storing images of fasteners to be disassembled, the recognition results of the two-stage neural network classifier, and the human-robot task allocation plan for the fasteners to be disassembled. (c) Computation module: A module for performing calculations based on the images of the fasteners to be disassembled and generating a human-robot task allocation plan for the fasteners to be disassembled. (d) Display module: A module for displaying the quality status of fasteners to be disassembled and a human-robot task allocation plan.

[0015] Third, computer equipment The apparatus includes a memory and a processor, the memory storing a computer program, and the processor executing the program to implement the steps of the method described above.

[0016] Fourth, computer-readable storage medium The medium stores a computer program, which, when executed by a processor, implements the steps of the above-described method. [Effects of the Invention]

[0017] (1) In the present invention, the problem of uncertain situations during disassembly is solved by photographing the fastening parts to be disassembled and detecting the quality state of the fastening parts to be disassembled using a two-stage neural network classifier.

[0018] (2) In this invention, a task allocation algorithm is used to rationally allocate human and robot disassembly tasks while taking into account the quality status of the fasteners to be disassembled, thereby improving the efficiency of disassembly work and optimizing the collaborative work model between human resources and automated equipment.

[0019] (3) Compared with conventional human-robot collaboration methods, the present invention uses a two-stage neural network classifier and an efficient algorithm to realize a flexible, rational, and efficient real-time allocation method for human-robot collaborative disassembly tasks of products. [Brief explanation of the drawings]

[0020] In order to further explain the contents of the present invention, specific embodiments of the present invention will be described in detail below in conjunction with the drawings, which are provided as typical examples and are not intended to limit the scope of the present invention. [Figure 1] 1 is an overall flow diagram of the method of the present invention. [Figure 2] 1 is a schematic diagram of the main functional modules of the overall apparatus of the method of the present invention; [Figure 3] 10 is a diagram showing the recognition results of the quality state of the fastening parts to be disassembled. [Figure 4] Condition classification diagram. DETAILED DESCRIPTION OF THE INVENTION

[0021] In order to clarify the objectives, technical solutions and advantages of the present invention, the present invention will be described in more detail below in combination with drawings and examples. The specific examples described here are for the purpose of illustrating the present invention only and are not intended to limit the present invention. In addition, the technical features of each embodiment of the present invention described below can be combined as long as they are not mutually inconsistent.

[0022] The present invention will be further described below in conjunction with the drawings and specific examples. As shown in FIG. 1, the method includes the following steps: (1) A step of acquiring an image of a fastening part to be disassembled using a camera, and obtaining an image of the fastening part to be disassembled. (2) using a two-stage neural network classifier to detect the quality status of the fasteners to be disassembled contained in the images of the fasteners to be disassembled;

[0023] In the above step (2), a two-stage neural network classifier is used to detect the quality state of the fasteners to be disassembled in the image of the fasteners to be disassembled. The two-stage neural network classifier is composed of a neural network for classifying the state of the fasteners to be disassembled and a neural network for classifying the quality of the fasteners to be disassembled, which are connected in series.

[0024] The condition classification neural network for disassembly target fasteners classifies the conditions of the disassembly target fasteners into "normal condition," "slippage condition," and "rust condition." The quality classification neural network for disassembly target fasteners outputs the quality conditions of the disassembly target fasteners, as shown in Figure 3. The quality conditions include "normal," "moderate slippage condition," "moderate rust," "severe slippage condition," and "severe rust" (see Figure 4).

[0025] In this embodiment, the quality states of the fastening parts to be disassembled are specifically shown in Table 1.

[0026] Table 1 Quality status of each fastening part to be disassembled JPEG2026013337000003.jpg26142

[0027] (3) A step of calculating the complexity of disassembly and the degree of disassembly fatigue of a worker based on the quality state of the fastening parts to be disassembled.

[0028] In step (3), the complexity of the disassembly is calculated based on the quality of the fastening parts to be disassembled. Specifically, this is done as follows: Based on the quality condition of the fastening parts to be disassembled, the disassembly weight, the tools used for disassembly, and the force during the disassembly operation are combined and fuzzy processing is performed to determine the disassembly complexity of the fastening parts to be disassembled. This disassembly complexity includes the disassembly complexity of the worker and the disassembly complexity of the robot arm. The specific steps are as follows:

[0029] First, based on the quality status of the fasteners to be disassembled and the table of natural language semantics information on the disassembly complexity attributes, JPEG2026013337000004.jpg26116

[0030] As a result, the following relational expression holds: JPEG2026013337000005.jpg22152In this case, the table of natural language semantics information regarding the decomposition complexity attribute is shown in Table 2.

[0031] Table 2. Natural language semantic information for decomposition complexity attributes JPEG2026013337000006.jpg83114

[0032] JPEG2026013337000007.jpg38155

[0033] JPEG2026013337000008.jpg39153

[0034] JPEG2026013337000009.jpg39159

[0035] JPEG2026013337000010.jpg10115

[0036] JPEG2026013337000011.jpg79115

[0037] In step (3), the degree of disassembly fatigue of the worker is calculated based on the quality state of the fastening parts to be disassembled.

[0038] Specifically, the following applies: Based on the quality state of the fastening parts to be disassembled, the strength of the disassembly work, the disassembly work time, the posture during the disassembly work, and the speed of the disassembly work are combined to perform fuzzy processing to determine the disassembly fatigue degree of the worker for the fastening parts to be disassembled, and the disassembly fatigue degree of the worker is obtained as a result. The specific steps are as follows:

[0039] First, based on the natural language semantics information table regarding the quality status of the fasteners to be disassembled and the disassembly fatigue attribute of the workers, JPEG2026013337000012.jpg15169

[0040] JPEG2026013337000013.jpg15116

[0041] JPEG2026013337000014.jpg8127Note that the natural language semantics information table for the worker disassembly fatigue attribute is shown in Table 4.

[0042] Table 4. Natural language semantics information table for worker fatigue attribute JPEG2026013337000015.jpg57170

[0043] JPEG2026013337000016.jpg32117

[0044] JPEG2026013337000017.jpg31116

[0045] JPEG2026013337000018.jpg31115

[0046] JPEG2026013337000019.jpg33116

[0047] JPEG2026013337000020.jpg55117

[0048] Table 6 shows the disassembly complexity and worker disassembly fatigue for each undisassembled fastener. JPEG2026013337000021.jpg50150(4) Based on the disassembly complexity and worker fatigue level of each undisassembled fastener to be disassembled, a human and robot task allocation plan for the undisassembled product is calculated and generated.

[0049] The details of step (4) are as follows: An optimization goal for disassembly task allocation is established, and the optimization goal for disassembly task allocation is solved using a genetic algorithm based on the disassembly complexity and worker disassembly fatigue of each undisassembled fastener, and a human-robot task allocation plan for the undisassembled product is obtained. At this time, the optimization goal for disassembly task allocation is expressed as follows:

[0050] JPEG2026013337000022.jpg36116

[0051] The final calculated human and robot task distribution plan is shown in Table 7. Table 7 Human-robot task allocation method JPEG2026013337000023.jpg20115As shown in Figure 2, the task real-time distribution device for human-robot collaborative decomposition includes an image collection module, a storage module, a calculation module, and a display module.

[0052] The image acquisition module is for acquiring images of the undisassembled product. The memory module is for storing the image of the undisassembled product, the classification result of the two-stage neural network classifier, and the human-robot task allocation plan for the undisassembled product. The calculation module is for solving and calculating the human and robot task distribution plan for the fasteners to be disassembled.

[0053] The display module is for displaying the quality status of the fasteners that are not yet disassembled and are the target of disassembly, and also for displaying the task distribution plan between humans and robots.

[0054] The above specific examples are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Therefore, those skilled in the art may make modifications or changes to the above examples without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes that do not depart from the disclosed spirit and technical concept of the present invention should still be encompassed by the claims of the present invention.

Claims

1. A real-time task allocation method for human-robot collaborative disassembly, comprising: (1) an image collection step for acquiring images of fastening components to be disassembled; (2) detecting the quality state of the fastening parts to be disassembled from the images of the fastening parts to be disassembled; (3) calculating the disassembly complexity and the worker disassembly fatigue level of the fastening parts to be disassembled based on the detected quality states of the fastening parts to be disassembled; (4) calculating a human-robot task distribution plan for each of the fastening parts to be disassembled based on the disassembly complexity and the worker disassembly fatigue level of each of the fastening parts to be disassembled; is executed by a computer, A real-time task distribution method.

2. In the step (2), a two-stage neural network classifier is used to detect the quality state of the fastening parts to be disassembled from the images of the fastening parts to be disassembled; The two-stage neural network classifier is configured by sequentially connecting a neural network for classifying the state of fasteners to be disassembled and a neural network for classifying the quality of fasteners to be disassembled.

2. The method for real-time task distribution according to claim 1.

3. The neural network for classifying the state of the fastening parts to be disassembled classifies the state of the fastening parts to be disassembled into "normal state," "slippage of threads," and "rust state," The quality classification neural network of the fasteners to be disassembled outputs the quality status of the fasteners to be disassembled, and the quality status includes "normal", "moderate thread slippage", "moderate rust", "severe thread slippage", and "severe rust".

3. The method for real-time task distribution according to claim 2.

4. In the step (3), a disassembly complexity of the fastening parts to be disassembled is calculated based on the quality state of the fastening parts to be disassembled; Specifically, based on the quality state of the fastening parts to be disassembled, the disassembly complexity of the fastening parts to be disassembled is calculated by performing fuzzy processing on the disassembly complexity of the fastening parts to be disassembled, taking into consideration the weight of the disassembly, the disassembly tool, and the operating force; The disassembly complexity includes the disassembly complexity by the worker and the disassembly complexity by the robot arm.

2. The method for real-time task distribution according to claim 1.

5. In the step (3), a worker disassembly fatigue level of the fastening parts to be disassembled is calculated based on the quality state of the fastening parts to be disassembled; Specifically, the worker disassembly fatigue level of the fastening parts to be disassembled is calculated by performing fuzzy processing on the worker disassembly fatigue level of the fastening parts to be disassembled, taking into consideration the work strength, work time, and operation speed based on the quality state of the fastening parts to be disassembled; 2. The method for real-time task distribution according to claim 1.

6. In the step (4), an optimization goal for decomposition task distribution is established; Based on the disassembly complexity and worker disassembly fatigue of each fastener to be disassembled, a genetic algorithm is used to solve the optimization goal of disassembly task allocation, thereby obtaining a human and robot task allocation plan for the fastener to be disassembled; 2. The method for real-time task distribution according to claim 1.

7. The optimization objective of the decomposition task distribution is expressed as follows:

7. The method for real-time task distribution according to claim 6.

8. A real-time task allocation device for human-robot collaborative disassembly for carrying out the method according to any one of claims 1 to 7, comprising: an image collection module for collecting images of fastening components to be disassembled; a storage module for storing an image of the fastener to be disassembled, a classification result of the two-stage neural network classifier, and a human-robot task distribution plan for the fastener to be disassembled; a calculation module for calculating a human-robot task distribution plan for the fastener to be disassembled based on an image of the fastener to be disassembled; A display module for displaying the quality status of fasteners to be disassembled and a human-robot task distribution plan. Equipped with A real-time task distribution device.

9. A computing device including a memory and a processor, The memory stores a computer program, and the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7. A computer device characterized in that:

10. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the method according to any one of claims 1 to 7 when executed by a processor. A computer-readable storage medium comprising:

Images