Visual identification method for multi-axis transfer robot

Abstract

The invention discloses a visual identification method for a multi-axis transfer robot. The method comprises the following steps that 1, the multi-axis transfer robot uses a CCD image sensor to collect an image of an object needing to be transferred; 2, designing a multi-axis transfer robot target detection system; step 3, incremental learning is carried out on the YOLOV5 target detection model; 4, designing a multi-axis transfer robot control model; and 5, control software of the multi-axis transfer robot is designed. Aiming at the visual identification and control problems of the multi-axis transfer robot, the invention provides a visual identification method for the multi-axis transfer robot, the target detection and control functions of the multi-axis robot are realized, and meanwhile, in order to improve the robustness and the accuracy of a detection model, a YOLOV5 model is optimized and upgraded on line, incremental learning of the YOLOV5 model is realized, and the robustness and the accuracy of the detection model are improved. A three-closed-loop control system is provided in the aspect of robot control, and nonlinear factors and system friction of a multi-axis robot are effectively overcome.

Description

Technical Field

[0001] The present invention relates to the field of robot vision, and particularly to a vision recognition method for a multi-axis handling robot. Background Art

[0002] Traditional industrial robots are all controlled by a central system to perform mechanical and repetitive actions. Industrial robots on an electrical automation production line can perform unified and standardized processing on products, but random and unpredictable mistakes are extremely likely to occur during the processing, causing unnecessary economic losses. Conventional mechanical sensors have great limitations. As the market's requirements for products gradually increase, industrial robots should also improve their production efficiency and quality to meet the development of society and the market. In recent years, with the rapid development of image processing technology and computer technology, vision technology has been widely applied in the field of robots. Vision-based robots can autonomously obtain information about the surrounding environment and can be used for autonomous positioning, attitude measurement, path planning, navigation, control, etc. of the robots. Based on the multi-axis robot, a vision unit is added, and a corresponding vision control system is designed to incorporate machine vision technology. The collected images are converted into the types and coordinate values of sorted products, and this information is directly or indirectly transmitted to the robot to achieve automatic grasping of the multi-axis robot. Summary of the Invention

[0003] To solve the above problems, the present invention proposes a vision recognition method for a multi-axis handling robot, and the specific steps are as follows, characterized in that:

[0004] Step 1, the multi-axis handling robot uses a CCD image sensor to collect images of the objects to be handled and uploads them to the image recognition data storage area of the application layer system through WIFI and Bluetooth;

[0005] Step 2, design a multi-axis handling robot target detection system: use the YOLOV5 target detection algorithm to identify the detection target, and train the YOLOV5 target detection model by reading the images collected by the CCD image sensor in the data storage area;

[0006] Step 3, for the situation where the detection target is incorrect, the target detection system will use the misrecognized samples as training samples and send them into the detection model to perform incremental learning on the YOLOV5 target detection model, thereby continuously improving the accuracy of the YOLOV5 target detection model.

[0007] Step 4, design a multi-axis handling robot control model: introduce the current negative feedback signal, speed negative feedback signal, and position feedback signal of the motor into the multi-axis handling robot control system, and respectively design the current feedback closed-loop, speed closed-loop, and position closed-loop control of the multi-axis handling robot to form a complete three-loop control system for the multi-axis handling robot;

[0008] Step 5, design the control software of the multi-axis handling robot: The vision recognition method software of the multi-axis handling robot has general functions and a convenient and friendly interface, which is divided into two control modes: manual and automatic. At the same time, it takes into account the scheduling, data storage, user interface, and communication functions of the multi-axis handling robot control tasks.

[0009] Furthermore, the process of designing the target detection system of the multi-axis handling robot in Step 2 can be expressed as follows:

[0010] The target detection system of the multi-axis handling robot of the present invention mainly uses the YOLOV5 target detection algorithm to complete the robot's recognition of the target. The Mosaci data augmentation algorithm is used at the input end of the model, and the training samples are spliced by means of random scaling, random cropping, and random arrangement. Algorithms such as automatic training prediction boxes and adaptive image scaling are used when training the training sample set. In the feature network for extracting the image features of the training sample set, the Focus algorithm and the CSP2 structure are used; in the Neck structure, the FPN+PAN structure is adopted to enhance the network's feature fusion ability; at the output end of the YOLOV5 network, GIOU_Loss is used as the loss function:

[0011]

[0012] In the formula, IoU is the IoU loss, A c is the area of the minimum circumscribed rectangle of the YOLOV5 detection box and the sample annotation, and U is the area of the union of the detection box and the annotation. The present invention uses non-maximum suppression to solve the problem of overlapping detection targets in the screening of the YOLOV5 target box;

[0013] The YOLOV5 model is trained on the NVIDIA CUDA platform using the GPU. After training the YOLOV5 target detection model, the trained model is processed by pruning and quantization, and then transplanted into the host computer software.

[0014] Furthermore, the process of performing incremental learning on the YOLOV5 target detection model in Step 3 can be expressed as follows:

[0015] The misclassified image data is correctly modified with its label and used as the training set. Incremental learning is performed on the basis of the trained YOLOV5 target detection model to obtain the updated threshold and weight, and the updated threshold and weight are transmitted to the host computer software to update the threshold and weight of the YOLOV5 target detection model in the software, and finally the optimization and upgrade of the model are realized.

[0016] Furthermore, the process of designing the control model of the multi-axis handling robot in Step 4 can be expressed as follows:

[0017] Each axis system of the multi-axis handling robot can achieve multiple operating modes: angular position, angular velocity, swaying, etc. modes to complete the handling of the target. To complete the handling actions of the multi-axis handling robot, in the present invention, a current loop is designed in the power amplifier of the motor, and at the same time, the speed loop of the motor is designed to broaden the system bandwidth, improve the speed response speed, and suppress the influence of motor torque fluctuation and dry friction on the system. Finally, on the basis of the current loop and the speed loop, a position loop of the system is added to control the actions of the robot.

[0018] To reduce the overshoot of the system, integral separation PID control is adopted for both the current loop, the speed loop and the position loop, and a logic function is introduced into the algorithm. The output sampling point value of the regulator is:

[0019]

[0020] Among them, u(k) represents the duty cycle of SPWM in the current loop, u(k) represents the current value in the speed loop, u(k) represents the speed value in the position loop, k is the discrete time variable, K p ,K i ,K d are the proportional coefficient, integral coefficient and differential coefficient of the PID algorithm respectively, e(j) is the system error at time j, A is the threshold of the PID control system. When the deviation is large, the integral term of the PID does not work. When the deviation is within the threshold, the integral algorithm is introduced to control the multi-axis operation of the robot.

[0021] Then, the parameters of the integral separation PID are tuned through the genetic algorithm. The steps are as follows:

[0022] Step 4.1, fuzzy domain coding; during the operation of the multi-axis handling robot, the actual range of the angular position deviation e(t) may reach 0 to 360 degrees. Therefore, its domain is defined as [0, 360], and the basic domain of the angular change rate is approximately [-40, 40]. After parameter tuning analysis, the output variables k p 、k i 、k d The actual domains are set as [-5, 5], [-0.4, 0.4], [-1, 1]; the fuzzy domains of the input and output are both set as [-6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6]; the system sets 7 language variable values of negative large NB, negative medium NM, negative small NS, zero ZO, positive small PS, positive medium PM, and positive large PB, and encodes them with 0, 1, 2, 3, 4, 5, 6.

[0023] Step 4.2, selection of the initial population; through the method of random generation, a group of initial PID parameters K p ,K i ,Kd As the population, the parameter range of the optimal solution of the initial population is set to [-6, 6], and an initial population of size 80 is randomly generated within the set range;

[0024] Step 4.3, selection of the fitness function; the fitness function determines the fitness of the individuals in the genetic algorithm through the performance indicators of the robot control. The fitness of the individuals is used as the criterion for evaluating the quality of the PID parameters. In the present invention, the overshoot, steady-state error, and response time indicators of the system are used to set the objective function of the genetic algorithm, and thus the fitness function is described as:

[0025]

[0026] In the formula, e 1 (t) is the steady-state error of the control system at time t, and e 2 (t) is the overshoot of the control system at time t, and e t (t) is the response time of the control system at time t, and ω 1 、ω 2 、ω 3 are weighting constants;

[0027] Step 4.4, selection of genetic operators; according to the fitness values, a part of the excellent individuals are selected as "parent generations" using the roulette wheel selection method. The probability of being selected is determined by the ratio of the fitness value of the individual in the overall population to the overall fitness value of the population. The formula is as follows:

[0028]

[0029] Among them, P i is the probability that individual i is genetically selected, f j is the overall fitness value of all individuals added together, and f i is the fitness value of individual i;

[0030] Step 4.5, crossover and mutation operations; for the selected "parent generation" individuals, crossover operations are performed to generate new individuals. In the present invention, a two-point crossover algorithm is adopted, and the crossover probability is set to 0.9, thus forming two new individuals; at the same time, to improve the local search ability, some genes of the individuals are randomly perturbed or changed, and the mutation probability is set to 0.03;

[0031] Step 4.6, update the population and judge the termination condition; add the generated new individuals to the original population to obtain the updated population, and judge whether the population meets the termination condition. If it does not meet, return to Step 4.4; if it meets, output the PID closed-loop parameters.

[0032] A visual recognition method for a multi-axis handling robot according to the present invention has the following beneficial effects:

[0033] 1. The present invention realizes the incremental learning of the YOLOV5 model, can perform online optimization and upgrade on the YOLOV5 model, and improve the robustness and accuracy of the detection model;

[0034] 2. The present invention proposes a three-closed-loop control system in the control of the robot, which effectively overcomes the non-linear factors and system friction of the multi-axis robot;

[0035] 3. The present invention provides an important technical means for the target recognition and control strategy of the multi-axis handling robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is the system structure diagram of the present invention;

[0037] Figure 2 is the three-loop PID control diagram of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0038] The present invention will be further described in detail below in conjunction with the drawings and the specific embodiments:

[0039] The present invention proposes a vision recognition method for a multi-axis handling robot, Figure 1 is the system structure diagram of the present invention. The steps of the present invention will be introduced in detail below in conjunction with the control structure diagram.

[0040] Step 1, the multi-axis handling robot uses a CCD image sensor to collect the image of the object to be handled, and uploads it to the image recognition data storage area of the application layer system through WIFI and Bluetooth;

[0041] Step 2, design a target detection system for the multi-axis handling robot: use the YOLOV5 target detection algorithm to identify the detection target, and train the YOLOV5 target detection model by reading the images collected by the CCD image sensor in the data storage area;

[0042] The process of designing the target detection system for the multi-axis handling robot in Step 2 can be expressed as follows:

[0043] The target detection system of the multi-axis handling robot of the present invention mainly uses the YOLOV5 target detection algorithm to complete the robot's recognition of the target. At the input end of the model, the Mosaci data augmentation algorithm is used, and the training samples are spliced in a random scaling, random cropping, and random arrangement manner. When training the training sample set, algorithms such as automatic training prediction boxes and adaptive image scaling are used. In the feature network for extracting the image features of the training sample set, the Focus algorithm and the CSP2 structure are used; in the Neck structure, the FPN+PAN structure is adopted to enhance the network feature fusion ability; at the output end of the YOLOV5 network, the GIOU_Loss is used as the loss function:

[0044]

[0045] Where IoU is the IoU loss, A c is the area of the minimum bounding rectangle of the YOLOV5 detection box and the sample annotation, and U is the area of the union of the detection box and the annotation. In the present invention, non-maximum suppression is used in the screening of YOLOV5 target boxes to solve the problem of overlapping detection targets;

[0046] The YOLOV5 model is trained on the NVIDIA CUDA platform using GPU. After training the YOLOV5 object detection model, the trained model is processed by pruning and quantization and then transplanted into the host computer software.

[0047] Step 3, for the case where the detected target is incorrect, the target detection system will use the misrecognized sample as a training sample and send it into the detection model to perform incremental learning on the YOLOV5 object detection model, thereby continuously improving the accuracy of the YOLOV5 object detection model.

[0048] The process of performing incremental learning on the YOLOV5 object detection model in Step 3 can be expressed as follows:

[0049] Correctly modify the label of the misclassified image data as the training set, perform incremental learning on it based on the already trained YOLOV5 object detection model to obtain the updated threshold and weight, and transfer the updated threshold and weight to the host computer software to update the threshold and weight of the YOLOV5 object detection model in the software, ultimately achieving the optimization and upgrade of the model.

[0050] Step 4, design a multi-axis handling robot control model: In the multi-axis handling robot control system, the current negative feedback signal, speed negative feedback signal, and position feedback signal of the motor are introduced, and the current feedback closed-loop, speed closed-loop, and position closed-loop control of the multi-axis handling robot are designed respectively to form a complete three-loop control system for the multi-axis handling robot;

[0051] The process of designing the multi-axis handling robot control model in Step 4 can be expressed as follows:

[0052] Each axis system of the multi-axis handling robot can achieve multiple operating modes: angular position, angular speed, swaying, etc. modes to complete the handling of the target; To complete the handling action of the multi-axis handling robot, in the present invention, a current loop circuit is designed in the power amplifier of the motor, and at the same time, the speed loop of the motor is designed to broaden the system bandwidth, improve the speed response speed, suppress the influence of motor torque fluctuation and dry friction on the system, and finally, on the basis of the current loop and speed loop, a position loop of the system is added to control the action of the robot;

[0053] To reduce the overshoot of the system, integral separation PID control is adopted for the current loop, speed loop, and position loop, and a logic function is introduced into the algorithm. The output sampling point value of the regulator is:

[0054]

[0055] Among them, K p ,K i ,K d are the proportional coefficient, integral coefficient, and differential coefficient of the PID algorithm respectively. E(k) represents the system error at the current moment k, and e(j) is the system error at moment j. A is the threshold of the PID control system. When the deviation is large, the integral term of the PID does not work. When the deviation is within the threshold, the integral algorithm is introduced to control the multi-axis operation of the robot. The three-loop PID control diagram is as shown in Figure 2 shown.

[0056] Then, the genetic algorithm is used to tune the parameters of the integral separation PID. The steps are as follows:

[0057] Step 4.1, fuzzy domain coding; during the operation of the multi-axis handling robot, the actual range of the position deviation e(t) of the angle may reach 0 to 360 degrees. Therefore, its domain is set as [0, 360]. The basic domain of the angle change rate is approximately [-40, 40]. After parameter tuning analysis, the output quantities k p 、k i 、k d have the actual domains set as [-5, 5], [-0.4, 0.4], [-1, 1]; the fuzzy domains of the input and output are both set as [-6, -5, -4, -3, 2, -1, 0, 1, 2, 3, 4, 5, 6]; the system sets 7 language variable values of negative large NB, negative medium NM, negative small NS, zero ZO, positive small PS, positive medium PM, and positive large PB, and encodes them with 0, 1, 2, 3, 4, 5, 6;

[0058] Step 4.2, selection of the initial population; through the method of random generation, a group of initial PID parameters K p ,K i ,K d of the current loop, speed loop, and position loop are initialized as the population. The optimal solution parameter range of the initial population is set as [-6, 6], and an initial population of size 80 is randomly generated within the set range;

[0059] Step 4.3, selection of the fitness function; the fitness function judges the fitness of the individuals of the genetic algorithm through the robot control performance index, and takes the fitness of the individuals as the criterion for evaluating the quality of the PID parameters. In the present invention, the overshoot, steady-state error, and response time index of the system are used to set the objective function of the genetic algorithm, so that the description based on the fitness function is:

[0060]

[0061] where e 1 is the steady-state error of the control system, and e 2 is the overshoot of the control system, and e t is the response time of the control system, and ω 1 and ω 2 and ω 3 are weighting constants;

[0062] Step 4.4, select genetic operators; according to the fitness value, use the roulette wheel selection method to select a part of excellent individuals as "parent generations", and use the ratio of the fitness value of the individuals in the overall population to the overall fitness value to determine the probability of being selected. The formula is as follows:

[0063]

[0064] where P i is the probability that individual i is genetically selected, and f j is the overall fitness value of all individuals added together, and f i is the fitness value of individual i;

[0065] Step 4.5, crossover and mutation operations; for the selected "parent generation" individuals, perform crossover operations to generate new individuals. The present invention adopts a two-point crossover algorithm and sets the crossover probability to 0.9, thereby forming two new individuals; at the same time, to improve the local search ability, randomly perturb or change some genes of the individuals, and set the mutation probability to 0.03;

[0066] Step 4.6, update the population and judge the termination condition; add the generated new individuals to the original population to obtain the updated population, and judge whether this population meets the termination condition. If it does not meet, return to Step 4.4. If it meets, output the PID closed-loop parameters.

[0067] Step 5, design the control software of the multi-axis handling robot: The software of the multi-axis handling robot vision recognition method has general functions and a convenient and friendly interface, and is divided into two control modes: manual and automatic. At the same time, it takes into account the scheduling, data storage, user interface, and communication functions of the multi-axis handling robot control tasks.

[0068] The above is only a preferred embodiment of the present invention, and it is not any other form of limitation to the present invention. Any modification or equivalent change made according to the technical essence of the present invention still belongs to the scope protected by the present invention.

Claims

1. A multi-axis handling robot visual recognition method, the specific steps are as follows, characterized in that: Step 1: The multi-axis handling robot uses a CCD image sensor to collect images of objects to be handled, and uploads them to the image recognition data storage area of ​​the application layer system via WIFI and Bluetooth; Step 2, design a multi-axis handling robot target detection system: use the YOLOV5 target detection algorithm to identify the detection target, and train the YOLOV5 target detection model by reading the images collected by the CCD image sensor in the data storage area; The process of designing the target detection system of the multi-axis handling robot in step 2 is as follows: The multi-axis handling robot target detection system uses the YOLOV5 target detection algorithm to complete the robot's target recognition. The Mosaci data enhancement algorithm is used at the input end of the model, and the training samples are spliced ​​by random scaling, random cropping, and random arrangement. When training the training sample set, the automatic training prediction box and adaptive image scaling algorithms are used. In the feature network that extracts the image features of the training sample set, the Focus algorithm and CSP2 structure are used; in the Neck structure, the FPN+PAN structure is used to enhance the ability of network feature fusion; at the output end of the YOLOV5 network, GIOU_Loss is used as the loss function: Where IoU is the IoU loss, A c is the area of ​​the minimum circumscribed rectangle of the YOLOV5 detection box and the sample annotation, and U is the area of ​​the union of the detection box and the annotation. The present invention uses non-maximum suppression in the screening of the YOLOV5 target box to solve the problem of overlapping detection targets; The YOLOV5 model is trained on the NVIDIA CUDA platform. After the YOLOV5 target detection model is trained, the trained model is pruned and quantized and then transferred to the host computer software. Step 3: In the case of an error in detecting the target, the target detection system will send the incorrectly identified samples as training samples into the detection model, and perform incremental learning on the YOLOV5 target detection model, thereby continuously improving the accuracy of the YOLOV5 target detection model. Step 4, design a control model for a multi-axis handling robot: introduce the current negative feedback signal, rate negative feedback signal and position feedback signal of the motor into the control system of the multi-axis handling robot, design the current feedback closed loop, rate closed loop and position closed loop control of the multi-axis handling robot respectively, and form a complete three-loop control system for the multi-axis handling robot; Step 5, design the control software of the multi-axis handling robot: The multi-axis handling robot visual recognition method software has general functions and a convenient and friendly interface. It is divided into manual and automatic control modes, and takes into account the scheduling, data storage, user interaction interface and communication functions of the multi-axis handling robot control tasks.

2. A multi-axis handling robot visual recognition method according to claim 1, characterized in that: The process of incremental learning of the YOLOV5 target detection model in step 3 can be expressed as follows: Correctly modify the labels of the misclassified image data and use them as training sets. Based on the trained YOLOV5 target detection model, incremental learning is performed on it to obtain updated thresholds and weights. The updated thresholds and weights are then transmitted to the host computer software to update the thresholds and weights of the YOLOV5 target detection model in the software, ultimately achieving model optimization and upgrade.

3. A multi-axis handling robot visual recognition method according to claim 1, characterized in that: The process of designing the control model of the multi-axis handling robot in step 4 is expressed as follows: Each axis system of the multi-axis handling robot can realize multiple operation modes: angular position, angular velocity, swing and other modes to complete the handling of the target; in order to complete the handling action of the multi-axis handling robot, the current loop is designed in the power amplifier of the motor, and the speed loop of the motor is designed to widen the system frequency band, improve the rate response speed, and suppress the influence of motor torque fluctuation and dry friction on the system. Finally, on the basis of the current loop and speed loop, the position loop of the system is added to the periphery to control the robot's motion; In order to reduce the overshoot of the system, the current loop, speed loop and position loop all use integral separation PID control, and the logic function is introduced into the algorithm. The output sampling point value of the regulator is: Among them, u(k) represents the duty cycle of SPWM in the current loop, u(k) represents the current value in the speed loop, u(k) represents the speed value in the position loop, and K p , K i , K d , are the proportional coefficient, integral coefficient and differential coefficient of the PID algorithm, e(j) is the system error at time j, A is the threshold of the PID control system. When the deviation is large, the integral term of PID does not work. When the deviation is within the threshold, the integral algorithm is introduced to control the multi-axis operation of the robot. The parameters of the integral separation PID are then adjusted using a genetic algorithm.

4. A multi-axis handling robot visual recognition method according to claim 3, characterized in that: The genetic algorithm is used to adjust the parameters of the integral separation PID, and the steps are as follows: Step 4.1, fuzzy domain coding; the actual range of the angle position deviation e(t) during the operation of the multi-axis handling robot may reach 0 to 360 degrees, so its domain is set to [0,360]. The basic domain of the angle change rate during the operation of the multi-axis handling robot is roughly [-40,40]. After parameter setting analysis, the output k p , k i , k d , the actual domain is set to [-5,5], [-0.4, 0.4], [-1, 1]; the fuzzy domains of input and output are all set to [-6, -5, -4, -3, 2, -1, 0, 1, 2, 3, 4, 5, 6]; the system sets 7 linguistic variable values ​​of negative large NB, negative medium NM, negative NS, zero ZO, positive small PS, positive medium PM, positive large PB, and uses 0, 1, 2, 3, 4, 5, 6 to encode and represent; Step 4.2, initial population selection: initialize a set of initial current loop, speed loop and position loop PID parameters K by random generation method p , K i , K d As the population, the optimal solution parameter range of the initial population is set to [-6, 6], and an initial population of 80 is randomly generated within the set range; Step 4.3, fitness function selection; the fitness function determines the fitness of the individual genetic algorithm through the robot control performance index, and takes the individual fitness as the quality standard for evaluating the PID parameters. The present invention sets the objective function of the genetic algorithm by the overshoot, steady-state error and response time index of the system, so that the fitness function is described as follows: In the formula, e1 is the steady-state error of the control system, e2 is the overshoot of the control system, and e t is the response time of the control system, ω1, ω2, ω3 are weighted constants; Step 4.4, select genetic operators; according to the fitness value, use the roulette wheel selection method to select a part of excellent individuals as "parents", and use the ratio of the individual fitness value to the overall fitness value in the whole population to determine the probability of being selected. The formula is as follows: Among them, P i is the probability of individual i being genetically selected, f j is the overall fitness value of all individuals superimposed, f i is the fitness value of individual i; Step 4.5, crossover and mutation operation; for the selected "parent" individual, a crossover operation is performed to generate a new individual. The present invention adopts a two-point crossover algorithm and sets the crossover probability to 0.9, thereby forming two new individuals; at the same time, in order to improve the local search capability, some genes of the individual are randomly disturbed or changed, and the mutation probability is set to 0.03; Step 4.6, update the population and determine the termination condition; add the generated new individuals to the original population to obtain the updated population, and determine whether the population meets the termination condition. If not, return to step 4.

4. If so, output the PID closed-loop parameters.

Images