Control method based on PLC pneumatic transfer robot

Abstract

A control method based on a PLC pneumatic transfer robot comprises the steps that firstly, an image of a target object in a working area is collected through a visual module of the pneumatic transfer robot, the image is preprocessed, and a preprocessed image is obtained; 2, the PLC control module carries out edge detection and analysis on the preprocessed image, extracts edge information of the target article, calculates the number of breakpoints, and judges whether the target article is damaged or not according to the number of the breakpoints; 3, further analyzing the preprocessed image and the edge image by using a convolutional neural network under the condition that the judgment cannot be carried out in the step 2, outputting a probability value of appearance damage of the target object, and carrying out comparison and judgment; fourthly, the damaged target object is moved to a damaged area; fifthly, the undamaged target objects are moved to a placement area; and sixthly, the pneumatic carrying robot returns to the working area. Therefore, when the design is used for processing appearance characteristics with non-uniform shapes, the processing efficiency is relatively high, and the accuracy is relatively high.

Description

Technical Field

[0001] The present invention relates to a control method for a robot, belonging to the technical field of robot control, and particularly relates to a control method for a PLC pneumatic handling robot. Background Art

[0002] In the logistics industry, especially in large logistics centers, the sorting and handling of packages are key links in the entire logistics operation. However, there are many problems with existing sorting technologies when dealing with damaged packages, making it difficult to meet the development needs of modern logistics. Currently, most logistics centers rely mainly on manual operations or simple automated equipment for sorting damaged packages. When manually sorting damaged packages, workers need to check the appearance of each package one by one, judge the degree of damage, and then carry and classify them. This operation method is not only inefficient but also greatly affected by subjective factors, and it is difficult to guarantee the accuracy. In addition, due to the different shapes, sizes, and degrees of damage of damaged packages, simple automated equipment has difficulty handling more complex sorting tasks when sorting damaged packages, resulting in low sorting efficiency and accuracy. Therefore, a control method for a PLC pneumatic handling robot is needed to improve work efficiency and accuracy.

[0003] Chinese Patent Application No. 202310114753.4, with an application date of February 15, 2023, discloses an online grading device and method for spherical fruits based on machine vision. An online grading device for spherical fruits based on machine vision includes a conveying device, a guide rod, a photoelectric sensor, an image acquisition device, an HMI unit, a host computer, and a grading device. The host computer presets an online grading detection algorithm for spherical fruits. The industrial computer runs the deployed online grading detection algorithm for spherical fruits, and obtains pose information by processing the image and template matching. The industrial computer controls the SCARA robot to drive the pneumatic fixture to clamp the fruits and place them in the corresponding fruit collection boxes to achieve intelligent online grading of fruits. Although this patent can improve the efficiency and automation of fruit sorting, it still has the following defects:

[0004] This design has low processing efficiency and accuracy when dealing with appearance features with inconsistent shapes using a 3D vision device.

[0005] Disclosing the information in this background art section is only intended to enhance the overall understanding of the present patent application and should not be construed as an admission or any form of suggestion that this information constitutes prior art already known to those of ordinary skill in the art. Summary of the Invention

[0006] The objective of the present invention is to overcome the defects and problems in the prior art, namely, when using a 3D vision device to process appearance features with non-uniform shapes, the processing efficiency is relatively low and the accuracy rate is relatively low. The present invention provides a control method for a PLC pneumatic handling robot with relatively high processing efficiency and accuracy rate when processing appearance features with non-uniform shapes.

[0007] To achieve the above objective, the technical solution of the present invention is: A control method for a PLC pneumatic handling robot, and the control method of the pneumatic handling robot includes the following steps:

[0008] First step: First, collect an image of a target item in the working area through the vision module of the pneumatic handling robot, preprocess the image to obtain a preprocessed image, and then transfer the preprocessed image to the PLC control module of the pneumatic handling robot;

[0009] Second step: The PLC control module performs edge detection on the received preprocessed image, extracts the edge information of the target item and calculates the number of breakpoints to obtain the number of edge breakpoints. The maximum threshold and minimum threshold of the number of breakpoints are preset in the PLC control module, and then the number of edge breakpoints is compared:

[0010] If the number of edge breakpoints is greater than the maximum threshold, it is determined that the appearance of the target item is damaged, and the fourth step is executed;

[0011] If the number of edge breakpoints is less than or equal to the minimum threshold, it is determined that the appearance of the target item is not damaged, and the fifth step is executed;

[0012] If the number of edge breakpoints is between the maximum threshold and the minimum threshold, it is impossible to determine whether the appearance of the target item is damaged, and the third step is executed;

[0013] Third step: Further analyze the edge of the target item with the number of edge breakpoints between the maximum threshold and the minimum threshold through the pre-trained convolutional neural network in the PLC control module, output the probability value that the appearance of the target item is damaged. The preset threshold of the probability value is set in the PLC control module, and then the probability value is compared:

[0014] If the probability value is greater than or equal to the preset threshold, it is determined that the appearance of the target item is damaged, and the fourth step is executed;

[0015] If the probability value is less than the preset threshold, it is determined that the appearance of the target item is not damaged, and the fifth step is executed;

[0016] Step 4: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target item, and at the same time controls the transportation module of the pneumatic handling robot to transport the damaged target item to the damaged area. After reaching the damaged area, it then controls the pneumatic clamping mechanism to release the damaged target item.

[0017] Step 5: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target item, and at the same time controls the transportation module of the pneumatic handling robot to transport the target item to the placement area. After reaching the placement area, it then controls the pneumatic clamping mechanism to release the target item.

[0018] Step 6: The pneumatic handling robot returns to the working area.

[0019] The control method of the pneumatic handling robot further includes:

[0020] Step 7: Repeat Steps 1 to 6 until a stop command is received or an irreparable fault occurs.

[0021] The vision module includes a vision monitoring sensor.

[0022] In the first step, the process of collecting an image of the target item in the working area through the vision module of the pneumatic handling robot and preprocessing the image means that the vision monitoring sensor collects an image of the target item in the working area and preprocesses the collected image. The preprocessing includes, but is not limited to, grayscale conversion, noise reduction, and image enhancement.

[0023] The imaging of the lens of the vision monitoring sensor follows the thin lens imaging law, and its imaging relationship is expressed by the following formula:

[0024]

[0025] Where, u is the object distance, that is, the distance from the object to the lens; v is the image distance, that is, the distance where the image is formed on the sensor; f is the focal length of the lens.

[0026] The magnification m of the imaging is expressed by the following formula:

[0027]

[0028] Through the magnification formula, determine the proportional relationship between the image formed by the vision monitoring sensor and the actual size of the target item.

[0029] In the second step, the edge detection of the preprocessed image received by the PLC control module means that the PLC control module has a built-in Canny edge detection algorithm, and the received preprocessed image is subjected to edge detection through the Canny edge detection algorithm to obtain an edge image;

[0030] In the second step, the extraction of the edge information of the target item and the calculation of the number of breakpoints to obtain the edge breakpoint number means that the edge image obtained by the Canny edge detection algorithm is subjected to contour extraction to identify the edge contour of the target item, and then the distance between adjacent points is checked point by point along the edge contour. If the distance between adjacent points exceeds a preset threshold, it is determined that there is a breakpoint at this position, the detected breakpoints are marked, and all breakpoints are counted to obtain the edge breakpoint number; the preset threshold is dynamically adjusted according to the size of the target item and the image resolution.

[0031] In the third step, the convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer;

[0032] In the third step, the further analysis of the edge of the target item with the edge breakpoint number between the maximum threshold and the minimum threshold by the pre-trained convolutional neural network in the PLC control module means that the preprocessed image and the edge image are used as dual-channel inputs and input into the convolutional neural network model; the input layer receives the dual-channel input data, where the first channel is the preprocessed image and the second channel is the edge image; the convolutional layer performs a convolutional operation on the input dual-channel data to extract high-level features of the image and generate multiple feature maps; the pooling layer downsamples the feature maps output by the convolutional layer; the fully connected layer flattens the feature maps output by the pooling layer into a one-dimensional vector and performs feature mapping through multiple fully connected neurons to further extract features and classify; the output layer outputs the probability value of the existence of damage to the target item through the Softmax activation function.

[0033] The convolutional neural network model is trained through the following steps:

[0034] First, collect an image data set containing damaged and non-damaged target items and annotate the edge regions in the images;

[0035] Second, divide the data set into a training set and a test set, use the training set to train the convolutional neural network model, and verify the accuracy of the model through the test set;

[0036] Finally, optimize the model parameters through the backpropagation algorithm until the model meets the preset accuracy requirements.

[0037] The pneumatic clamping mechanism of the pneumatic handling robot includes a cylinder, a gripper, a control valve, and a sensor system; the gripper is fixedly connected to the cylinder, and the cylinder is connected to the control valve;

[0038] The sensor system includes a pressure sensor and a position sensor;

[0039] The pressure sensor and the position sensor are respectively fixedly connected to the gripper; the pressure sensor and the position sensor are respectively electrically connected to the PLC control module, and the control valve is electrically connected to the PLC control module.

[0040] In the fourth step, when the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target item, it means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the position of the damaged target item, the PLC control module then sends an electrical signal to the control valve to inflate the cylinder, driving the gripper to close to clamp the damaged target item. During this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the gripper on the damaged target item is within the preset pressure range;

[0041] In the fourth step, when controlling the pneumatic clamping mechanism to release the damaged target item, it means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the placement position of the damaged target item, the PLC control module sends an electrical signal to the control valve to deflate the cylinder, driving the gripper to open, and monitors the pressure in the cylinder in real time through the pressure sensor to ensure that the cylinder is completely deflated and the gripper is completely open. At the same time, it monitors the position state of the gripper in real time through the position sensor to confirm that the gripper has completely opened and is separated from the damaged target item;

[0042] In the fifth step, when the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target item, it means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the position of the target item, the PLC control module then sends an electrical signal to the control valve to inflate the cylinder, driving the gripper to close to clamp the target item. During this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the gripper on the target item is within the preset pressure range;

[0043] In the fifth step, the control of the pneumatic clamping mechanism to release the target item means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the placement position of the target item, the PLC control module sends an electrical signal to the control valve to deflate the cylinder, driving the gripper to open. At the same time, the pressure sensor monitors the pressure in the cylinder in real time to ensure that the cylinder is completely deflated and the gripper is fully open. Meanwhile, the position sensor monitors the position state of the gripper in real time to confirm that the gripper is fully open and detached from the target item.

[0044] The transportation module of the pneumatic handling robot includes a base and a stepping motor; the stepping motor is connected to the base through a transmission mechanism;

[0045] The ground of the working area, the damaged area, and the placement area are all provided with connected working chutes;

[0046] The base is slidably connected to the working chute, and the stepping motor is controlled by the PLC control module.

[0047] In the fourth step, the control of the transportation module of the pneumatic handling robot to transport the damaged target item to the damaged area means that: the PLC control module controls the stepping motor to drive the base to move along the working chute from the working area to the damaged area, transporting the damaged target item to the damaged area;

[0048] In the fifth step, the control of the transportation module of the pneumatic handling robot to transport the target item to the placement area means that: the PLC control module controls the stepping motor to drive the base to move along the working chute from the working area to the placement area, transporting the target item to the placement area;

[0049] In the sixth step, the pneumatic handling robot returning to the working area means that: the PLC control module first controls the pneumatic clamping mechanism to return to its original position, and then controls the stepping motor to drive the base to move along the working chute from the placement area or the damaged area to the working area.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] 1. A control method for a PLC pneumatic handling robot according to the present invention. The PLC control module performs edge detection on the preprocessed image collected and preprocessed by the vision module, extracts the edge information of the target item and calculates the number of breakpoints, then compares the number of edge breakpoints. The convolutional neural network further analyzes the edges of the target items whose appearance damage cannot be judged by the number of edge breakpoints. In application, first, the vision module collects and preprocesses the image of the target item (such as an express package) to obtain a preprocessed image. Then, the PLC control module performs edge detection on the preprocessed image to obtain an edge image, and extracts information from the edge image to obtain the number of edge breakpoints. By comparing the number of edge breakpoints of the target item with the preset threshold in the PLC control module, it is initially judged whether the appearance of the target item is damaged. For the target items that cannot be clearly judged by the number of breakpoints, the convolutional neural network in the PLC control module further performs feature extraction and analysis, and outputs the probability value of the existence of damage to the target item, so as to further judge whether the appearance of the target item is damaged. Finally, according to the judgment result, the pneumatic handling robot is controlled to transport the damaged target items to the damaged area and the undamaged target items to the placement area. Since the edge detection algorithm and the convolutional neural network are combined to detect the appearance of the target item, it can not only quickly screen out the obviously damaged items, improve the processing efficiency, but also improve the detection accuracy by deeply analyzing and processing complex features, especially suitable for items with inconsistent shapes, such as express packages. Therefore, the present invention can not only process the appearance features with inconsistent shapes, but also improve the processing efficiency and accuracy of sorting and handling.

[0052] 2. In a control method for a PLC pneumatic handling robot according to the present invention, the preprocessed image and the edge image are used as dual-channel inputs and input into the convolutional neural network model. In application, the dual-channel input combines the global information of the preprocessed image and the local structure information of the edge image, enabling the convolutional neural network to obtain richer and more comprehensive image features, thereby enhancing the expression ability of the features of the target item and further improving the classification accuracy. At the same time, the convolutional neural network extracts high-level features of the image through convolutional layers, pooling layers and fully connected layers and performs classification, and can process items of various shapes and sizes, significantly improving the stability and reliability of the processing results. Therefore, the present invention can not only improve the processing efficiency and accuracy of sorting and handling, but also improve the stability and reliability of processing appearance features with inconsistent shapes.

[0053] 3. In the control method of a PLC pneumatic handling robot according to the present invention, the PLC control module monitors the position state of the gripper in real time through a position sensor. After confirming that the gripper has reached the position of the damaged target item, the PLC control module monitors the pressure in the cylinder in real time through a pressure sensor and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the gripper on the damaged target item is within the preset pressure range. During application, the PLC control module first controls the gripper of the pneumatic clamping mechanism to move to the position of the target item. The position sensor feeds back the position information of the gripper in real time to ensure that the gripper accurately reaches the target position. After reaching the position, the PLC control module accurately controls the inflation or deflation of the cylinder according to the pressure signal fed back by the pressure sensor, adjusts the clamping force of the gripper, and keeps it within the preset pressure range to ensure that the gripper can stably and safely clamp the damaged target item and avoid damage or slipping of the item caused by excessive or insufficient clamping force. Due to the combined use of the position sensor and the pressure sensor, precise control of the position and clamping force of the gripper is achieved, ensuring the accuracy of the gripper during the handling process, improving the stability and reliability of the handling process. At the same time, it is also applicable to the clamping and handling of target items with different sizes and shapes. Therefore, the present invention can not only improve the stability and reliability, but also improve the handling accuracy and universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a schematic flowchart of the control method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0056] See Figure 1 , a control method of a PLC pneumatic handling robot, the control method of the pneumatic handling robot includes the following steps:

[0057] The first step: First, collect the image of the target item in the working area through the vision module of the pneumatic handling robot, preprocess the image to obtain a preprocessed image, and then transmit the preprocessed image to the PLC control module of the pneumatic handling robot;

[0058] The second step: The PLC control module performs edge detection on the received preprocessed image, extracts the edge information of the target item and calculates the number of breakpoints to obtain the number of edge breakpoints. The maximum threshold and minimum threshold of the number of breakpoints are preset in the PLC control module, and then the number of edge breakpoints is compared:

[0059] If the number of edge breakpoints is greater than the maximum threshold, it is determined that the appearance of the target item is damaged, and the fourth step is executed;

[0060] If the number of edge breakpoints is less than or equal to the minimum threshold, it is determined that the appearance of the target item is not damaged, and the fifth step is executed;

[0061] If the number of edge breakpoints is between the maximum threshold and the minimum threshold, it cannot be determined whether the appearance of the target item is damaged, and the third step is executed;

[0062] Third step: Further analyze the edge of the target item whose number of edge breakpoints is between the maximum threshold and the minimum threshold through the pre-trained convolutional neural network in the PLC control module, and output the probability value that the appearance of the target item is damaged. A preset threshold of the probability value is set in the PLC control module, and then the probability values are compared:

[0063] If the probability value is greater than or equal to the preset threshold, it is determined that the appearance of the target item is damaged, and the fourth step is executed;

[0064] If the probability value is less than the preset threshold, it is determined that the appearance of the target item is not damaged, and the fifth step is executed;

[0065] Fourth step: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target item, and at the same time controls the transportation module of the pneumatic handling robot to transport the damaged target item to the damaged area. After reaching the damaged area, it then controls the pneumatic clamping mechanism to release the damaged target item;

[0066] Fifth step: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target item, and at the same time controls the transportation module of the pneumatic handling robot to transport the target item to the placement area. After reaching the placement area, it then controls the pneumatic clamping mechanism to release the target item;

[0067] Sixth step: The pneumatic handling robot returns to the working area.

[0068] The control method of the pneumatic handling robot further includes:

[0069] Seventh step: Loop through the first step to the sixth step until a stop command is received or an irrecoverable fault occurs.

[0070] The vision module includes a vision monitoring sensor;

[0071] In the first step, the process of collecting an image of the target item in the working area through the vision module of the pneumatic handling robot and preprocessing the image means that the vision monitoring sensor collects an image of the target item in the working area and preprocesses the collected image. The preprocessing includes, but is not limited to, grayscale conversion, noise reduction, and image enhancement;

[0072] The lens imaging of the visual monitoring sensor follows the thin lens imaging law, and its imaging relationship is expressed by the following formula:

[0073]

[0074] where u is the object distance, i.e., the distance from the object to the lens; v is the image distance, i.e., the distance where the image is formed on the sensor; f is the focal length of the lens;

[0075] The magnification m of the imaging is expressed by the following formula:

[0076]

[0077] Through the magnification formula, the proportional relationship between the image formed by the visual monitoring sensor and the actual size of the target item is determined.

[0078] In the second step, the edge detection of the preprocessed image received by the PLC control module means that: the PLC control module has a Canny edge detection algorithm built-in, and the received preprocessed image is subjected to edge detection through the Canny edge detection algorithm to obtain an edge image;

[0079] In the second step, the extraction of the edge information of the target item and the calculation of the number of breakpoints to obtain the number of edge breakpoints means that: the contour of the edge image obtained by the Canny edge detection algorithm is extracted to identify the edge contour of the target item, and then the distance between adjacent points is checked point by point along the edge contour. If the distance between adjacent points exceeds a preset threshold, it is determined that there is a breakpoint at this position, the detected breakpoints are marked, and all breakpoints are counted to obtain the number of edge breakpoints; the preset threshold is dynamically adjusted according to the size of the target item and the image resolution.

[0080] In the third step, the convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer;

[0081] In the third step, the further analysis of the edges of the target item with the number of edge breakpoints between the maximum threshold and the minimum threshold by the pre-trained convolutional neural network in the PLC control module means: taking the preprocessed image and the edge image as dual-channel inputs and inputting them into the convolutional neural network model; the input layer receives the dual-channel input data, where the first channel is the preprocessed image and the second channel is the edge image; the convolutional layer performs convolutional operations on the input dual-channel data to extract high-level features of the image and generate multiple feature maps; the pooling layer downsamples the feature maps output by the convolutional layer; the fully connected layer flattens the feature maps output by the pooling layer into a one-dimensional vector and performs feature mapping through multiple fully connected neurons to further extract features and classify them; the output layer outputs the probability value of the existence of damage to the target item through the Softmax activation function.

[0082] The convolutional neural network model is trained through the following steps:

[0083] First, collect an image data set containing damaged and non-damaged target items and label the edge regions in the images;

[0084] Second, divide the data set into a training set and a test set, use the training set to train the convolutional neural network model, and verify the accuracy of the model through the test set;

[0085] Finally, optimize the model parameters through the backpropagation algorithm until the model meets the preset accuracy requirements.

[0086] The pneumatic clamping mechanism of the pneumatic handling robot includes a cylinder, a gripper, a control valve and a sensor system; the gripper is fixedly connected to the cylinder, and the cylinder is connected to the control valve;

[0087] The sensor system includes a pressure sensor and a position sensor;

[0088] The pressure sensor and the position sensor are respectively fixedly connected to the gripper; the pressure sensor and the position sensor are respectively electrically connected to the PLC control module, and the control valve is electrically connected to the PLC control module.

[0089] In the fourth step, the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target item, which means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the position of the damaged target item, the PLC control module then sends an electrical signal to the control valve to inflate the cylinder, driving the gripper to close to clamp the damaged target item. During this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the gripper on the damaged target item is within the preset pressure range;

[0090] In the fourth step, the control of the pneumatic clamping mechanism to release the damaged target item means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the placement position of the damaged target item, the PLC control module sends an electrical signal to the control valve to deflate the cylinder, driving the gripper to open, and monitors the pressure in the cylinder in real time through the pressure sensor to ensure that the cylinder is completely deflated and the gripper is fully open. At the same time, it monitors the position state of the gripper in real time through the position sensor to confirm that the gripper has fully opened and is separated from the damaged target item;

[0091] In the fifth step, the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target item, which means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the position of the target item, the PLC control module then sends an electrical signal to the control valve to inflate the cylinder, driving the gripper to close to clamp the target item. During this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the gripper on the target item is within the preset pressure range;

[0092] In the fifth step, the control of the pneumatic clamping mechanism to release the target item means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor. After confirming that the gripper has reached the placement position of the target item, the PLC control module sends an electrical signal to the control valve to deflate the cylinder, driving the gripper to open, and monitors the pressure in the cylinder in real time through the pressure sensor to ensure that the cylinder is completely deflated and the gripper is fully open. At the same time, it monitors the position state of the gripper in real time through the position sensor to confirm that the gripper has fully opened and is separated from the target item.

[0093] The transportation module of the pneumatic handling robot includes a base and a stepping motor; the stepping motor is connected to the base through a transmission mechanism;

[0094] The ground of the working area, the damaged area, and the placement area are all provided with connected working chutes;

[0095] The base is slidably connected to the working chute, and the stepping motor is controlled by a PLC control module.

[0096] In the fourth step, when the transportation module of the pneumatic handling robot transports the damaged target item to the damaged area, it means that: the PLC control module controls the stepping motor to drive the base to move from the working area to the damaged area along the working chute, and transports the damaged target item to the damaged area;

[0097] In the fifth step, when the transportation module of the pneumatic handling robot transports the target item to the placement area, it means that: the PLC control module controls the stepping motor to drive the base to move from the working area to the placement area along the working chute, and transports the target item to the placement area;

[0098] In the sixth step, when the pneumatic handling robot returns to the working area, it means that: the PLC control module first controls the pneumatic clamping mechanism to return to its original position, and then controls the stepping motor to drive the base to move from the placement area or the damaged area to the working area along the working chute.

[0099] The supplementary description of the present invention is as follows:

[0100] Preferably, the present invention turns on the power switch to make the pneumatic handling robot enter the working state, and the switch includes but is not limited to a physical switch component and a remote control component.

[0101] Preferably, in the present invention, the stepping motor is connected to the base through a transmission mechanism, and the transmission mechanism is at least one of a gear transmission, a belt transmission, or a screw transmission.

[0102] Preferably, the present invention uses the Canny edge detection algorithm to analyze the continuity of each extracted contour through contour analysis, including the connectivity check of contour points (checking each point along the contour one by one to determine whether the distance between adjacent points exceeds a certain threshold. If the distance is too large, it is considered that there is a breakpoint); the smoothness analysis of the contour (fitting the contour using curve fitting and then checking the discontinuous points of the fitted curve); the geometric feature analysis of the contour (calculating the curvature change of the contour. If the curvature changes suddenly, it may be a breakpoint).

[0103] Preferably, the calculation method for the number of edge breakpoints in the present invention is: first, mark the breakpoints (recording the position of each detected breakpoint), and then count the number of breakpoints (counting the number of breakpoints detected in all contours to obtain the final total number of breakpoints).

[0104] Preferably, the convolutional layer of the present invention includes a plurality of convolutional kernels, each convolutional kernel having a size of 3x3, a stride of 1, and using the ReLU activation function.

[0105] Preferably, the present invention employs a max pooling operation with a pooling kernel size of 2x2 and a stride of 2.

[0106] Preferably, the output layer of the present invention outputs the probability value of the target item being damaged through the Softmax activation function, and the range of the probability value is from 0 to 1.

[0107] Preferably, the training process of the convolutional neural network of the present invention includes: First, use the labeled image dataset to train the convolutional neural network, and the dataset includes images of damaged and non-damaged target items; Second, during the training process, use the cross-entropy loss function to calculate the error between the predicted value and the true label; Third, update the parameters of the convolutional neural network through the Adam optimization algorithm until the loss function converges; Finally, deploy the trained convolutional neural network model to the PLC control module for real-time analysis of the edge information of the target item.

[0108] Preferably, a soft buffer pad is provided on one side of the gripper to increase the friction with the target item and reduce the damage to the target item.

[0109] Example 1:

[0110] See Figure 1 , a control method for a PLC pneumatic handling robot, and the control method of the pneumatic handling robot includes the following steps:

[0111] The first step: First, collect the image of the target item in the working area through the vision module of the pneumatic handling robot, preprocess the image to obtain a preprocessed image, and then transfer the preprocessed image to the PLC control module of the pneumatic handling robot;

[0112] The second step: The PLC control module performs edge detection on the received preprocessed image, extracts the edge information of the target item and calculates the number of breakpoints to obtain the edge breakpoint number. The maximum threshold and minimum threshold of the breakpoint number are preset in the PLC control module, and then the edge breakpoint number is compared:

[0113] If the number of edge breakpoints is greater than the maximum threshold, it is determined that the appearance of the target item is damaged, and the fourth step is executed;

[0114] If the number of edge breakpoints is less than or equal to the minimum threshold, it is determined that the appearance of the target item is not damaged, and the fifth step is executed;

[0115] If the number of edge breakpoints is between the maximum threshold and the minimum threshold, it is impossible to determine whether there is damage to the appearance of the target item, and the third step is executed;

[0116] Third step: Further analyze the edges of the target item whose number of edge breakpoints is between the maximum threshold and the minimum threshold through the pre-trained convolutional neural network in the PLC control module, and output the probability value that there is damage to the appearance of the target item. A preset threshold of the probability value is set in the PLC control module, and then the probability values are compared:

[0117] If the probability value is greater than or equal to the preset threshold, it is determined that there is damage to the appearance of the target item, and the fourth step is executed;

[0118] If the probability value is less than the preset threshold, it is determined that there is no damage to the appearance of the target item, and the fifth step is executed;

[0119] Fourth step: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target item, and at the same time controls the transportation module of the pneumatic handling robot to transport the damaged target item to the damaged area. After reaching the damaged area, it then controls the pneumatic clamping mechanism to release the damaged target item;

[0120] Fifth step: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target item, and at the same time controls the transportation module of the pneumatic handling robot to transport the target item to the placement area. After reaching the placement area, it then controls the pneumatic clamping mechanism to release the target item;

[0121] Sixth step: The pneumatic handling robot returns to the working area.

[0122] In application, the edge detection algorithm can quickly extract the edge information of the preprocessed image, calculate the number of breakpoints, and initially judge whether there are obvious damage features on the target item. It has the characteristics of fast processing speed and low consumption of computing resources, and is suitable for the recognition of simple and regular damage features; while the convolutional neural network extracts high-level features of the image through convolutional layers, pooling layers and fully connected layers, can process complex image features, adapt to items of various shapes and sizes, has high accuracy and generalization ability, but the processing speed is relatively slow and the consumption of computing resources is relatively high; by combining these two methods, not only can obvious damaged items be quickly screened out to improve the processing efficiency, but also complex features can be processed through in-depth analysis to improve the accuracy of detection, especially suitable for items with inconsistent shapes such as express packages.

[0123] Example 2:

[0124] The basic content is the same as that of Embodiment 1, except that: the control method of the pneumatic handling robot further includes: Step 7: Repeatedly execute Step 1 to Step 6 until a stop command is received or an irrecoverable fault occurs.

[0125] During application, by repeatedly executing Step 1 to Step 6 in Step 7, automated and continuous sorting and handling operations can be achieved, significantly improving the working efficiency and automation level of the logistics center.

[0126] Embodiment 3:

[0127] The basic content is the same as that of Embodiment 1, except that: the vision module includes a vision monitoring sensor; in Step 1, collecting an image of the target item in the working area through the vision module of the pneumatic handling robot and preprocessing the image means: the vision monitoring sensor collects an image of the target item in the working area and preprocesses the collected image, and the preprocessing includes but is not limited to grayscale conversion, noise reduction, and image enhancement; the lens imaging of the vision monitoring sensor follows the thin lens imaging law, and its imaging relationship is expressed by the following formula:

[0128]

[0129] where, u is the object distance, that is, the distance from the object to the lens; v is the image distance, that is, the distance where the image is formed on the sensor; f is the focal length of the lens;

[0130] The magnification m of the imaging is expressed by the following formula:

[0131]

[0132] Through the magnification formula, determine the proportional relationship between the image formed by the vision monitoring sensor and the actual size of the target item.

[0133] During application, the vision monitoring sensor first collects an image of the target item in the working area to obtain the original image of the target item, and then preprocesses the collected image, specifically including operations such as grayscale conversion, noise reduction, and image enhancement, to improve the image quality and facilitate subsequent edge detection and feature extraction; convert the color image to a grayscale image through grayscale conversion to reduce the amount of data; the noise reduction operation removes the noise in the image to make the image clearer; image enhancement highlights the important features in the image to improve the recognizability of the image; at the same time, according to the thin lens imaging law, calculate the object distance, image distance, and lens focal length of the target item through the imaging relationship formula, determine the magnification of the imaging, so as to accurately obtain the proportional relationship between the actual size of the target item and the imaging size in the image, providing a basis for subsequent precise control and operation.

[0134] Embodiment 4:

[0135] The basic content is the same as that of Embodiment 1, except that: the visual monitoring sensor uses a sensor with infrared recognition function.

[0136] During application, the visual monitoring sensor with infrared recognition function can collect visible light images and infrared images of the target item simultaneously; during the image collection process, the infrared recognition function can enhance the detection ability of the target item under low light conditions; at the same time, this infrared recognition function can stop working in time when a person is detected approaching to avoid accidental injury, thus improving safety.

[0137] Embodiment 5:

[0138] The basic content is the same as that of Embodiment 1, except that: the pneumatic clamping mechanism cylinder, gripper, control valve and sensor system of the pneumatic handling robot; the gripper is fixedly connected to the cylinder, and the cylinder is connected to the control valve; the sensor system includes a pressure sensor and a position sensor; the pressure sensor and the position sensor are respectively fixedly connected to the gripper; the pressure sensor and the position sensor are respectively electrically connected to the PLC control module, and the control valve is electrically connected to the PLC control module; in the fourth step, the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target item, which means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor, and after confirming that the gripper has reached the position of the damaged target item, the PLC control module then sends an electrical signal to the control valve to inflate the cylinder and drive the gripper to close to clamp the damaged target item. During this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the gripper on the damaged target item is within the preset pressure range; in the fourth step, controlling the pneumatic clamping mechanism to release the damaged target item means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor, and after confirming that the gripper has reached the placement position of the damaged target item, the PLC control module sends an electrical signal to the control valve to deflate the cylinder, drive the gripper to open, and monitors the pressure in the cylinder in real time through the pressure sensor to ensure that the cylinder is completely deflated and the gripper is completely open. At the same time, the position state of the gripper is monitored in real time through the position sensor to confirm that the gripper has completely opened and separated from the damaged target item; in the fifth step, the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target item, which means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor, and after confirming that the gripper has reached the position of the target item, the PLC control module then sends an electrical signal to the control valve to inflate the cylinder and drive the gripper to close to clamp the target item. During this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the gripper on the target item is within the preset pressure range; in the fifth step, controlling the pneumatic clamping mechanism to release the target item means that: the PLC control module first monitors the position state of the gripper in real time through the position sensor, and after confirming that the gripper has reached the placement position of the target item, the PLC control module sends an electrical signal to the control valve to deflate the cylinder, drive the gripper to open, and monitors the pressure in the cylinder in real time through the pressure sensor to ensure that the cylinder is completely deflated and the gripper is completely open. At the same time, the position state of the gripper is monitored in real time through the position sensor to confirm that the gripper has completely opened and separated from the target item.

[0139] During application, when clamping a damaged or undamaged target item, the PLC control module first controls the jaws of the pneumatic clamping mechanism to move to the position where the damaged or undamaged target item is located. The position sensor real-time feedbacks the position information of the jaws to ensure that the jaws accurately reach the target position. After reaching the position, the PLC control module precisely controls the inflation or deflation of the cylinder according to the pressure signal feedback by the pressure sensor, adjusts the clamping force of the jaws, and keeps it within the preset pressure range to ensure that the jaws can stably and safely clamp the damaged or undamaged target item, avoiding damage or slipping of the item caused by excessive or insufficient clamping force. Due to the combined use of the position sensor and the pressure sensor, precise control of the jaw position and clamping force is achieved, ensuring the accuracy of the jaws during the handling process, improving the stability and reliability of the handling process. At the same time, it is also applicable to the clamping and handling of target items with different sizes and shapes.

[0140] Embodiment 6:

[0141] The basic content is the same as that of Embodiment 1, with the differences being: The transportation module of the pneumatic handling robot includes a base and a stepping motor; the stepping motor is connected to the base through a transmission mechanism; The ground of the working area, the damaged area, and the placement area are all provided with connected working chutes; The base is slidably connected to the working chute, and the stepping motor is controlled by the PLC control module; In the fourth step, controlling the transportation module of the pneumatic handling robot to transport the damaged target item to the damaged area means that the PLC control module controls the stepping motor to drive the base to move from the working area to the damaged area along the working chute, and transports the damaged target item to the damaged area; In the fifth step, controlling the transportation module of the pneumatic handling robot to transport the target item to the placement area means that the PLC control module controls the stepping motor to drive the base to move from the working area to the placement area along the working chute, and transports the target item to the placement area; In the sixth step, the pneumatic handling robot returning to the working area means that the PLC control module first controls the pneumatic clamping mechanism to return to its original position, and then controls the stepping motor to drive the base to move from the placement area or the damaged area to the working area along the working chute.

[0142] During application, after the pneumatic handling robot completes the appearance detection and identification of the target item in the working area, the PLC control module determines the status of the target item based on the detection result. If the target item is identified as damaged, the PLC control module will start the stepping motor to drive the base to move along the working chute from the working area to the damaged area, and accurately place the damaged target item at the designated position in the damaged area. If the target item is not identified as damaged, the PLC control module controls the stepping motor to drive the base to move along the working chute from the working area to the placement area, and place the target item at the designated position in the placement area. After completing the handling task of the item, the PLC control module first controls the pneumatic clamping mechanism to return to its original position to ensure that the clamping mechanism is in a safe state, and then starts the stepping motor again to drive the base to return from the placement area or the damaged area to the working area along the working chute, preparing for the next handling task. In this way, the pneumatic handling robot can efficiently and accurately handle the target item between different working areas, realizing the functions of automatic sorting and handling.

[0143] Embodiment 7:

[0144] The basic content is the same as that of Embodiment 1, except that: the PLC-based pneumatic handling robot further includes an acoustic-optic system, the acoustic-optic system is electrically connected to the PLC control module, and the acoustic-optic system includes a warning light and a speaker; when a damaged target item is identified, the PLC control module controls the warning light and the speaker to work respectively.

[0145] During application, after the PLC control module detects that the target item is damaged, it simultaneously sends a control signal to the acoustic-optic system to control the warning light to work in a set mode (such as flashing or constant on), emitting an obvious optical signal to remind the operator or on-site personnel of the abnormal situation. At the same time, it controls the speaker to emit a preset sound signal (such as an alarm sound or a voice prompt), further reminding the relevant personnel of the information that the target item is damaged, so as to take corresponding measures in a timely manner. Through the coordinated work of the acoustic-optic system, the information of the damaged item can be quickly and effectively transmitted to the relevant personnel, ensuring that the operator can discover and handle the abnormal situation in a timely manner, thereby improving safety and reliability.

[0146] The above description is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. Any equivalent modification or change made by those of ordinary skill in the art according to the disclosed content of the present invention shall be included in the protection scope recorded in the claims.

Claims

1. A control method for a PLC-based pneumatic handling robot, characterized in that: Control method of the pneumatic handling robot The following steps are involved: The first step: firstly, the image of the target object in the working area is collected by the visual module of the pneumatic handling robot, and the image is preprocessed to obtain a preprocessed image, and then the preprocessed image is transmitted to the PLC control module of the pneumatic handling robot; Step 2: The PLC control module performs edge detection on the received pre-processed image, extracts edge information of the target object and calculates the number of breakpoints to obtain the number of edge breakpoints. The PLC control module has a preset maximum threshold and a minimum threshold for the number of breakpoints, and then compares the number of edge breakpoints: If the number of edge breakpoints is greater than the maximum threshold, it is determined that the appearance of the target object is damaged, and the fourth step is executed; If the number of edge breakpoints is less than or equal to the minimum threshold, it is determined that the appearance of the target object is not damaged, and the fifth step is executed; If the number of edge breakpoints is between the maximum threshold and the minimum threshold, it is impossible to determine whether the appearance of the target object is damaged, and the third step is executed; Step 3: Further analyze the edge of the target object whose edge breakpoint number is between the maximum threshold and the minimum threshold through the pre-trained convolutional neural network in the PLC control module, and output the probability value of the target object's appearance being damaged. The PLC control module is provided with a preset threshold of the probability value, and then compare the probability values: If the probability value is greater than or equal to the preset threshold, it is determined that the appearance of the target object is damaged, and the fourth step is executed; If the probability value is less than the preset threshold, it is determined that the appearance of the target object is not damaged, and the fifth step is executed; Step 4: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target object, and at the same time controls the transport module of the pneumatic handling robot to transport the damaged target object to the damaged area. After arriving at the damaged area, the pneumatic clamping mechanism is controlled to release the damaged target object. Step 5: The PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target object, and at the same time controls the transport module of the pneumatic handling robot to transport the target object to the placement area. After arriving at the placement area, the pneumatic clamping mechanism is controlled to release the target object. Step 6: The pneumatic transport robot returns to the working area.

2. A control method based on a PLC pneumatic handling robot according to claim 1, characterized in that: The control method of the pneumatic handling robot also includes: Step 7: Loop through steps 1 to 6 until a stop command is received or an unrecoverable fault occurs.

3. A control method based on a PLC pneumatic handling robot according to claim 1 or 2, characterized in that: The visual module includes a visual monitoring sensor; In the first step, the collecting of images of target objects in the working area by the visual module of the pneumatic handling robot and preprocessing the images means that the visual monitoring sensor collects images of target objects in the working area and preprocesses the collected images, wherein the preprocessing includes but is not limited to graying, noise reduction and image enhancement; The lens imaging of the visual monitoring sensor follows the thin lens imaging law, and its imaging relationship is expressed by the following formula: Among them, u is the object distance, that is, the distance from the object to the lens; v is the image distance, that is, the distance of the image on the sensor; f is the focal length of the lens; The imaging magnification m is expressed by the following formula: The magnification formula is used to determine the proportional relationship between the image formed by the visual monitoring sensor and the actual size of the target object.

4. A control method based on a PLC pneumatic handling robot according to claim 1 or 2, characterized in that: In the second step, the PLC control module performs edge detection on the received pre-processed image, which means: the PLC control module has a built-in Canny edge detection algorithm, and performs edge detection on the received pre-processed image by using the Canny edge detection algorithm to obtain an edge image; In the second step, the edge information of the target object is extracted and the number of breakpoints is calculated to obtain the number of edge breakpoints, which means: contour extraction is performed on the edge image obtained by the Canny edge detection algorithm to identify the edge contour of the target object, and then the distance between adjacent points is checked point by point along the edge contour. If the distance between adjacent points exceeds a preset threshold, it is determined that there is a breakpoint at the position, the detected breakpoint is marked, and all breakpoints are counted to obtain the number of edge breakpoints; the preset threshold is dynamically adjusted according to the size of the target object and the image resolution.

5. A control method for a PLC-based pneumatic handling robot according to claim 4, characterized in that: In the third step, the convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer; In the third step, further analyzing the edge of the target object whose number of edge breakpoints is between the maximum threshold and the minimum threshold by the convolutional neural network pre-trained in the PLC control module refers to: inputting the preprocessed image and the edge image as dual-channel input into the convolutional neural network model; the input layer receives dual-channel input data, wherein the first channel is the preprocessed image and the second channel is the edge image; the convolution layer performs a convolution operation on the input dual-channel data, extracts high-level features of the image, and generates multiple feature maps; the pooling layer downsamples the feature map output by the convolution layer; the fully connected layer flattens the feature map output by the pooling layer into a one-dimensional vector, and performs feature mapping through multiple fully connected neurons to further extract features and classify; the output layer outputs the probability value of the target object being damaged through the Softmax activation function.

6. A control method for a PLC-based pneumatic handling robot according to claim 5, characterized in that: The convolutional neural network model is trained by the following steps: First, we collect image datasets containing damaged and non-damaged target objects and annotate the edge regions in the images. Secondly, the data set is divided into a training set and a test set, the convolutional neural network model is trained using the training set, and the accuracy of the model is verified using the test set; Finally, the model parameters are optimized through the back-propagation algorithm until the model reaches the preset accuracy requirements.

7. A control method based on a PLC pneumatic handling robot according to claim 1 or 2, characterized in that: The pneumatic clamping mechanism of the pneumatic handling robot includes a cylinder, a clamping claw, a control valve and a sensor system; the clamping claw is fixedly connected to the cylinder, and the cylinder is connected to the control valve; The sensor system includes a pressure sensor and a position sensor; The pressure sensor and the position sensor are fixedly connected to the clamping jaws respectively; the pressure sensor and the position sensor are electrically connected to the PLC control module respectively, and the control valve is electrically connected to the PLC control module.

8. The control method of a PLC-based pneumatic handling robot according to claim 7, characterized in that: In the fourth step, the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the damaged target object, which means: the PLC control module first monitors the position state of the clamping claw in real time through the position sensor, and after confirming that the clamping claw has reached the position of the damaged target object, the PLC control module sends an electrical signal to the control valve to inflate the cylinder and drive the clamping claw to close to clamp the damaged target object. In this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor, and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the clamping claw on the damaged target object is within a preset pressure range; In the fourth step, the control of the pneumatic clamping mechanism to release the damaged target object means that: the PLC control module first monitors the position state of the clamping jaws in real time through the position sensor, and after confirming that the clamping jaws have reached the placement position of the damaged target object, the PLC control module sends an electrical signal to the control valve to deflate the cylinder and drive the clamping jaws to open, and monitors the pressure in the cylinder in real time through the pressure sensor to ensure that the cylinder is completely deflated and the clamping jaws are fully opened, and at the same time monitors the position state of the clamping jaws in real time through the position sensor to confirm that the clamping jaws are fully opened and separated from the damaged target object; In the fifth step, the PLC control module first controls the pneumatic clamping mechanism of the pneumatic handling robot to clamp the target object, which means that: the PLC control module first monitors the position state of the clamping claw in real time through the position sensor, and after confirming that the clamping claw has reached the position of the target object, the PLC control module sends an electrical signal to the control valve to inflate the cylinder and drive the clamping claw to close to clamp the target object. In this process, the PLC control module monitors the pressure in the cylinder in real time through the pressure sensor, and adjusts the inflation or deflation of the cylinder according to the pressure signal to ensure that the clamping force of the clamping claw on the target object is within a preset pressure range; In the fifth step, controlling the pneumatic clamping mechanism to release the target object means that: the PLC control module first monitors the position status of the clamping jaws in real time through the position sensor, and after confirming that the clamping jaws have reached the placement position of the target object, the PLC control module sends an electrical signal to the control valve to deflate the cylinder and drive the clamping jaws to open, and monitors the pressure in the cylinder in real time through the pressure sensor to ensure that the cylinder is completely deflated and the clamping jaws are fully opened, and at the same time monitors the position status of the clamping jaws in real time through the position sensor to confirm that the clamping jaws are fully opened and detached from the target object.

9. A control method based on a PLC pneumatic handling robot according to claim 1 or 2, characterized in that: The transport module of the pneumatic transport robot includes a base and a stepper motor; the stepper motor is connected to the base through a transmission mechanism; The floors of the working area, the damaged area, and the placement area are all provided with connected working slides; The base is slidably connected to the working slide, and the stepper motor is controlled by a PLC control module.

10. A control method for a PLC-based pneumatic handling robot according to claim 9, characterized in that: In the fourth step, the transport module of the pneumatic transport robot is controlled to transport the damaged target object to the damaged area, which means that the PLC control module controls the stepper motor to drive the base to move from the working area to the damaged area along the working slide, so as to transport the damaged target object to the damaged area; In the fifth step, the transport module of the pneumatic transport robot is controlled to transport the target object to the placement area, which means that the PLC control module controls the stepper motor to drive the base to move along the working slide from the working area to the placement area to transport the target object to the placement area; In the sixth step, the pneumatic transport robot returns to the working area, which means that the PLC control module first controls the pneumatic clamping mechanism to return to its original position, and then controls the stepper motor to drive the base to move along the working slide from the placement area or the damaged area to the working area.

Images