Sock picking method based on visual recognition

Abstract

The present application relates to a kind of automatic production methods of socks, especially a kind of sock pickup method based on visual identification, including material preparation, identification, sock taking and the like steps, by using the way of visual identification to determine the position of the midpoint of the mouth of sock, compared with the way of traditional visual identification sock contour, it is relatively small to be influenced by sock mutual entanglement, relatively high recognition accuracy, while picking up socks, first use vacuum suction tube to lift socks up again, and then the mouth of sock is clamped, both can avoid clamping more than two socks at the same time, and enough clamping force can be provided to pull out socks from sock pile, can replace manual realization to pick up socks in disorder, and further improve sock production efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of automatic production method of socks, and particularly to a kind of sock picking method based on visual identification. BACKGROUND

[0002] After knitting, the knitted socks are usually placed in a disordered manner in a frame or a bag, and the subsequent processes need to pick up the socks one by one for flattening, folding or packaging, etc. In the traditional production, the above picking action is usually completed by manual work, and the production efficiency is relatively low.

[0003] With the development of automatic production technology, the application of mechanical hand clamping device in sock production is also more and more, however, the existing mechanical hand clamping device can usually only pick up the socks which have been arranged one by one, and it is difficult to pick up the socks which are placed in a disordered manner in a frame or a bag, which is mainly because the socks are soft, and it is inevitable to be intertwined with each other without stacking, it is difficult to identify the outline of the whole sock through the camera, so that the mechanical hand device is difficult to determine the placement position of the sock, in addition, even if the position of the sock in the sock pile is determined, it is easy to clamp two or more socks together when picking up. Therefore, for the socks which are woven and placed in a disordered manner in a frame or a bag, manual picking is still needed, and the production efficiency is relatively low.

[0004] Therefore, the present application has made a deep research on the above problems, and the present application is produced. SUMMARY

[0005] The purpose of the present application is to provide a sock picking method based on visual identification which helps to improve the production efficiency of socks.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] A sock picking method based on visual identification, comprising the following steps:

[0008] S1, preparing materials, placing socks in a first working area;

[0009] S2, identifying, using a depth camera to take a picture of the first working area where the socks are placed to obtain a shooting image, and obtaining the midpoint position information of the center point of the cuff of all socks from the shooting image;

[0010] S3, taking socks, according to the midpoint position information, the corresponding socks are clamped one by one to a second working area, when clamping the socks, first use a vacuum suction tube to adsorb the socks at the center point position of the cuff of the socks according to the corresponding position information, and then lift the first distance, then use the clamping jaw to clamp the cuff of the socks, clamp the socks and place them in the second working area.

[0011] As an improvement of the present application, the vacuum suction tube and the gripper are installed at the end of the same robot, and the vacuum suction tube is located between the two clamping rods of the gripper.

[0012] As an improvement of the present application, in step S3, before the socks are clamped, each of the midpoint position information is sorted according to the depth, the area occupied in the captured image and the Euclidean distance between the depth camera origin, and then the corresponding sock is clamped to the second working area according to the midpoint position information.

[0013] As an improvement of the present application, in step S1, the socks are spread on the conveying section of the belt conveyor, and the conveying section passes through the first working area.

[0014] As an improvement of the present application, in step S3, after the gripper clamps the cuff of the sock, it first moves a second distance in a direction away from the second working area, and then moves to the second working area to perform the placing action.

[0015] As an improvement of the present application, in step S3, when clamping the sock, the vacuum suction tube is lifted upward while detecting whether the corresponding sock cuff is adsorbed, and if so, it continues to lift upward, otherwise it gives up clamping the sock.

[0016] As an improvement of the present application, in step S2, the method for obtaining the position information of the center point of the cuff of all socks from the captured image includes the following steps:

[0017] S2.1, train an R-CNN model for identifying the center point of the cuff of the sock, and obtain a pre-trained model;

[0018] S2.2, perform ROI processing on the captured image to remove the edge non-sock area and obtain an effective image;

[0019] S2.3, input the effective image into the pre-trained model to obtain the midpoint position information.

[0020] As an improvement of the present application, in step S3, first determine whether the corresponding midpoint position is located within the working range of the vacuum suction tube according to each of the midpoint position information, if yes, retain the corresponding midpoint position information, otherwise delete the corresponding midpoint position information, and then clamp the corresponding sock to the second working area according to the midpoint position information.

[0021] As an improvement of the application, the training method of the R-CNN model comprises: determining a rosette pattern with at least three corner points according to a training image input into the R-CNN model, and then determining the midpoint position of the rosette pattern according to each corner point of the rosette pattern and outputting.

[0022] By adopting the above scheme, the application has the following beneficial effects:

[0023] By using the visual recognition rosette midpoint position method to determine the position of the sock, compared with the traditional visual recognition sock contour method, the influence of the mutual entanglement of the socks is relatively small, and the recognition accuracy is relatively high. At the same time, when picking up the socks, the rosette of the sock is first lifted upward by the vacuum suction pipe and then clamped, which can avoid clamping more than two socks at the same time, and can provide sufficient clamping force to pull the sock out of the sock pile, which can replace manual picking of the socks in disorder, thereby improving the production efficiency of the socks. DETAILED DESCRIPTION

[0024] The application will be further described below in conjunction with specific embodiments.

[0025] The embodiment provides a sock picking method based on visual recognition, which comprises the following steps:

[0026] S1, material preparation, placing socks in a first working area, wherein the first working area can be a conventional workbench, and a depth camera is arranged above the first working area. The socks can be manually stacked in the first working area, or can be stacked in the first working area by using automatic equipment. Preferably, in the embodiment, the socks are laid flat on the conveying section of the belt conveyor, and the conveying section passes through the first working area, so that the socks can be placed at the feeding end of the conveyor, and the socks enter the first working area under the conveying of the conveyor, thereby avoiding the action of placing the socks affecting the photographing timing of the depth camera. At the same time, laying the socks flat can also make more sock rosettes present on the upper surface of the sock pile, improve the photographing efficiency, and also avoid the movement of other socks due to mutual entanglement with the sock when picking up one sock.

[0027] Of course, a sock turnover device disclosed in the utility model patent with the authorization announcement No. CN 217349686 U can also be used to replace the above-mentioned belt conveyor, and at this time, the turnover frame of the turnover device is used as the first working area.

[0028] S2, recognition, using the depth camera to take a photograph of the first working area where the socks are placed to obtain a shooting image, and obtaining the midpoint position information of the center point of the rosette of all the socks from the shooting image. The specific center position information acquisition method is as follows:

[0029] S2.1, training an R-CNN model for identifying the center point of the sock cuff, and obtaining a pre-trained model; the training method of the R-CNN model can adopt a conventional method, and preferably, in the embodiment, when the R-CNN model is trained, at least three corner points of the cuff pattern are determined according to the training image input into the R-CNN model, and then the midpoint position of the cuff pattern is determined according to the corner points of the cuff pattern and is output; specifically, when the R-CNN model is trained, the sock is placed in the first working area, a depth camera is used to take a photograph of the first working area in which the sock is placed to obtain a photographed image, then the edge non-sock area in the photographed image is manually cut off to obtain a square image, the sock whose cuff can be seen is manually found in the image, and then it is manually judged whether the sock can be sucked and extracted from the first working area by the vacuum suction tube, if yes, at least three corner points of the cuff of the sock are manually marked, and at the same time, the best suction position of the vacuum suction tube at the cuff of the sock (usually the center position of the cuff) is manually marked, otherwise, no processing is performed, then the manually marked information is input into the R-CNN model, and finally the above training steps are repeated for a predetermined number of times to obtain the pre-trained model; the specific number of repetitions needs to be set according to the actual production fault tolerance requirement, for example, it can be repeated for 100-300 times, etc., and of course, each time the photographed image should be different.

[0030] It should be noted that the above-mentioned corner points refer to the corner points of the cuff contour displayed in the image, and usually the cuff contour is square and has four corner points, which can be marked by manual marking, but the cuff of the sock can be partially blocked by other socks, and only part of the corner points are displayed in the image, in this case, if only one corner point is displayed, it is usually considered that the sock is difficult to be extracted by the vacuum suction tube, if only two or three corner points are displayed, it is manually judged according to experience whether the sock can be extracted by the vacuum suction tube, if yes, after the two or three displayed corner points are manually marked, the possible positions of the remaining corner points (which are blocked by other socks) are manually judged and marked, and when the manually marked information is input into the R-CNN model, the corner points not displayed in the image need to be marked in order to be distinguished from the corner points that can be displayed in the image.

[0031] S2.2, performing ROI processing on the photographed image, that is, the region to be processed is outlined in a square, circular, elliptical or irregular polygonal manner from the processed image to remove the edge non-sock area and obtain an effective image; the specific ROI processing method is a conventional method and is widely used in various types of image processing, which is not the focus of the embodiment, and will not be described in detail here.

[0032] S2.3, input the effective image into the pre-trained model obtained in step S2.1, to obtain the midpoint position information of the center points of the cuffs of all the socks in the image. It should be noted that the midpoint position information output by the pre-trained model is determined based on the state in which the cuff of the sock is displayed in the image, and the center point of the corresponding sock cuff is not necessarily the physical center point of the sock cuff.

[0033] S3, take the socks, and according to each midpoint position information, take the corresponding sock to the second working area one by one, wherein the second working area can be a conventional workbench or the input end of a belt conveying device. Preferably, before taking the socks, sort the midpoint position information according to the depth (i.e. the distance between the corresponding center point and the depth camera), the area occupied in the captured image, and the Euclidean distance between each center position information and the origin of the depth camera (i.e. the origin of the three-dimensional coordinate system established based on the captured image). The weights of the above factors can be set according to actual production needs, of course, other factors that affect the sorting and their weights can also be determined according to actual production needs. After sorting, the corresponding socks are taken to the second working area one by one according to the midpoint position information, which helps to improve the efficiency of taking socks.

[0034] When taking the socks, first use the vacuum suction tube to adsorb the sock at the center point position of the cuff of the sock according to the corresponding position information, and lift it up by a first distance, so that the cuff of the sock is separated from other socks. The vacuum suction tube is lifted up while detecting whether the cuff of the corresponding sock is adsorbed. If so, continue to lift up, otherwise give up taking the sock. The specific detection method can be a conventional method, such as detecting the vacuum degree of the vacuum suction tube to confirm whether the cuff of the sock is adsorbed. After the lifting action is completed, the cuff of the sock is clamped by the clamping jaw, the sock is clamped and placed in the second working area, which can avoid the socks from being entangled with each other and unable to be pulled out from the sock pile. During the process of placing the sock in the second working area, the lower segment of the sock should be contacted with the second working area first, and then the cuff of the sock is released after the sock is dragged in the second working area by a certain distance by the clamping jaw, so as to ensure that the socks can be arranged flat in the second working area.

[0035] Preferably, the vacuum suction pipe and the clamping jaw are installed at the end of the same manipulator, and the vacuum suction pipe is located between the two clamping rods of the clamping jaw, which helps to improve the accuracy of the clamping position. The manipulator can be a conventional multi-joint serial manipulator or a parallel manipulator, which will not be described in detail. After the clamping jaw clamps the cuff of the sock, it moves a second distance in a direction away from the second working area, and then moves to the second working area to perform the placing action, which can further ensure that even if the sock is long (such as a stocking), it can be pulled out from the sock pile. In addition, in step S3 of the embodiment, first, it is judged according to each midpoint position information whether the midpoint position of the corresponding sock cuff is located within the working range of the vacuum suction pipe or the manipulator. If so, the corresponding midpoint position information is retained, otherwise the corresponding midpoint position information is deleted. Then, according to the midpoint position information (the midpoint position information at this time does not contain the deleted midpoint position information), the corresponding sock is clamped to the second working area one by one, which helps to avoid errors in the sock taking action.

[0036] The above has made a detailed description of the present application, but the embodiments of the present application are not limited to the above-mentioned embodiments, and those skilled in the art can make various changes and applications to the present application according to the prior art, which are all within the protection scope of the present application.

Claims

1. A sock picking method based on visual recognition, characterized in that, Includes the following steps: S1. Prepare materials: Place the socks in the first work area; S2. Identification: A depth camera is used to take pictures of the first working area where socks are placed to obtain a captured image, and the midpoint position information of the center point of the cuff of all socks is obtained from the captured image. S3. Take socks. According to the midpoint position information, take the corresponding socks one by one into the second working area. When taking socks, first use a vacuum tube to suck the sock at the center point of the sock's cuff according to the corresponding position information, and lift it up a first distance. Then use the claw to clamp the cuff of the sock, pick up the sock and place it in the second working area.

2. The sock picking method based on visual recognition as described in claim 1, characterized in that, The vacuum suction tube and the gripper are mounted at the end of the same robotic arm, and the vacuum suction tube is located between the two gripping bars of the gripper.

3. The sock picking method based on visual recognition as described in claim 2, characterized in that, In step S3, before picking up the socks, the midpoint position information is sorted according to the depth of each midpoint position information, the area occupied in the captured image, and the Euclidean distance between each midpoint position information and the origin of the depth camera. Then, the corresponding socks are picked up into the second working area in sequence according to the midpoint position information.

4. The sock picking method based on visual recognition as described in claim 1, characterized in that, In step S1, the socks are laid flat on the conveyor section of the belt conveyor, which passes through the first working area.

5. The sock picking method based on visual recognition as described in claim 1, characterized in that, In step S3, after the gripper clamps the cuff of the sock, it first moves a second distance in a direction other than the second working area, and then moves to the second working area to perform the placement action.

6. The sock picking method based on visual recognition as described in claim 1, characterized in that, In step S3, while the vacuum tube is lifting the sock, it simultaneously detects whether the cuff of the sock is being sucked up. If it is, it continues to lift the tube upward; otherwise, it abandons the attempt to pick up the sock.

7. The sock picking method based on visual recognition as described in any one of claims 1-6, characterized in that, In step S2, the method for obtaining the position information of the center point of the cuff of all socks from the captured image includes the following steps: S2.1, Train an R-CNN model to identify the center point of the sock cuff to obtain a pre-trained model; S2.2, Perform ROI processing on the captured image to remove the edge-free area and obtain a valid image; S2.3, Input the effective image into the pre-trained model to obtain the midpoint position information.

8. The sock picking method based on visual recognition as described in claim 7, characterized in that, In step S3, firstly, based on the midpoint position information, it is determined whether the corresponding midpoint position is within the working range of the vacuum suction tube. If it is, the corresponding midpoint position information is retained; otherwise, the corresponding midpoint position information is deleted. Then, based on the midpoint position information, the corresponding socks are clamped into the second working area one by one.

9. The sock picking method based on visual recognition as described in claim 7, characterized in that, The training method of the R-CNN model includes: determining a grooving pattern with at least three corner points based on the training image input to the R-CNN model, and then determining and outputting the midpoint position of the grooving pattern based on each of the corner points.