Image processing system for finding the head of a beaming drum
By combining industrial cameras and depth cameras with point cloud filtering and template matching, three-dimensional positioning and automatic grasping of yarn bobbin ends were achieved, solving the problem of yarn end automation in the warping process and improving the level of automation and safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DONGHUA UNIV
- Filing Date
- 2025-04-21
- Publication Date
- 2026-05-29
AI Technical Summary
Automation of yarn end finding in the warping process is difficult to achieve, manual operation is dangerous and inefficient, and existing two-dimensional vision inspection is not effective in complex backgrounds and cannot effectively eliminate background interference from the yarn frame and obtain yarn depth information.
An industrial camera equipped with a depth camera is used, combined with point cloud filtering, background segmentation and point cloud template matching, to achieve yarn bobbin head location through 3D line segment detection. A robotic arm is used to grasp the yarn head, integrating point cloud filtering, point cloud downsampling, point cloud template matching and 3D line segment detection algorithms.
It improves the automation level of the warping process, reduces the risk of safety accidents, enables precise grasping and pulling of yarn ends, solves the problems of lighting dependence and background interference in two-dimensional image processing, and obtains the depth information of the yarn.
Smart Images
Figure CN122115308A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of yarn detection and image processing technology, specifically to an image processing system for finding the head of warp yarn bobbins. Background Technology
[0002] Warping is an important process in the textile printing and dyeing industry. It involves winding pre-wound yarn into parallel sections of a specified number and length onto a warp beam of a specified width. However, the warping process largely relies on manual methods for hanging yarn and finding yarn ends. Workers need to use ladders to replace yarn bobbins at higher positions and retrieve yarn ends, while workers must bend over for extended periods to handle yarn on lower bobbins. This method is highly dangerous and prone to accidents or omissions. Therefore, automating the yarn end finding process in warping and using machines to replace manual yarn end retrieval has practical application value.
[0003] A search revealed the patent "A Yarn Detection System Based on Image Processing" (202411277774.9), which discloses a yarn detection system based on image processing. It comprehensively and deeply extracts multi-dimensional feature information from yarn images through various techniques such as edge detection, texture feature extraction, color feature extraction, shape feature extraction, region feature extraction, and grayscale feature extraction. The patent "A Broken Yarn Finding Device Based on Machine Vision" (202010380839.8) discloses a machine vision-based broken yarn finding device and its usage method, belonging to the field of automatic splicing technology for ring-spun yarn. It uses a robot to pick up and place yarn tubes, uses a brush to remove broken yarns attached to the surface of the yarn tubes, introduces machine vision methods to identify and locate broken yarns and detect whether the broken yarn has been successfully captured, and finally uses a negative pressure air pump to capture the broken yarn.
[0004] Most of the yarn breakage detection methods described above operate at the two-dimensional image level, using a camera platform that holds the yarn in place to detect yarn breaks. This requires specific environmental and lighting conditions, or a dedicated adsorption mechanism to capture the yarn breakage. However, warping frames with densely packed yarn bobbins cannot accommodate yarn-fixing devices. Furthermore, complex backgrounds and uncontrollable factory lighting conditions make it difficult for two-dimensional visual inspection to achieve the required positioning accuracy. Summary of the Invention
[0005] The purpose of this invention is to provide an image processing system for finding the end of warp yarn bobbins. The hardware system includes an industrial camera, a host computer, and a yarn end grasping component. The software system includes image acquisition, image preprocessing, and algorithm implementation. The image preprocessing includes point cloud filtering, background segmentation, and point cloud downsampling. The algorithm implementation includes point cloud template matching and three-dimensional line segment detection. The yarn end grasping component is a yarn end grasping robotic arm. The industrial camera is mounted on the yarn end grasping robotic arm. During image acquisition, a depth camera mounted on the robotic arm is used to photograph the yarn bobbins hanging on the yarn rack from the front.
[0006] Preferably, the image data preprocessing module includes the following specific steps: S1: Point cloud filtering to determine the region of interest and eliminate as many interfering point clouds as possible around the non-identifiable target; S2: Planar culling to separate the background planar point cloud of the unwanted yarn frame from the point cloud of the yarn bobbin and yarn.
[0007] S3: Point cloud is downsampled to reduce the complexity of point cloud data and reduce storage space, while retaining the main features and shape information of the point cloud.
[0008] Preferably, the specific steps in the point cloud template matching module are as follows: S1: Point cloud feature description, which obtains attribute information such as the position, normal direction, and curvature of the points, realizes data dimensionality reduction, and enables the matching algorithm to quickly lock the potential corresponding region; S2: Coarse registration of point cloud template. The source point cloud and the template point cloud roughly overlap to obtain the preliminary registration matrix of the yarn tube spatial position. S3. Point cloud template fine registration: Based on the initial registration matrix obtained in the previous step, further iterative optimization is performed to improve the registration accuracy, obtain the final registration matrix, and obtain the spatial range of the yarn tube.
[0009] Preferably, the yarn detection module performs the following steps: S1: First, the input point cloud is divided into three-dimensional planes. For each three-dimensional plane, all the points belonging to it are projected onto the plane to form a two-dimensional image. S2: Then perform two-dimensional contour extraction and yarn feature extraction to obtain two-dimensional line segments for each plane; S3: Then these two-dimensional line segments are reprojected onto a three-dimensional plane to obtain three-dimensional line segments.
[0010] The beneficial effects of the image processing system for finding yarn ends in warping bobbins in this invention are that, in situations where the yarn count is small and ordinary point cloud processing algorithms easily identify yarn as outliers, a composite point cloud detection algorithm is proposed. This algorithm can avoid the problems of high requirements for lighting conditions, inability to effectively eliminate interference from the background information of the yarn frame, and inability to obtain the depth information of yarn breaks in two-dimensional image processing. It can also obtain the yarn depth information that cannot be obtained in two-dimensional vision processing, thereby detecting the three-dimensional coordinates of the yarn ends hanging from the yarn bobbin on the warping frame in the warping process. This solves the problem that single point cloud detection algorithms are prone to judging the point clouds of yarn with extremely small yarn counts as outliers and failing to detect them. At the same time, it can also make false detections caused by misjudging the straight features of the yarn rod and yarn bobbin as yarn. Furthermore, it facilitates the subsequent use of robots to automatically grab the yarn ends and guide them to the warping and yarn collection point, improving the automation level of the entire warping process and reducing the possibility of safety accidents caused by textile workers climbing up and down ladders to grab yarn ends. Attached Figure Description
[0011] Figure 1 This is a hardware and software system diagram of the warping bobbin finding system of the present invention; Figure 2 This is a diagram illustrating the implementation steps of the software algorithm of the present invention; Figure 3 This is a diagram illustrating the working steps of the preprocessing module of the present invention; Figure 4 This is a diagram illustrating the point cloud template matching process of the present invention. Figure 5 This is a diagram illustrating the working steps of the yarn detection module of the present invention; Figure 6 This is a diagram showing the results of the preprocessing module of the present invention. Figure 7 This is a diagram showing the result of point cloud template matching in this invention. Figure 8 This is a diagram showing the results of the three-dimensional line segment detection operation of the present invention; Detailed Implementation
[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0013] Please see Figure 1-8 The present invention provides a technical solution: The warp yarn bobbin finding image processing system includes a hardware system and a software system. The hardware system includes an industrial camera, a host computer, and a yarn end grasping component. The software system includes image acquisition, image preprocessing, and algorithm implementation. Image preprocessing includes point cloud filtering, background segmentation, and point cloud downsampling. Algorithm implementation includes point cloud template matching and 3D line segment detection. The yarn end grasping component is a yarn end grasping robotic arm. The industrial camera is mounted on the yarn end grasping robotic arm. During image acquisition, a depth camera mounted on the robotic arm is used to photograph the yarn bobbins hanging on the yarn rack from the front.
[0014] In some embodiments: the hardware system refers to a series of physical devices specifically used for image capture and control execution, mainly including industrial cameras, host computers, and yarn end gripping robotic arms, while the software system is a computer application system consisting of a collection of programs that implement various algorithms in image processing. Based on point cloud data algorithms, template matching is first used to determine the range of point clouds containing yarn bobbin features, and then a point cloud line segment detection algorithm based on three-dimensional projection is used to find all possible yarn point clouds. After that, line segment information within the yarn bobbin range is removed to obtain accurate yarn point cloud coordinates. During the image acquisition process, a depth camera mounted on the robotic arm is used to take a frontal picture of the yarn bobbins hanging on the yarn rack.
[0015] In some embodiments: the host computer is the main control device in industrial control, automation and other systems, and is mostly a general-purpose PC or industrial control computer. It is connected to the slave computer through communication interfaces such as serial port and Ethernet. It can collect and display data from the slave computer in real time, such as the operating parameters of the equipment on the production line; provide an intuitive operation interface to facilitate remote operation of the equipment; and process and analyze data. The industrial camera is mainly composed of optical lens, image sensor, data transmission interface and housing, etc. All of the above belong to the prior art.
[0016] Specifically, the image data preprocessing module includes the following steps: S1: Point cloud filtering to determine the region of interest and eliminate as many interfering point clouds as possible around the non-identifiable target; S2: Planar culling to separate the background planar point cloud of the unwanted yarn frame from the point cloud of the yarn bobbin and yarn.
[0017] S3: Point cloud is downsampled to reduce the complexity of point cloud data and reduce storage space, while retaining the main features and shape information of the point cloud.
[0018] In some embodiments, the above-mentioned preprocessing methods such as filtering, culling, and downsampling can eliminate unwanted environmental background, obtain high-quality yarn bobbin point clouds while retaining sufficient information and reducing the size of the point cloud, thereby improving the efficiency of subsequent algorithm processing and reducing the demand for computing resources.
[0019] Furthermore, in the point cloud template matching module, the specific steps are as follows: S1: Point cloud feature description, which obtains attribute information such as the position, normal direction, and curvature of the points, realizes data dimensionality reduction, and enables the matching algorithm to quickly lock the potential corresponding region; S2: Coarse registration of point cloud template. The source point cloud and the template point cloud roughly overlap to obtain the preliminary registration matrix of the yarn tube spatial position. S3. Point cloud template fine registration: Based on the initial registration matrix obtained in the previous step, further iterative optimization is performed to improve the registration accuracy, obtain the final registration matrix, and obtain the spatial range of the yarn tube.
[0020] In some embodiments: the above steps involve preprocessing the point cloud data and extracting features from the point cloud for template matching. Successfully identifying and locating the portion of the yarn bobbin point cloud provides a guarantee for subsequent accurate yarn detection.
[0021] Furthermore, based on the aforementioned steps, the yarn detection module performs the following steps: S1: First, the input point cloud is divided into three-dimensional planes. For each three-dimensional plane, all the points belonging to it are projected onto the plane to form a two-dimensional image. S2: Then perform two-dimensional contour extraction and yarn feature extraction to obtain two-dimensional line segments for each plane; S3: Then these two-dimensional line segments are reprojected onto a three-dimensional plane to obtain three-dimensional line segments.
[0022] Furthermore, the above steps can detect straight line information of all suspected yarns in the input point cloud, but not the straight line information of non-yarns, such as yarn bobbin outlines and yarn rods, which are basically located in the yarn bobbin area. Combined with the yarn bobbin range determined in the point cloud template matching module, the point cloud within the range of the yarn bobbin can be removed by filtering, thereby accurately retaining the true yarn straight line detection information.
[0023] In some embodiments: the result of the image preprocessing module is as follows Figure 6 As shown, (a) is the original data of the collected point cloud image, and (b) is a high-quality point cloud containing only yarn bobbins and yarns; In the point cloud template matching module, the initial positions of the template point cloud data and the actual point cloud data of the yarn bobbin are as follows: Figure 7 As shown in (a), the source point cloud data contains 15,340 points, while the template point cloud data contains 100,011 points. The number of corresponding point pairs, s, is set to 3 for each iteration, and the minimum distance between each sample point on the same point cloud is... d min The sample size is 5 mm, and the number of samples is 10. The registration result is as follows: Figure 7As shown in (b)-(e), the number of iterations increases sequentially, and the mismatch score decreases sequentially. After 250 iterations, the value remains unchanged, reaching 0.00134565, which is considered the optimal matching effect. The coarse registration transition matrix at this point is [-0.43796 -0.89847 -0.0307046 191.19 ;-0.179211 0.0537859 0.982339 -945.173 ;-0.880951 0.435728 -0.184572 210.964 ;0 0 0 1;]. For fine registration, the minimum distance dmin between each pair of matching points is set to 1 mm, and the number of iterations is 500. The final mismatch score is 0.0012502, yielding the final transformation matrix and obtaining the optimal spatial range of the yarn bobbin, as shown below. Figure 7 As shown in (e).
[0024] In the yarn detection module, the input point cloud is segmented into three-dimensional planes, and the three-dimensional coordinates on the planes are converted into two-dimensional coordinates. These points are then converted into binary image pixels using a meshing formula. Pixels mapped from the two-dimensional planes are assigned a value of 255, while others are set to 0. Each plane corresponds to a binary image. Based on the binary images generated in the above steps, edge detection combined with Hough transform is used to quickly detect line segment information, which is then drawn using green lines, as shown below. Figure 8 As shown in (a), after extracting the two-dimensional line segments from the binary image, it is necessary to recover the corresponding three-dimensional line segment information through inverse projection from two-dimensional to three-dimensional. The recovery result is shown in (a). Figure 8 As shown in (b). After excluding the linear information within the yarn bobbin area, the true three-dimensional information indicating the yarn can be obtained, represented by the coordinates as follows. Figure 8 As shown in (c).
[0025] In some embodiments, the image processing system for finding yarn ends in warping bobbins can identify the pre-reserved yarn ends in the suspended state of the bobbins mounted on the warping frame. It proposes a yarn detection algorithm that combines point cloud registration with three-dimensional line segment recognition, thereby solving the problems of high requirements for lighting conditions, inability to effectively eliminate interference from the background information of the yarn frame, and inability to obtain the depth information of yarn breaks in two-dimensional image processing. It also solves the problem that single point cloud detection algorithms are prone to judging yarn point clouds with extremely low yarn counts as outliers and failing to detect them. At the same time, it can also make false detections by judging the straight features of the yarn rod and bobbin as yarn. This system can provide the coordinates of the grasping point for subsequent use of robots to grasp yarn ends, improve the automation level of the entire warping process, and reduce the possibility of safety accidents caused by textile workers climbing up and down ladders to grasp yarn ends.
[0026] The working principle of this invention is as follows: Based on point cloud data algorithms, the system first uses template matching to determine the range of point clouds containing yarn bobbin features. Then, a point cloud line segment detection algorithm based on 3D projection is used to find all possible yarn point clouds. After that, line segment information within the yarn bobbin range is removed to obtain accurate yarn point cloud coordinates. During image acquisition, a depth camera mounted on a robotic arm is used to photograph the yarn bobbins mounted on the yarn rack from the front. In the image data preprocessing module, the specific steps are as follows: point cloud filtering to determine the region of interest and eliminate as many interfering point clouds as possible around non-identifiable targets; planar culling to separate the background planar point cloud of the yarn rack from the yarn bobbin point cloud; point cloud downsampling to reduce point cloud data complexity and storage space, while retaining the main features and shape information of the point cloud. Preprocessing techniques such as elimination and downsampling can remove unwanted environmental background, obtain high-quality yarn bobbin point clouds while retaining sufficient information and reducing the size of the point cloud, thereby improving the efficiency of subsequent algorithm processing and reducing the demand for computing resources. In the point cloud template matching module, the specific steps are as follows: Point cloud feature description, obtaining attribute information such as point position, normal direction, and curvature, realizing data dimensionality reduction, enabling the matching algorithm to quickly lock potential corresponding regions; coarse registration of point cloud template, the source point cloud and the template point cloud roughly overlap, obtaining a preliminary registration matrix for the spatial position of the yarn bobbin; fine registration of point cloud template, based on the initial registration matrix obtained in the previous step, further iterative optimization to improve the registration accuracy, obtaining the final registration matrix, obtaining the spatial range of the yarn bobbin, and using the above steps to preprocess the point cloud data, extracting features from the point cloud for template matching. Successfully identifying and locating the portion of the yarn bobbin point cloud provides a guarantee for subsequent accurate yarn detection. Based on the aforementioned steps, the yarn detection module performs the following steps: First, the input point cloud is segmented into three-dimensional planes. For each three-dimensional plane, all points belonging to it are projected onto the plane to form a two-dimensional image. Then, two-dimensional contour extraction and yarn feature extraction are performed to obtain two-dimensional line segments for each plane. Next, these two-dimensional line segments are reprojected onto the three-dimensional plane to obtain three-dimensional line segments. This allows the detection of straight line information for all suspected yarns in the input point cloud, as well as non-yarn straight line information, such as yarn bobbin contours and yarn rods, which are basically located in the yarn bobbin area. Combined with the yarn bobbin range determined in the point cloud template matching module, filtering methods can be used to remove point clouds within the yarn bobbin area, thereby accurately retaining the true yarn straight line detection information.
[0027] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection defined by the claims submitted herein.
Claims
1. A warping bobbin head-finding image processing system, comprising a hardware system and a software system, characterized in that: The hardware system includes an industrial camera, a host computer, and a yarn end gripping component. The software system includes image acquisition, image preprocessing, and algorithm implementation. The image preprocessing includes point cloud filtering, background segmentation, and point cloud downsampling. The algorithm implementation includes point cloud template matching and three-dimensional line segment detection. The yarn end gripping component is a yarn end gripping robotic arm. The industrial camera is mounted on the yarn end gripping robotic arm. During image acquisition, a depth camera mounted on the robotic arm is used to photograph the yarn bobbins hanging on the yarn rack from the front.
2. The image processing system for finding the warp yarn bobbin head according to claim 1, characterized in that: The specific steps in the image data preprocessing module are as follows: S1: Point cloud filtering to determine the region of interest and eliminate as many interfering point clouds as possible around the non-identifiable target; S2: Planar culling to separate the background planar point cloud of the unwanted yarn frame from the point cloud of the yarn bobbin; S3: Point cloud is downsampled to reduce the complexity of point cloud data and reduce storage space, while retaining the main features and shape information of the point cloud.
3. The image processing system for finding the warp yarn bobbin head according to claim 1, characterized in that: The specific steps in the point cloud template matching module are as follows: S1: Point cloud feature description, which obtains attribute information such as the position, normal direction, and curvature of the points, realizes data dimensionality reduction, and enables the matching algorithm to quickly lock the potential corresponding region; S2: Coarse registration of point cloud template. The source point cloud and the template point cloud roughly overlap to obtain the preliminary registration matrix of the yarn tube spatial position. S3. Point cloud template fine registration: Based on the initial registration matrix obtained in the previous step, further iterative optimization is performed to improve the registration accuracy, obtain the final registration matrix, and obtain the spatial range of the yarn tube.
4. The image processing system for finding the warp yarn bobbin head according to claim 1, characterized in that: The yarn detection module performs the following steps: S1: First, the input point cloud is divided into three-dimensional planes. For each three-dimensional plane, all the points belonging to it are projected onto the plane to form a two-dimensional image. S2: Then perform two-dimensional contour extraction and yarn feature extraction to obtain two-dimensional line segments for each plane; S3: Then these two-dimensional line segments are reprojected onto the three-dimensional plane to obtain three-dimensional line segments.