A fault detection device and shutdown control method for tufting machines based on high-speed image capture and recognition

Abstract

A fault detection device and shutdown control method for a tufting machine based on high-speed image capture and recognition includes multiple yarns arranged longitudinally, tufting hooks for hooking the yarns, a transverse mechanism positioned between the yarns and the upper surface of the untufted portion of the fabric, an array of tufting needles for inserting the yarns into the fabric, and several high-speed cameras for photographing the upper surface of the fabric. The high-speed cameras are fixedly mounted above the fabric and arranged sequentially along the direction of the tufting needle array at the same height, with their cameras pointing vertically downwards towards the upper surface of the fabric. The shooting ranges of adjacent high-speed cameras overlap. This method only requires the acquisition of lightweight features and simple kinematic relationships to synthesize and stitch together a complete image of the upper surface of the fabric, greatly saving computation time and further resolving the contradiction between the real-time requirements for fault detection and the inherent time delay in image recognition processing in existing technologies.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of image recognition technology, and in particular to a fault detection device and shutdown control method for a tufting machine based on high-speed image capture and recognition. Background Technology

[0002] Detecting yarn breakage faults on looms using high-speed image recognition technology faces a core contradiction: the conflict between the real-time requirement for fault detection and the inherent time delay in image recognition processing. Tufting machines operate at extremely high speeds. Once a yarn breakage fault occurs, it needs to be accurately detected within a very short time (milliseconds, typically <20-50ms) to trigger a shutdown or alarm, preventing serious fabric defects or yarn entanglement in other components that could damage the equipment.

[0003] In image processing, feature extraction, object detection, and image stitching are the most time-consuming core steps. Whether it's traditional edge detection, texture analysis, template matching, or more complex deep learning models, all require a large amount of computing resources. High-precision models are usually accompanied by even greater computational demands, and each step introduces latency.

[0004] In existing technologies, high-speed cameras are typically arranged horizontally at the top of the loom. When taking pictures for identification, the images taken by adjacent cameras are stitched together. In order to improve the stitching accuracy and avoid misprocessing of the images at the stitching point, it is necessary to improve the shooting accuracy and use complex edge detection algorithms for stitching processing. This greatly prolongs the processing time and leads to delays in fault identification.

[0005] Therefore, it is necessary to design a fault detection device and shutdown control method for tufting machines based on high-speed image capture and recognition, so as to resolve the contradiction between the real-time requirements for fault detection and the inherent time delay of the image recognition and processing process in the existing technology. Summary of the Invention

[0006] The purpose of this invention is to provide a fault detection device and shutdown control method for tufting machines based on high-speed image capture and recognition, so as to solve the technical problems in the prior art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A fault detection device for a tufting machine based on high-speed image capture and recognition includes multiple yarns arranged longitudinally, tufting hooks for hooking the yarns, a transverse mechanism disposed between the yarns and the upper surface of the untufted portion of the fabric, a tufting needle array for inserting the yarns into the fabric, and several high-speed cameras for photographing the upper surface of the fabric. The several high-speed cameras are fixedly disposed above the fabric and are arranged sequentially along the direction of the tufting needle array, and have the same height. The camera lenses of the high-speed cameras face vertically downward toward the upper surface of the fabric. The shooting ranges of adjacent high-speed cameras have overlapping areas.

[0008] Preferably, each high-speed camera has a corresponding rectangular shooting range of the same size, wherein the shooting range of all cameras covers the entire upper side of the fabric in the direction of the tufting needle array and covers the range from the yarn, through the cross mechanism to the tufting needle array and downstream of the tufted fabric in the direction of the yarn.

[0009] Preferably, the transverse mechanism moves back and forth along the width of the fabric, and during the reciprocating motion of the transverse mechanism, the high-speed camera can capture the position of the transverse mechanism from an overhead view.

[0010] Preferably, the traversing mechanism is a mechanism driven by a cylinder, motor, or similar device that moves rapidly traversing on a precision slide rail.

[0011] Preferably, a position sensor is provided to detect the position of the traversing mechanism in real time, so as to detect and record the accurate position of the traversing mechanism relative to the frame in real time.

[0012] A shutdown control method for a tufting machine fault detection device based on high-speed image capture and recognition specifically includes the following steps: Step 1: Take a photo each time the traversing mechanism is used; Step 2: Stitch together the photos taken during one crossing; Step 3: Perform yarn identification on the stitched image, count the number of yarns, and locate the broken yarn position.

[0013] Preferably, in step one, each time the transverse traverse mechanism performs a transverse traverse, the synchronous tufting needle array performs a row of tufting operations, and the fabric also moves forward one unit. The high-speed cameras arranged along the yarn threading direction are controlled to take pictures in sequence according to the movement position of the transverse traverse mechanism.

[0014] Preferably, in step one, the shooting method is as follows: when the traversing mechanism moves to exactly enter an overlapping area, the two adjacent high-speed cameras constituting the overlapping area are controlled to take pictures.

[0015] Preferably, the image stitching component in step two consists of two steps. First, the images captured simultaneously are stitched together. Then, the features of the transverse mechanism in the images captured simultaneously by adjacent high-speed cameras are identified. Based on the features of the transverse mechanism, several preliminary stitched images are obtained by direct stitching. Then, several preliminary stitched images are combined and stitched together in the final composite image. This final composite image is based on the movement position relationship of the traversing mechanism. The features of the traversing mechanism in several preliminary stitched images are extracted. The position of the traversing mechanism entering different overlapping areas during its movement relative to the frame is detected and recorded in real time by the position sensor. Based on the relative position and distance of the traversing mechanism in different overlapping areas by the position sensor, several preliminary stitched images are stitched together to obtain the final composite image.

[0016] Preferably, in step three, yarn identification and yarn breakage analysis are performed on the final composite image of the entire upper surface of the spliced ​​fabric. First, directional filtering is used to enhance the vertical texture and suppress interference information such as tufted needle arrays. Then, an adaptive threshold segmentation algorithm is used to accurately separate the target yarn area. Yarn quantity statistics are based on connected component analysis. The peak value is locked and the number of peak values ​​is counted by the longitudinal projection histogram. Morphological closing operation is combined to close gaps to eliminate interference from minor breaks. After ensuring the continuity of a single yarn, the total number of connected regions is obtained. Yarn breakage location is achieved by depth traversing the grayscale curve of each column of pixels: for each identified yarn, the grayscale change law is fitted in real time along the longitudinal scanning trajectory. When the grayscale of consecutive pixels deviates significantly from the baseline and the width exceeds the set threshold, the breakage edge is confirmed by edge detection, and a yarn breakage can be determined. At the same time, the transverse structure analysis is integrated to eliminate interference from folds and wrinkles. Finally, the physical location of the broken yarn is accurately located by the pixel coordinates of the breakage point, and the yarn number and longitudinal offset are output to achieve automated defect detection. When a defect is detected, a stop control operation is performed.

[0017] The beneficial effects of this invention are: This invention, during image stitching and synthesis, sets overlapping shooting areas and captures images of the traversing mechanism within these overlapping areas. It only requires collecting lightweight features and using simple kinematic relationships to complete the overall image synthesis and stitching of the fabric's upper surface. Then, yarn identification and yarn breakage detection are performed. Furthermore, each traversing process is combined with the tufting process and the yarn stepping process of the same column, which is carried out synchronously, greatly saving computation time and further resolving the contradiction between the real-time requirements for fault detection and the inherent time delay in image recognition and processing in existing technologies. Attached Figure Description

[0018] Figure 1 This is a top view of a tufting machine fault detection device based on high-speed image capture and recognition; Figure 2This is the main view of a tufting machine fault detection device based on high-speed image capture and recognition; Figure 3 This is a schematic diagram of the shutdown control detection method of the present invention; Figure 4 This is a side view of the high-speed image capture and recognition tufting machine fault detection device of the present invention; The meanings of the reference numerals in the figure are as follows: 1. Yarn; 2. Tufting hook; 3. Cross-cutting mechanism; 4. Tufting needle array; 7. Fabric; 5a. High-speed camera one; 5b. High-speed camera two; 5c. High-speed camera three; 5d. High-speed camera four; H1. Overlapping area one; H2. Overlapping area two; H3. Overlapping area three; 3a. First position; 3b. Second position; 3c. Third position. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in the embodiments of this specification, the technical solutions in the embodiments of this specification will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art should fall within the scope of protection.

[0020] 1. Device Structure: Figure 1 / 4 are top and side views of the loom fault detection device based on high-speed image capture and recognition in this embodiment. Figure 2 This is a front view of the loom fault detection device based on high-speed image capture and recognition according to this embodiment.

[0021] It mainly comprises multiple yarns 1 arranged longitudinally, tufting hooks 2 that hook the yarns 1, a transverse piercing mechanism 3 disposed between the yarns 1 and the upper surface of the untufted portion of the fabric 7, an array of tufting needles 4 for inserting the yarns into the fabric 7, and high-speed cameras 5a, 5b, 5c, and 5d for photographing the upper surface of the fabric 7.

[0022] A horizontally arranged array of tufting needles 4 and tufting hooks 2 together tuft the yarn 1 into the fabric 7, and the tufted fabric 7 is wound onto the take-up drum 6. Figure 1-2 In 4, with Figure 1 The direction in which the middle yarn 1 extends and is conveyed is the X direction, and the direction in which the tufted needle array extends is the Y direction. The X and Y directions are perpendicular to each other. Figure 2 The direction perpendicular to the fabric surface is the Z direction.

[0023] like Figure 2As shown, high-speed cameras 5a, 5b, 5c, and 5d are fixedly mounted above the fabric 7, arranged sequentially along the Y-direction and at the same height in the Z-direction. The cameras' lenses are positioned vertically downwards along the Z-direction, facing the upper surface of the fabric 7. Each high-speed camera has a corresponding rectangular shooting area of ​​the same size. Specifically, the shooting area of ​​high-speed camera 5a is rectangular region A, and its corresponding image is also image A; the shooting area of ​​high-speed camera 5b is rectangular region B, and its corresponding image is also image B; the shooting area of ​​high-speed camera 5c is C, and its corresponding image is also image C; and the shooting area of ​​high-speed camera 5d is D, and its corresponding image is also image D. The area formed by images A / B / C / D covers the entire upper side of fabric 7 in the Y direction and the entire range of tufted and untufted fabric 7 in the X direction. Most importantly, it covers the range from yarn 1, through mechanism 3 to tufting needle array 4 and downstream tufted fabric 7 in the X direction.

[0024] At the same time, such as Figure 1-2 As shown, the shooting ranges of adjacent high-speed cameras have overlapping areas. For example, images A and B have overlapping region H1, images B and C have overlapping region H2, and images C and D have overlapping region H3. Overlapping regions H1, H2, and H3 are rectangular areas of the same size and shape, meaning that the shooting ranges of each pair of adjacent high-speed cameras have the same overlapping portion.

[0025] Furthermore, since the lateral span of the loom, i.e. the Y-direction in the figure, is usually quite long, this embodiment of the invention only records four cameras—high-speed camera 5a, high-speed camera 5b, high-speed camera 5c, and high-speed camera 5d—covering the fabric span in the Y-direction. However, it is not limited that this invention can also use more or fewer than four high-speed cameras. The number of high-speed cameras arranged laterally is not limited, as long as it is ensured that the shooting range of each group of adjacent high-speed cameras has the same overlapping part.

[0026] Furthermore, regarding the transverse movement mechanism 3, the transverse movement mechanism 3 on the loom of the present invention is mainly responsible for reciprocating the structure with a certain special shape (such as a rectangle) along the fabric width direction (i.e., the Y direction). In the present invention, the transverse movement mechanism 3 is disposed between the yarn 1 and the upper surface of the untufted portion of the fabric 7. Due to the spaced arrangement of the yarn 1, a high-speed camera can capture the position of the transverse movement mechanism 3 from an overhead view during the reciprocating movement of the transverse movement mechanism 3. The transverse movement mechanism 3 of the present invention is a mechanism that rapidly moves laterally on a precision slide rail driven by a cylinder, motor, or similar means. A position sensor (not shown) is also provided to detect the position of the transverse movement mechanism 3 in real time. The position sensor can be a conventional displacement sensor or speed sensor to detect and record the accurate position of the transverse movement mechanism 3 relative to the frame (not shown) in real time.

[0027] Furthermore, it should be noted that high-speed cameras 1 (5a), 2 (5b), 3 (5c), and 4 (5d) are also fixed relative to the frame, so their shooting ranges A / B / C / D are also fixed relative to the frame. Similarly, overlapping areas 1 (H1), 2 (H2), and 3 (H3) are also relatively fixed relative to the frame.

[0028] 2. Shutdown control method: The following is combined with Figure 3 The shutdown control detection method of the present invention is described.

[0029] Step 1: Take a photo each time the traversing mechanism is used; Step 2: Stitch together the photos taken during one crossing; Step 3: Perform yarn identification on the stitched image, count the number of yarns, and locate the broken yarn position.

[0030] In step one above, each time the transverse mechanism traverses, the synchronous tufting needle array performs a row of tufting operations, and the fabric also moves forward one unit. Based on the movement position of the transverse mechanism, the high-speed cameras arranged along the yarn threading direction are controlled to take pictures sequentially. Specifically, when the transverse mechanism moves to exactly enter an overlapping area, the two adjacent high-speed cameras constituting that overlapping area are controlled to take pictures.

[0031] by Figure 3Taking the illustrated embodiment as an example, when the traversing mechanism 3 moves from left to right to the first position 3a, the head of the traversing mechanism 3 at the first position 3a is just fully inside the overlapping area H1. At this time, the high-speed camera 5a and the high-speed camera 5b are controlled to take pictures, resulting in photos A1 and B1 respectively. Both photos A1 and B1 show the traversing mechanism 3 at the first position 3a. Similarly, when the traversing mechanism 3 moves from left to right to the second position 3b, the head of the traversing mechanism 3 at the second position 3b is just fully inside the overlapping area H2. At this time, the high-speed camera 5b and the high-speed camera 5c are controlled to take pictures, resulting in photos B2 and C1 respectively. Both photos B2 and C1 show the traversing mechanism 3 at the second position 3b. Next, when the traversing mechanism 3 moves from left to right to the third position 3c, the head of the traversing mechanism 3 at the third position 3c is exactly inside the overlapping area 3H3. At this time, the high-speed camera 35c and the high-speed camera 45d are controlled to take pictures, obtaining photos C2 and D1 respectively. Both photos C2 and D1 show the traversing mechanism 3 at the third position 3c. Through step one, photos A1, B1, B2, C1, C2, and D1 are finally obtained.

[0032] In step two above, the photos taken during a single traverse are stitched together, that is, the photos A1, B1, B2, C1, C2, and D1 from step one are stitched together to obtain a top view of the complete fabric 7, so as to perform image recognition on the complete fabric surface. This can avoid misjudgment of edge areas when recognizing individual images separately.

[0033] Step two, image stitching, consists of two steps. The first step involves stitching together images taken simultaneously. Figure 3 Taking the illustrated embodiment as an example, since the transverse mechanism 3 at the first position 3a is captured in both images A1 and B1, it is only necessary to identify the features of the transverse mechanism 3 in images A1 and B1, and then directly stitch them together to obtain image A1B1, without needing to scan and identify the same pixels in the entire image of A1 and B1 before compositing. Since the position of each image is fixed during capture and the relative position of the transverse mechanism 3 is fixed, identifying the image of the transverse mechanism 3 is a lightweight feature extraction compared to scanning and identifying all pixels in the entire image. This invention greatly reduces the time for feature extraction and target detection during image stitching by extracting lightweight features, while also reducing computational power. Furthermore, since the transverse mechanism 3 undergoes almost no deformation during capture, a very high image resolution is not required, significantly reducing the computational load compared to identifying pixels in latitude and longitude regions and then performing matrix comparisons. Then, images B2CI and C2D1 are obtained in the same manner.

[0034] The images taken at different times are then stitched together based on the positional relationships of the traversing mechanism 3. Images A1B1, B2CI, and C2D1 contain the traversing mechanism 3 at positions 3a, 3b, and 3c, respectively. Features of the traversing mechanism 3 are extracted from these three images. Due to the presence of position sensors, which use displacement or velocity sensors to detect and record the precise position of the traversing mechanism 3 relative to the frame in real time, images A1B1, B2CI, and C2D1 are stitched together based on the relative positions and distances of the traversing mechanism 3 at positions 3a, 3b, and 3c, resulting in the final image ABCD. In this step, a global scan and recognition of the images is unnecessary; only the identified features of the traversing mechanism 3 and the relative positions of the three positions require direct pixel overlay stitching, eliminating the need for complex pixel matrix calculations and comparisons.

[0035] In step three, warp identification and yarn breakage analysis are performed on the overall image ABCD of the spliced ​​fabric 7. First, directional filtering (such as Gabor filter or FFT spectrum analysis) is used to enhance the vertical texture and suppress interference information such as tufted needle array. Then, an adaptive threshold segmentation algorithm (such as Otsu's method based on local gray-scale statistics) is used to accurately separate the warp target area. The warp count is based on connected component analysis. The peak value is locked by the longitudinal projection histogram and the number of peak values ​​is counted. Morphological closing operation is combined to close the gaps to eliminate the interference of small breaks. After ensuring the continuity of a single yarn, the total number of connected regions is obtained. Broken yarn location is achieved by traversing the grayscale curves of each column of pixels in depth: for each identified yarn, the grayscale change pattern is fitted in real time (such as moving average) along the longitudinal scanning trajectory. When the grayscale of consecutive pixels deviates significantly from the baseline (or a transverse break appears in the adjacent area) and the width exceeds a set threshold (such as more than twice the yarn diameter), the breakage edge is confirmed by edge detection, and a yarn breakage can be determined. At the same time, transverse structure analysis is integrated to eliminate interference from folds and wrinkles. Finally, the physical location of the broken yarn is accurately located by the pixel coordinates of the break point (combined with the image stitching mapping relationship), and the yarn number and longitudinal offset of the broken yarn are output to achieve automated defect detection. When a defect is detected, a stop control operation is performed.

[0036] Using the above method, only lightweight features need to be collected and simple kinematic relationships are needed to complete the overall image synthesis and stitching of the fabric's upper surface during image stitching. Yarn identification and yarn breakage detection are then performed, and this is done during each weft insertion process, greatly saving computation time. This further resolves the contradiction between the real-time requirements for fault detection and the inherent time delay in image recognition processing in existing technologies. The above description is merely a specific implementation of the embodiments of this specification. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of the embodiments of this specification, and these improvements and modifications should also be considered within the protection scope of the embodiments of this specification.

Claims

1. A tufting machine fault detection device based on high-speed image capture and recognition, characterized in that: The device includes multiple yarns arranged longitudinally, tufting hooks to hold the yarns, a transverse traversing mechanism positioned between the yarns and the upper surface of the untufted portion of the fabric, a tufting needle array for inserting the yarns into the fabric, and several high-speed cameras for photographing the upper surface of the fabric. The high-speed cameras are fixedly positioned above the fabric and arranged sequentially along the direction of the tufting needle array, with the same height. The camera lenses of the high-speed cameras face vertically downwards towards the upper surface of the fabric. The shooting ranges of adjacent high-speed cameras overlap, and the overlapping area is a rectangular area of ​​the same size and shape. The transverse traversing mechanism moves back and forth along the width of the fabric. During the reciprocating movement of the transverse traversing mechanism, the high-speed cameras can capture the position of the transverse traversing mechanism from an overhead perspective. The high-speed cameras arranged along the yarn insertion direction are controlled to take pictures sequentially based on the movement position of the transverse traversing mechanism. When the transverse traversing mechanism moves to exactly enter an overlapping area, the two adjacent high-speed cameras constituting that overlapping area are controlled to take pictures.

2. The tufting machine fault detection device based on high-speed image capture and recognition as described in claim 1, characterized in that: Each high-speed camera has a corresponding rectangular shooting range of the same size. The shooting range of all cameras covers the entire upper side of the fabric in the direction of the tufting needle array and covers the range from the yarn, through the cross mechanism to the tufting needle array and downstream tufted fabric in the direction of the yarn.

3. The tufting machine fault detection device based on high-speed image capture and recognition as described in claim 2, characterized in that: Position sensors are installed to detect the position of the traversing mechanism in real time, so as to detect and record the accurate position of the traversing mechanism relative to the frame in real time.

4. A shutdown control method using the high-speed image capture and recognition-based tufting machine fault detection device as described in claim 3, characterized in that: Specifically, the following steps are included: Step 1: Take a picture each time the traversing mechanism traverses. Each time the traversing mechanism traverses, the synchronous tufting needle array performs a row of tufting operations, and the fabric also moves forward one unit. Control the high-speed cameras arranged along the yarn threading direction to take pictures in sequence according to the movement position of the traversing mechanism. When the traversing mechanism moves to just completely enter an overlapping area, control the two adjacent high-speed cameras that constitute the overlapping area to take pictures. Step 2: Stitch and combine the photos taken during a single crossing. The image stitching process in Step 2 consists of two steps. First, the photos taken simultaneously are stitched and combined. The features of the crossing mechanism in the photos taken simultaneously by adjacent high-speed cameras are identified. Based on the features of the crossing mechanism, several preliminary stitched images are obtained by directly stitching them together. Then, several preliminary stitched images are finally combined and stitched together. This final combined stitching is based on the movement position relationship of the traversing mechanism. The features of the traversing mechanism in several preliminary stitched images are extracted. The position of the traversing mechanism entering different overlapping areas during its movement relative to the frame is detected and recorded in real time by the position sensor. Based on the relative position and distance of the traversing mechanism in different overlapping areas by the position sensor, several preliminary stitched images are stitched together to obtain the final combined stitched image. Step 3: Perform yarn identification on the stitched image, count the number of yarns, and locate the broken yarn position.

5. The shutdown control method as described in claim 4, characterized in that: In step three, yarn identification and yarn breakage analysis are performed on the final composite image of the entire upper surface of the spliced ​​fabric. First, directional filtering is used to enhance the vertical texture and suppress interference from the tufted needle array. Then, an adaptive threshold segmentation algorithm is used to accurately separate the target yarn region. Yarn quantity statistics are based on connected component analysis. The peak value is locked and the number of peak values ​​is counted by using the longitudinal projection histogram. Morphological closing operations are used to close gaps to eliminate interference from minor breaks. After ensuring the continuity of individual yarns, the total number of connected regions is obtained. Yarn breakage location is achieved by depth traversing the grayscale curve of each column of pixels: for each identified yarn, the grayscale change pattern is fitted in real time along the longitudinal scanning trajectory. When the grayscale of consecutive pixels deviates significantly from the baseline and the width exceeds the set threshold, the breakage edge is confirmed by edge detection, and a yarn breakage can be determined. At the same time, lateral structure analysis is integrated to eliminate interference from folds and wrinkles. Finally, the physical location of the broken yarn is accurately located by the pixel coordinates of the breakage point, and the yarn number and longitudinal offset are output to achieve automated defect detection. When a defect is detected, a stop control operation is performed.

Images