Method for processing digital image of fabric from output of textile machine
By applying neural networks to analyze yarn images on the textile machine, the problem of identifying yarn irregularities and defects in the textile fabric is solved, and real-time control and optimization of the textile process is achieved.
Patent Information
- Application Number
- CN202411954212.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-01-19
- Filing Date
- 2024-12-27
- Publication Date
- 2025-07-22
AI Technical Summary
The prior art is difficult to accurately identify yarn irregularities and defects in textile fabric detection, and it is impossible to achieve real-time control of the textile process.
Through a neural network-based method, the fabric image is obtained using a digital video acquisition device, positioning and classifying yarns, and the important parameters of the fabric are analyzed based on the geometric feature trend of the yarn, including yarn consumption, tension and row length, etc., real-time control is achieved.
It improves the accuracy of yarn irregularity and defect identification, realizes real-time control of the textile process, and optimizes yarn consumption and quality consistency.
Smart Images
Figure CN120355909A_ABST
Abstract
Description
[0001] The present invention relates to a method for processing digital images of fabrics in the output from a textile machine.
[0002] The present invention can be advantageously (but not exclusively) applied to knitted fabrics produced on circular knitting machines.
[0003] It is well known that automated systems for visually inspecting fabrics have been the focus of research for many years and, due to the latest developments in the fields of artificial intelligence and computer vision based on neural networks, they are evolving very rapidly.
[0004] Despite the above, the automatic assessment of fabric properties and the detection of any defects remains a highly demanding task because some visual characteristics (such as the uniformity of the yarn or the surface condition of the fabric) are qualitative aspects of the final product that are difficult to inspect.
[0005] It is well known that, unfortunately, any undetected defects can have a serious impact on the entire production batch.
[0006] In known solutions applied to circular knitting machines, the fabric slides in a continuous rotary motion in front of a digital camera, which is positioned to inspect the front ("technical front") or the back ("technical back") of the fabric in the area close to the forming point of the fabric. In some cases, two cameras facing each other can be used to inspect both sides of the fabric.
[0007] The images captured by the camera can be processed by a computer programmed with an image analysis algorithm using one or more neural networks.
[0008] In known solutions applied to circular knitting machines, the neural networks used benefit from developments in the field of artificial intelligence, particularly in artificial vision, i.e., in the field of computer vision. Over the years, the architectures of the neural networks used have become increasingly complex and high-performance, and the trend of development continues to grow.
[0009] In the most common conventional systems, the neural networks are designed and trained to identify a preset number of defects, such as double threads, holes, dimple points, oil stains, missing Lycra, etc., based on visual features and specific geometries. The conventional systems are also able to identify the approximate location of the defects and frame them in a bounding box within the image.
[0010] Depending on the neural network architecture used, it is also possible to identify in detail (at the pixel level) the location of the defects within the image by means of image segmentation techniques known per se, particularly semantic segmentation techniques. An example of a neural network commonly used to implement semantic segmentation techniques is U-Net.
[0011] As is known, in semantic segmentation techniques, each pixel of a digital image is classified with a label, and portions of the image that contain pixels similar to each other are identified with a "segmentation mask", typically identified with different colors (a different color for each type of mask).
[0012] In addition to directly identifying defects, semantic segmentation techniques are sometimes also used to analyze the structure of a fabric, particularly a fabric produced with a weaving frame, in order to distinguish the weft yarns from the warp yarns. Thus, in such a case, a neural network is capable of using the segmentation mask to distinguish two different families within the image.
[0013] As is known to those skilled in the art, while the latter solution is advantageous, it has limitations because the algorithms (neural networks) used for semantic segmentation can at most distinguish two different families within the image (e.g., weft and warp yarns), and infer the presence of irregularities in the fabric based on the distribution of the two families rather than on the distribution expected to be found in a complete fabric.
[0014] In fact, it is desirable to obtain more detailed data, both for accurately identifying any irregularities or defects and for real-time control of the textile process.
[0015] Therefore, an object of the present invention is to provide a method for processing a digital image of a fabric in the output from a textile machine, which method overcomes the limitations of conventional systems, particularly by providing more detailed data that is useful for the accurate identification of any irregularities or defects and for the real-time control of the textile process.
[0016] The above object and other objects will become more apparent from the following description, and these objects and other objects are achieved by a method having the features described in claim 1 appended hereto, while the appended dependent claims define other features of the present invention, which other features, although minor, are also advantageous.
[0017] The present invention application provides the following:[[]]END]]
[0018] 1). A method for processing a digital image of a fabric in the output from a textile machine, comprising the step of sequentially acquiring a digital image of the area through which the fabric passes by means of a digital video acquisition device, characterized in that, in each acquired digital image, the method comprises:[[]]END]]
[0019] locating and classifying at least one of the yarns forming the fabric in the area by means of a processing device based on at least one neural network, and
[0020] obtaining at least one geometric feature related to an important parameter of interest of the fabric in each of the located and classified yarns,
[0021] And it is characterized in that the method obtains information related to the important parameter of interest based on the trend of the at least one geometric feature in the acquired digital image sequence.
[0022] 2). The method according to 1), characterized in that in each acquired digital image, all the yarns are located, and the dimensions of the yarns transverse to the direction of travel of the fabric are completely included.
[0023] 3). The method according to 1) or 2), characterized in that the yarns are located and classified at the pixel level by means of a segmentation technique, the segmentation technique generates different segmentation masks for each located and classified yarn, and the geometric feature is at least one of the group including the area, perimeter and ratio of area to perimeter of each generated segmentation mask.
[0024] 4). The method according to 3), characterized in that the segmentation technique is an instance segmentation technique.
[0025] 5). The method according to 3), characterized in that the segmentation technique is a panoramic image segmentation technique.
[0026] 6). The method according to 1) or 2), characterized in that the yarns are located and classified within a bounding box enclosing the boundary of the coverage area of the corresponding yarn, and the geometric feature is the path of the yarn determined based on the positions of a set of key points identified within the bounding box.
[0027] 7). The method according to 6), characterized in that the yarns are located and classified by a first neural network, and the key points are identified by a second neural network.
[0028] 8). The method according to 1), characterized in that the important parameter of interest is fabric integrity, and the information is the possible presence of defects.
[0029] 9). The method according to 1), characterized in that the important parameter of interest is yarn consumption, and the information is the trend of yarn consumption during the textile process.
[0030] 10). The method according to 1), characterized in that the important parameter of interest is yarn tension, and the information is the trend of yarn tension during the textile process.
[0031] 11). The method according to 10), characterized in that the information related to the trend of the yarn tension is used to generate a reference tension, and the reference tension is used for the closed-loop control of the tension at which the yarn is fed into the textile machine.
[0032] 12). The method according to 1), characterized in that the important parameter of interest is the row length, and the information is the trend of the row length during the textile process.
[0033] 13). The method according to 1), wherein the continuous knots between the yarns forming the fabric change their configuration in a periodic trend, characterized in that:
[0034] The method configures the parameters of the digital video acquisition device such that each digital image sent to the neural network covers a fabric area that accurately contains a pattern with periodically repeated and identical yarn configurations;
[0035] The method synchronizes the acquisition of digital images with the rotational speed of the textile machine in order to avoid gaps or overlaps between consecutive digital images.
[0036] 14). The method according to 1), wherein the continuous knots between the yarns forming the fabric change their configuration in a periodic trend, characterized in that:
[0037] The method synchronizes the acquisition of digital images with the rotational speed of the textile machine to avoid gaps or overlaps between consecutive digital images;
[0038] The method acquires a sequence of digital images of a sample fabric, the sequence of digital images being a part of the rotation of the textile machine and preferably containing the entire periodically repeated pattern;
[0039] The method derives a sequence of reference values from the sequence of digital images of the sample fabric;
[0040] The method compares the sequence of actual values measured on the sequence of digital images acquired during the textile process with the sequence of reference values measured on the sample fabric;
[0041] If a difference is detected, the method identifies the desired information based on the detected difference.
[0042] 15). The method according to 1), characterized in that the step of locating and classifying at least one of the yarns forming the fabric in the region is obtained as a direct output of the at least one neural network.
[0043] 16). The method according to 1), characterized in that the position of each of the located and classified yarns is plotted using object tracking techniques in order to identify and predict any changes in the position of the yarns in the image sequence.
[0044] 17). According to the method of 1), wherein the textile machine is a circular knitting machine provided with a cylinder, and the fabric advances from the forming point in a helical motion around the axis of the cylinder so as to form a knitted tube, characterized in that the digital video acquisition device comprises a digital camera placed in a stationary position near the forming point of the fabric, and the knitted tube slides in front of the digital camera in a substantially horizontal continuous rotational motion.
[0045] 18). According to the method of 1), wherein the textile machine is a circular knitting machine provided with a cylinder, and the fabric advances from the forming point in a helical motion around the axis of the cylinder so as to form a knitted tube, characterized in that the digital video acquisition device comprises a digital camera that rotates horizontally around the axis of the cylinder synchronously with the circular knitting machine, and the knitted tube slides in front of the digital camera in a vertical continuous linear motion. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The present invention will now be described in more detail with reference to its preferred but non-exclusive embodiments, which are shown in the drawings for the purpose of non-limiting examples, wherein:
[0047] Figure 1 is a schematic view of a device for implementing the method according to the present invention associated with a circular knitting machine.
[0048] Figure 2A 、 Figure 2B and Figure 2C show a part of the fabric taken at three different operating steps during the execution of the method according to the present invention.
[0049] Figure 3 is a schematic view of a yarn supply device using the method according to the present invention for controlling the supply process.
[0050] Figure 4A 、 Figure 4B and Figure 4C show a part of the fabric taken at three different operating steps during the execution of the method according to different embodiments of the present invention.
[0051] Figure 5 shows with respect to Figure 4A 、 Figure 4B and Figure 4C an enlarged part of the fabric.
[0052] Figure 6 shows a first type of fabric to which the method according to the present invention can be applied.
[0053] Figure 7 shows a second type of fabric to which the method according to the present invention can be applied.
[0054] Figure 8 It is a schematic diagram of an alternative embodiment of the method according to the present invention.
[0055] Reference Figure 1 , the automated device 10 for visually inspecting the fabric in a manner known per se includes digital video acquisition means, and in this embodiment, the digital video acquisition means includes a digital camera 12 equipped with a lens 14 and a lighting device 16, and a processing device 18.
[0056] In a conventional manner, the digital camera 12 can be provided with a CCD or CMOS sensor and can be of the type operating in area scan mode (frame camera) or of the type operating in line scan mode (line scan camera). The digital camera 12 can also advantageously be provided with an infrared sensor (IR) capable of operating in the infrared frequency range, or with an RGB-IR sensor capable of recording both visible light and infrared light.
[0057] The lighting device 16 can be of the type commonly used in solutions applied to circular knitting machines and includes light-emitting diodes (LEDs). As is well known, LEDs are low-power, economical and suitable for acquiring images at high frame rates, as in the applications discussed here.
[0058] As will be readily understood by those skilled in the art, if a digital camera operating in the infrared spectrum is used, it is also necessary to select a lighting device with a suitable infrared light source.
[0059] In addition, the camera can be installed together with the lens and the lighting system in a protective housing (not shown). The camera can also be provided with means (not shown) known per se for cleaning the lens (or its protective glass) of the lens assembly, for example by compressed air.
[0060] In this embodiment, the digital camera 12 is housed inside a cylinder C of a circular knitting machine CKM in a stationary position so as to frame an area A of the back surface (technical back) of the fabric F (shown on a screen M associated with the processing device 18), and this area A is close to the forming point of the fabric ( Figure 1 not shown in Figure 1 ). In a conventional manner, the fabric F advances from the forming point in a helical motion around the axis of the cylinder C so as to form a knitted tube, and this knitted tube slides in front of the digital camera 12 with a substantially horizontal continuous rotational motion and at a speed equal to the rotational speed of the circular knitting machine CKM.
[0061] In a conventional manner, the knitted tube is collected by a take-up roller ( Figure 1 not shown in
[0062] The digital camera 12 is of a known type and is capable of sequentially acquiring digital images of the area A through which the fabric passes at a relatively high frame rate. As is well known, a high frame rate makes it possible to prevent image distortion and information loss.
[0063] In a known manner, for example by means of a hardware synchronization device (such as the encoder 20) or a software synchronization device (such as a digital signal exchanged through the API (Application Programming Interface) of the digital camera 12), the acquisition of digital images by the digital camera 12 is synchronized with the rotation of the circular knitting machine CKM.
[0064] In this embodiment, the digital camera 12 is fixed to the structure of the circular knitting machine CKM by means of a support device 22. Advantageously, and in a manner known per se, the support device 22 is configured to allow adjustment of the position of the digital camera 12 along three Cartesian axes and its tilt, rotation and convergence.
[0065] Advantageously, the support device 22 may be provided with a mechanical and / or electronic measurement system which allows precise monitoring of the distance between the digital camera and the fabric to be inspected, for the purpose of precisely calibrating the digital camera.
[0066] In a known manner, for example, from the scientific article "Defect Detection in Polyurethane Fabrics Based on Computer Vision Technology" (2015), the identification of defects in fabrics requires a calibration process of the digital camera, which is capable of establishing a relationship between the measurements (in pixels) made by the camera and the real-world dimensions.
[0067] As is also known, for example, from the scientific article "Calibration of Line Scan Cameras for Fabric Imaging" (2012), calibration of the digital camera is a necessary step in order to obtain the relationship between the coordinates of a three-dimensional object and the coordinates of the image. During the calibration process of the digital camera, this relationship is determined by obtaining internal and external calibration parameters. The internal calibration parameters include the basic intrinsic array of the digital camera (intrinsic calibration), which affects the way the sensor samples the scene. The external parameters indicate the position and orientation of the sensor relative to the global coordinate system (external calibration).
[0068] Over the years, several different camera calibration methods have been developed. For example, the "Zhang method" is one of the most commonly used methods, although there are other methods available for this application.
[0069] The subject of digital camera calibration will not be discussed further, as it is outside the scope of the present invention and, as mentioned above, it forms part of the common knowledge of the person skilled in the art.
[0070] The digital images captured by the digital camera 12 are processed by a processing device 18, which is programmed with an image analysis algorithm using a neural network.
[0071] Still referring now to Figure 2A 、 Figure 2B and Figure 2C , the method according to the present invention requires, in each acquired digital image:
[0072] identifying, by means of a processing device 18 based on at least one neural network, at least one of the yarns Y1 - Y7 forming the fabric F in the region A( Figure 2A ), and
[0073] obtaining, on each of the identified yarns Y1 - Y7, at least one geometric feature related to an important parameter of interest of the fabric F; and
[0074] obtaining information related to the important parameter of interest based on the trend of the geometric features in the acquired sequence of digital images.
[0075] In this specification, the term "identify" or "identifying" in relation to one or more yarns includes the steps of locating and classifying these yarns within the acquired digital image.
[0076] Advantageously, the step of locating and classifying at least one of the yarns Y1 - Y7 forming the fabric F in the region A is obtained as a direct output of the neural network.
[0077] Preferably, all the yarns Y1 - Y7 are identified, the lateral footprint of which with respect to the direction of sliding of the fabric( Figure 2A 、 Figure 2B 、 Figure 2C horizontal direction in) is completely contained within the acquired digital image.
[0078] In a first embodiment, a known instance segmentation technique is used to identify the yarn identifications Y1 - Y7. As is known, in instance segmentation techniques, the pixels of each instance of the digital image (i.e., the "yarns" in this application) are classified with different labels, thus making it possible to distinguish between different instances within the same digital image.
[0079] An example of a known neural network that can be used to implement instance segmentation techniques is Mask R - CNN.
[0080] In this description, the possible architectures of the neural network will not be described in detail, as they are outside the scope of the present invention and are part of the common general knowledge of those skilled in the art.
[0081] Still referring now to Figure 2B and Figure 2C, the neural network generates different segmentation masks SM1 - SM7 for each of the yarns Y1 - Y7 located within region A. In a manner known per se, the segmentation masks consist of digital images having the same dimensions as the starting digital image, but are composed of solid - color regions defining the individual contours of the instances (i.e., the yarns) detected as being at least visible to digital camera 12. In Figure 2B and Figure 2C , the regions belonging to different instances are shown in different shading styles.
[0082] Preferably, object - tracking techniques are used to plot the position of each of the located and classified yarns, which in a manner known per se enables any changes in the position of the yarns in the image sequence to be identified and predicted.
[0083] In this first embodiment, an important parameter of interest is fabric integrity.
[0084] In this case, the geometric features can advantageously be the area and / or perimeter and / or the ratio between the area and the perimeter of each generated segmentation mask SM1 - SM7, advantageously in pixels, which represents the corresponding yarn in the digital image.
[0085] The area can be calculated as the sum of the pixels identified within the segmentation mask. The perimeter can be calculated based on the positions of all the pixels defining the contour of the segmentation mask.
[0086] Once digital camera 12 is calibrated, clearly it will be possible to derive measurements in decimal metric starting from pixel - based measurements.
[0087] As Figure 2B and Figure 2C show, which respectively show a part of a fabric without defects and a part of a fabric with a defect F, any change in the area and / or perimeter and / or the ratio between these two geometric features (e.g., with respect to a nominal value or between consecutive digital images) is correlated with the presence of defect F. Thus, in the presence of such a change, the information obtained is that there is a defect. This information can be used to trigger an alarm.
[0088] Clearly, it is preferable to establish a tolerance interval within which such changes are considered acceptable as they can be attributed to the inherent irregularities of the material being inspected (the yarn).
[0089] In a second embodiment where instance - segmentation techniques are used to re - locate the yarns Y1 - Y7 again, an important parameter of interest is yarn consumption.
[0090] Also in this case, the geometric features can advantageously be the area and / or perimeter and / or the ratio between the area and the perimeter of each generated segmentation mask, advantageously in pixels.
[0091] Any variation in the area and / or perimeter and / or ratio between these two geometric features, e.g., with respect to a nominal value or between consecutive digital images, will act as an indication of the variation in the yarn consumption. Thus, in the presence of such a variation, the information obtained is the trend (in terms of increase, decrease, or constancy) of the yarn consumption during the textile process.
[0092] In a third embodiment where instance segmentation techniques are used again to locate the yarns Y1 - Y7, an important parameter of interest is the yarn tension.
[0093] Also in this case, the geometric features to be considered can be the area and / or perimeter measured in pixels of each segmentation mask, but more advantageously, the ratio between the area and the perimeter is used.
[0094] Any variation in the above - mentioned ratio, e.g., with respect to a nominal value or between consecutive digital images, will act as an indication of the variation in the yarn tension. In particular, a decrease in the ratio between the area and the perimeter of the segmentation mask indicates an increase in the tension of the corresponding yarn due to clamping. In contrast, an increase in the ratio between the area and the perimeter of the segmentation mask indicates a decrease in the tension of the corresponding yarn.
[0095] Thus, in the presence of such a variation, the information obtained is the trend (in terms of increase, decrease, or constancy) of the yarn tension during the textile process.
[0096] As Figure 3 schematically shown, the information related to the trend of the yarn tension can be advantageously used to control, in feedback, the tension applied to the n - th yarn Yn which is part of the fabric F in the monitored area during the textile process. In this regard, referring to Figure 3 , generally, a yarn feeder 100 receives the n - th yarn Yn from a bobbin R. The yarn feeder 100 is provided with tension - controlling means 102 and tension - sensing means 106. The tension - controlling means 102 is configured to vary the tension with which the n - th yarn Yn is delivered to a textile machine 104, and the tension - sensing means 106 is configured to measure the tension of the n - th yarn Yn downstream of the tension - controlling means 102.
[0097] If a positive yarn feeder is used, where the yarn is wound on a motorized bobbin (not shown) which actively delivers the yarn to the textile machine, the tension - controlling means 102 is configured to vary the rotational speed of the bobbin.
[0098] If an accumulative yarn feeder is used, in which the yarn is unwound from a roller (not shown) that passively releases the yarn to the textile machine as required, the device 102 for controlling the tension is configured to brake the yarn in the output from the roller.
[0099] Regardless of the type of yarn feeder, in a manner known per se, the tension of the nth yarn Yn is controlled in a closed loop by comparing the measured tension signal T_meas generated by the tension sensor device 106 with a reference tension T_ref in the first subtractor node 108 to obtain a tension error signal T_err. The tension error signal T_err is sent to the first PID (Proportional Integral Derivative) controller 110, which generates a reference signal Ref_sig for the device 102 for controlling the tension at the output.
[0100] The area of the segmentation mask SMn associated with the nth yarn Yn calculated using the method according to the invention or the calculated area A_calc (preferably after passing through the filter 111) is compared with a reference area A_ref in the second subtractor node 112 to obtain an area error signal A_err. The area error signal A_err is sent to the second PID controller 114, and the second PID controller 114 generates a reference tension T_ref at the output.
[0101] As an alternative to the area, as will be readily understood by those skilled in the art, the perimeter of the segmentation mask or the ratio between the area and the perimeter of the segmentation mask can be used.
[0102] In a fourth embodiment where instance segmentation techniques are used again to locate the yarns Y1 - Y7, an important parameter of interest is the row length.
[0103] As is known, the row length is equal to the length of the yarn obtained by unwinding the loops formed in a complete turn of the knitting machine. Measuring the row length is very important for both meeting quality requirements and saving production costs.
[0104] In this case, the geometric feature can advantageously be the area and / or perimeter and / or the ratio between the area and the perimeter of each segmentation mask measured in pixels.
[0105] Any change in the area and / or perimeter and / or the ratio between the area and the perimeter (e.g., relative to a nominal value or between consecutive digital images) will serve as an indication of a change in the row length.
[0106] In particular, the perimeter can be used to estimate the row length because half of the total perimeter of each mask can be considered a good approximation of the path of the line within the image.
[0107] Thus, in the presence of such variations, the information obtained is the trend of the row length (in terms of increase, decrease, or constancy) during the textile process.
[0108] To keep the row length at a preset value that is substantially constant, a positive yarn feeder associated with or integrated in a circular knitting machine is typically used. When the speed of the knitting machine increases, the speed of the positive yarn feeder also increases, such that the ratio between the amount of yarn and the number of needles remains constant (i.e., a constant row length).
[0109] Thus, precise information about the trend of the row length can be advantageously used to optimize the yarn consumption in the textile process.
[0110] The real-time estimation of the row length can also be used as a feedback control signal in the closed-loop control of the yarn tension, similar to that described previously and Figure 3 shown in, instead of the area and / or perimeter of the segmentation mask.
[0111] In the fifth embodiment, as shown in Figure 4A , Figure 4B , Figure 4C and Figure 5 the important parameter of interest is again the row length.
[0112] In this fifth embodiment, the geometric feature to be obtained is the path B of the yarn ( Figure 4C ), which is determined based on the positions of a set of key points P1, P2, P3, ……, Pn, and the key points P1, P2, P3, ……, Pn are preferably identified within the corresponding bounding box D by a first neural network designed and trained for this purpose, and the bounding box D is preferably generated by a second neural network designed and trained for this purpose, and the bounding box D limits the coverage area of the corresponding yarn within the acquired digital image.
[0113] Obviously, it would be possible to use a single neural network, especially designed and trained to achieve these two goals.
[0114] The path B of the yarn can in turn be obtained using a neural network or by using known interpolation methods (linear, polynomial, spline, etc.), where the equations generated by the interpolation algorithm are used to calculate the row length. For example, if linear interpolation is used, the row length can be calculated as the sum of the distances between the key points identified as P1, P2, P3, ……, Pn.
[0115] It should be noted that as Figure 5As shown, key points can be located along the central axis of the yarn as P1, P2, P3, …, Pn (hollow points), on the upper edge of the yarn profile as P1', P2', P3', …, Pn' (solid points), on the lower edge of the yarn profile as P1'', P2'', P3'', …, Pn'' (cross-hatched points), or at any other points included between the upper and lower edges of the yarn profile.
[0116] In the above embodiment, reference has been made to a "single jersey" fabric, where the continuous loops formed between the yarns are all identical to each other, as Figure 6 shown, where a single jersey SJT part of the fabric is shown. However, the method according to the present invention can also be applied to fabrics with complex patterns, such as jacquard fabrics, where the continuous knots between the yarns change their configuration in a periodic trend, as Figure 7 shown, which shows a jacquard JQT part of a fabric containing a periodically repeating pattern.
[0117] To apply the method according to the present invention to complex patterns, the first possibility requires:
[0118] Configuring the parameters of the digital camera such that each digital image sent to the neural network covers a fabric area that accurately contains a pattern with periodic repetition and having the same yarn configuration;
[0119] Synchronizing the acquisition of digital images with the rotational speed of the textile machine in order to avoid gaps or overlaps between consecutive digital images.
[0120] If the area of the fabric containing the periodically repeating pattern is too large to be captured in a single digital image, the second possibility specifically pointed out is based on the assumption that the measured values (area and / or perimeter and / or ratio between area and perimeter and / or estimation of row length) performed on the digital image always repeat in the same order for each part of the machine rotation.
[0121] The second possibility requires:
[0122] Acquiring a sequence of digital images of a sample fabric, where the sequence of digital images of the sample fabric refers to a part of the rotation of the textile machine and accurately contains a periodically repeating pattern;
[0123] Deriving a sequence of reference values from the sequence of digital images of the sample fabric;
[0124] Comparing the sequence of actual values measured on the sequence of digital images acquired during the textile process with the sequence of reference values measured on the sample fabric;
[0125] Identifying the desired information (presence of defects, change in tension, etc.) based on the detected differences (if any).
[0126] Obviously, in order for the comparison to be meaningful, the size of the digital image acquired on the sample fabric must be the same as the size of the digital image acquired during the textile process.
[0127] In this case, the acquisition of the digital image also needs to be synchronized with the rotational speed of the machine in order to avoid gaps or overlaps between consecutive digital images during the sampling step and during the textile process.
[0128] If it is not possible (or optimal) to ensure that the completed image sequence accurately covers the periodically repeating pattern, the same method can be applied to different parts of the knitted fabric. The maximum limit will be when the following reference part of the fabric is reached: the reference part is equal to the complete (helical) circumference of the knitting tube, or one complete rotation of the machine.
[0129] In fact, in this case, with the important parameters of interest remaining unchanged, the measurements performed on the sample fabric should also be consistent with the measurements performed during the textile process.
[0130] According to another variant of the present invention, in this case specifically referring to the circular knitting machine and as Figure 8 schematically shown in, the digital camera 1012 is not stationary, but rotates horizontally around the axis Z of the cylinder, as Figure 8 indicated by the arrow D in, in synchronization with the machine. For example, the digital camera 1012 can be mechanically coupled to the rotating part of the machine, or moved in a mechanically axially aligned or electrically axially aligned manner, in synchronization with the machine.
[0131] In this case, the digital camera 1012 acquires images of the fabric only for a limited part of the machine circumference corresponding to the horizontal aperture of the field of view V of the digital camera 1012.
[0132] However, by means of a helical movement (shown by Figure 8 S in) around the axis Z of the cylinder (not shown in Figure 8 ), the fabric is released from the forming point P where the knitting tube U is generated by this helical movement. In this case, the knitting tube also slides in front of the digital camera 1012, but with a continuous vertical (instead of substantially horizontal) linear movement from top to bottom, and at a speed equal to the winding speed of the take-up roller W. As previously mentioned, the take-up roller W rotates around the horizontal axis X and collects the fabric at the base of the knitting tube U. Therefore, in this case, all the yarns fed into the machine slide repeatedly in front of the digital camera 1012 in a continuous ordered sequence from the first yarn to the last yarn.
[0133] Therefore, in this case, all the yarns can also be identified because they sequentially appear in front of the sensor of the digital camera 1012 in the monitored part of the knitting tube U.
[0134] Due to the continuous repetition of the yarn sequence, it is advantageous in this case that:
[0135] Configure the parameters of the digital camera such that each digital image sent to the neural network covers a fabric area that accurately contains all the yarns (or a finished part of the total number of yarns) fed into the machine;
[0136] Synchronize the acquisition of digital images with the winding speed of the winding roller W to avoid gaps or overlaps between consecutive digital images;
[0137] When necessary (and preferably, if the previous steps are not applicable), use object tracking techniques to track the positions of all the identified yarns in order to identify and predict changes in the positions of the yarns in the acquired image sequence.
[0138] For each complete rotation of the machine, measurements can be performed on each yarn "by sampling" using the same method as previously described, but only for the fabric part seen by the digital camera 1012.
[0139] These measured values can also be used to adjust the reference tension in the yarn feeder in a manner similar to that described in the third embodiment shown in Figure 3 To monitor the entire circumference of the knitted tube U, a plurality of aligned digital still cameras are required, one next to the other and supported by a structure rotating inside the cylinder of the circular knitting machine.
[0140] Obviously, in this case, the still cameras can also be positioned outside rather than inside the knitted tube U, below the cylinder, in order to frame the technical front of the fabric, and they can be coupled to the rotating parts of the machine or move in synchronization with the machine.
[0141] The design and training of the neural network for performing the method according to the present invention belong to the normal knowledge of those skilled in the art and will not be discussed in more detail here.
[0142] Some preferred embodiments of the present invention have been described, but obviously, those skilled in the art can make various modifications and variations within the scope of protection of the claims.
[0143] For example, in embodiments not described in detail, the measurement of the positions of key points and the line length can also be used to identify defects in the fabric, for example by verifying that the key points are equally spaced within the image, and any significant deviation between consecutive images can be used to trigger an alarm.
[0144]
[0145] In other embodiments not described, the row length measurements estimated based on the positions of the key points can be used to adjust key parameters in real time, such as yarn tension, fabric take-up tension, sinker height, and the rotational speed of the textile machine.
[0146] In particular, the row length measurements estimated based on the positions of the key points can also be used to adjust the reference tension in the yarn feeder in a manner similar to that described for the third embodiment shown in Figure 3 . The only difference is that, in the second subtractor node, instead of comparing the calculated area with the reference area, the estimated row length is compared with the reference row length in order to obtain a row length error signal to be sent to the second PID controller in order to generate the reference tension at the output.
[0147] Obviously, neural networks with different architectures can be used to identify the yarn, which are capable of using different segmentation techniques but still able to obtain results equivalent to the instance segmentation techniques of the embodiments described herein. For example, it is possible to use panoramic segmentation techniques known per se, which consist of a combination of semantic segmentation and instance segmentation techniques.
[0148] The position of the digital video acquisition device can also vary with respect to the embodiments described and explained herein. For example, the digital camera can be mounted below the cylinder, outside the textile machine, in order to inspect the front (technical front) of the fabric.
[0149] In this regard, in a manner known per se, the digital video acquisition device can be coupled to both the structure of the textile machine and a dedicated structure positioned outside the perimeter of the textile machine.
[0150] Furthermore, as described above, two digital still cameras facing each other can be provided for inspecting both the front and the back of the fabric.
[0151] Last but not least, although only circular knitting machines have been referred to in this specification and the method according to the invention is particularly advantageous for circular knitting machines, it is obvious that it can also be applied to different types of textile machines, such as warp knitting machines, flat knitting machines, or warp and weft knitting machines.
Claims
1. A method for processing digital images of a fabric in an output from a textile machine, comprising the step of sequentially acquiring digital images of a region (A) through which the fabric (F) passes by means of a digital video acquisition device (12, 14, 16), characterized in that, In each acquired digital image, the method comprises: locating and classifying at least one of the yarns (Y1 - Y7) forming the fabric (F) in the region (A) by means of a processing device (18) based on at least one neural network, and for each of the located and classified yarns (Y1 - Y7), obtaining at least one geometric feature related to an important parameter of interest of the fabric, and wherein the method obtains information related to the important parameter of interest based on the trend of the at least one geometric feature in the acquired sequence of digital images.
2. The method according to claim 1, wherein In each acquired digital image, all the yarns (Y1 - Y7) are located, the dimensions of which are transverse to the direction of travel of the fabric are fully contained.
3. The method according to claim 1 or 2, characterized in that, The yarns are located and classified at the pixel level by means of a segmentation technique, the segmentation technique generating a different segmentation mask (SM1 - SM7) for each located and classified yarn (Y1, Y7), and the geometric feature being at least one of the group comprising the area, perimeter and ratio of area to perimeter of each generated segmentation mask.
4. The method according to claim 3, wherein The segmentation technique is an instance segmentation technique.
5. The method according to claim 3, wherein The segmentation technique is a panoramic image segmentation technique.
6. The method according to claim 1 or 2, characterized in that, The yarns (Y1 - Y7) are located and classified within a bounding box (D) surrounding the boundary of the coverage area of the corresponding yarn, and the geometric feature is the path of the yarn determined based on the positions of a set of key points (P1, P2, P3, ……, Pn) identified within the bounding box (D).
7. The method according to claim 6, wherein The yarns (Y1 - Y7) are located and classified by a first neural network, and the key points (P1, P2, P3, ……, Pn) are identified by a second neural network.
8. The method according to claim 1, characterized in that, The important parameter of interest is fabric integrity, and the information is the possible presence of defects.
9. The method according to claim 1, characterized in that, The important parameter of interest is yarn consumption, and the information is the trend of yarn consumption during the textile process.
10. The method according to claim 1, characterized in that The important parameter of interest is yarn tension, and the information is the trend of yarn tension during the textile process.
11. The method according to claim 10, wherein The information related to the trend of the yarn tension is used to generate a reference tension (T_ref), which is used for the closed-loop control of the tension of the yarn fed to the textile machine (104).
12. The method according to claim 1, characterized in that, The important parameter of interest is row length, and the information is the trend of row length during the textile process.
13. The method according to claim 1, wherein, The continuous knots between the yarns forming the fabric change their configuration in a periodic trend, characterized in that: The method configures the parameters of the digital video acquisition device such that each digital image sent to the neural network covers a fabric area accurately containing a pattern that is periodically repeated and has the same yarn configuration; The method synchronizes the acquisition of digital images with the rotational speed of the textile machine in order to avoid gaps or overlaps between consecutive digital images.
14. The method according to claim 1, wherein, The continuous knots between the yarns forming the fabric change their configuration in a periodic trend, characterized in that: The method synchronizes the acquisition of digital images with the rotational speed of the textile machine to avoid gaps or overlaps between consecutive digital images; The method obtains a digital image sequence of a sample fabric, the digital image sequence being a part of the rotation of the textile machine and preferably containing the entire periodically repeating pattern; The method derives a reference value sequence from the digital image sequence of the sample fabric; The method compares a sequence of actual values measured on a digital image sequence obtained during the textile process with the sequence of reference values measured on the sample fabric; If a difference is detected, the method identifies the desired information based on the detected difference.
15. The method according to claim 1, wherein The step of locating and classifying at least one of the yarns (Y1 - Y7) forming the fabric (F) in the region (A) is obtained as a direct output of the at least one neural network.
16. The method according to claim 1, wherein Object tracking techniques are used to plot the position of each of the located and classified yarns in order to identify and predict any changes in the position of the yarns in the image sequence.
17. The method according to claim 1, wherein, The textile machine is a circular knitting machine provided with a cylinder (C), the fabric (F) advancing from a forming point in a helical motion around the axis of the cylinder (C) so as to form a knitted tube, characterized in that the digital video acquisition device comprises a digital camera (12) placed in a stationary position near the forming point of the fabric (F), the knitted tube sliding in front of the digital camera (12) in a substantially horizontal continuous rotational motion.
18. The method according to claim 1, wherein The textile machine is a circular knitting machine provided with a cylinder, the fabric advancing from a forming point (P) in a helical motion around the axis (Z) of the cylinder so as to form a knitted tube (T), characterized in that the digital video acquisition device comprises a digital camera (1012) that rotates horizontally around the axis (Z) of the cylinder synchronously with the circular knitting machine, the knitted tube (T) sliding in front of the digital camera (1012) in a vertical continuous linear motion.