A machine vision-based electromagnetic needle selector trajectory measurement method

Abstract

This invention discloses a machine vision-based method for measuring the trajectory of an electromagnetic needle selector. The method includes: acquiring a continuous image sequence of the needle selector during operation using a high-speed industrial camera; adaptively cropping the Region of Interest (ROI) using sparse optical flow fusion and image differencing; performing Otsu thresholding, connected component extraction, minimum bounding rectangle fitting, and gray-level centroid calculation on each needle selector region within the ROI to extract feature points of multiple needle selectors; constructing an inter-frame feature matching and trajectory reconstruction algorithm to continuously identify and sort the temporal trajectory of each needle selector; and finally outputting a complete and smooth multi-target motion trajectory. Compared to traditional single-point measurement methods, this method has advantages such as non-contact operation, high concurrency, and high precision, and is suitable for applications such as electromagnetic needle selector status assessment, fault detection, and performance analysis.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision and intelligent manufacturing, specifically relating to a machine vision-based method for measuring the motion trajectory of an electromagnetic needle selector in knitting machinery, which is particularly suitable for synchronous monitoring and performance evaluation of the high-speed oscillation behavior of multi-select needle pieces in circular knitting machines. Background Technology

[0002] The electromagnetic needle selector is the core actuator of a circular knitting machine, enabling jacquard knitting. It uses electromagnetic drive to control the periodic oscillation of the needle selector plates, thus determining the selection of needles and the loop formation path. The accuracy of the needle selection directly affects fabric quality and equipment operational stability. However, during long-term operation, factors such as coil aging, magnetic core demagnetization, and mechanical wear can easily lead to problems like response delays, insufficient oscillation amplitude, and incomplete return to position, resulting in misaligned patterns, broken needles, or even machine shutdowns. Existing detection methods have significant limitations: laser displacement measurement methods can only acquire the displacement of a single cutter head, making it difficult to achieve simultaneous monitoring of multiple needles; while image processing-based detection systems often focus on specific fault mode identification, lacking the ability to reconstruct complete trajectories and failing to support multi-type anomaly analysis and performance evaluation. Therefore, there is an urgent need for a measurement method that can simultaneously acquire the complete motion trajectories of multiple needle selector plates, providing technical support for overall needle selector health status assessment, early fault warning, and intelligent operation and maintenance. Summary of the Invention

[0003] The purpose of this invention is to provide an efficient, accurate, and scalable machine vision-based method for measuring the trajectory of an electromagnetic needle selector, addressing the problems of difficulty in simultaneous multi-target monitoring, incomplete trajectory reconstruction, and weak anti-interference capability in existing technologies. By integrating sparse optical flow fusion and image difference techniques, combined with adaptive ROI extraction, Otsu threshold segmentation, and grayscale centroid localization, non-contact, high-concurrency, and high-precision synchronous measurement of the motion trajector trajector of multiple needle selectors is achieved, supporting performance evaluation, anomaly localization, and intelligent diagnosis of the needle selector system.

[0004] To achieve the above objectives, the present invention proposes the following technical solution.

[0005] S1. Acquire a continuous image sequence of the electromagnetic needle selector during its operation using a high-speed industrial camera under low-exposure conditions. ,in This represents the grayscale image of frame t; S2. Perform grayscale conversion and mean filtering preprocessing on the image sequence sequentially to obtain the preprocessed image. For preprocessed images Perform sparse optical flow fusion and image difference processing to extract motion feature points in the target region and automatically generate the region of interest (ROI). S3. Perform Gaussian filtering on the region of interest (ROI) and use Otsu thresholding and connected component analysis to extract candidate regions for multiple needle selection pieces. S4. For each candidate region Perform minimum bounding rectangle fitting, and calculate the feature point coordinates of each needle selection piece in its corresponding original grayscale sub-image based on the grayscale centroid method. ; S5. Construct an inter-frame feature matching and trajectory reconstruction algorithm to output the temporal motion trajectory of the multi-target needle selection piece.

[0006] Preferably, the sparse optical flow fusion processing in step S2 involves extracting an initial feature point set using the Shi-Tomasi corner detection algorithm. ,in The pyramid-shaped Lucas-Kanade sparse optical flow algorithm is employed. Track to the next frame Obtain the matching point set and state vector ; retain valid matching points ,based on The initial region of interest (ROI) 0 is calculated, and its boundary coordinates are determined as follows: ; in, This represents the minimum x-coordinate of all valid matching points. This represents the minimum value of the y-coordinates of all valid matching points. The maximum x-coordinate of all valid matching points. The maximum ordinate of all valid matching points, W and H are the image width and height, and 10 is the number of pixels for boundary expansion; Preferably, the image difference processing in step S2 involves performing Gaussian filtering on the sub-image distributions of two adjacent frames within the ROI0 region, with a size of 9x9 and a standard deviation of [missing value]. =0, thus obtaining and Calculate the absolute image difference D: ; in, yes Sub-image after cropping ROI0 and applying Gaussian filtering; yes Sub-image after cropping ROI0 and applying Gaussian filtering; Binarize D and set a threshold. ;

[0007] Then, perform closing and opening operations on B in sequence, resulting in a 5x5 matrix of all 1s for the structuring element K:

[0008] according to The spatial distribution of connected regions (e.g., concentrated on the right side) allows for the precise selection of the Region of Interest (ROI).

[0009] Preferably, in step S3, the optimal binarization threshold T is calculated using Otsu thresholding, where T is the gray level k that maximizes the inter-class variance. The inter-class variance is calculated using the following formula:

[0010] in, This represents the percentage of pixels with a grayscale value ≤ k. This represents the percentage of pixels with a grayscale value greater than k. The average gray level of pixels with gray values ​​≤ k. The average gray level of pixels with a gray value greater than k; Binarize the Region of Interest (ROI) image according to T. The optimal threshold T is determined by the following formula: .

[0011] Preferably, in step S3, all possible target regions are extracted through connected component analysis, and effective candidate regions that meet the morphological characteristics of the needle piece are selected based on preset geometric constraints (such as area range, aspect ratio, and position distribution).

[0012] Preferably, in step S4, the feature point coordinates of each needle selection piece are calculated based on the gray-level centroid method. Let the gray-level value of each pixel (i, j) in the image be g(i, j), then the gray-level centroid coordinates of the image are... Defined as: g(i,j) represents the gray value of the pixel located in the i-th row and j-th column; The numerator represents the weighted sum of column index j and row index i with the gray value; Denominator: The sum of all pixel grayscale values; Preferably, in step S5, an inter-frame feature matching and trajectory reconstruction algorithm is constructed to... The feature points of each needle selection piece are sorted by size using an array correspondence method, and then stored in the array in sequence for subsequent trajectory curve creation. When the number of detected feature points in a certain frame is insufficient, sorting correction or interpolation compensation is performed based on historical trajectory trends and the arrangement order of the needle selection pieces to ensure the consistency of each trajectory number.

[0013] The beneficial effects of this invention are as follows: This invention uses an industrial camera to acquire image sequences and combines sparse optical flow fusion and image difference technology to dynamically generate ROIs, achieving synchronous, non-contact, and dynamic measurement of the motion trajectories of all 16 needle selectors. This completely overcomes the limitations of traditional sensors or single-point vision methods that can only perform "single-point" measurements. Based on the acquired high-precision, time-series motion trajectory data, this invention can systematically extract multiple key dynamic performance parameters such as swing amplitude, response time, synchronization, phase difference, and motion stability. This provides a multi-dimensional and quantifiable data foundation for the status evaluation of the needle selector. This invention overcomes the shortcomings of existing methods that typically only address a specific fault, by comprehensively analyzing... By analyzing the aforementioned multi-dimensional motion parameters, it is possible to accurately identify and differentiate various faults such as wear, fatigue, inconsistent electromagnetic coil characteristics, mechanical jamming, and assembly errors. This significantly improves the comprehensiveness, coverage, and practicality of fault diagnosis. Through trend analysis and threshold warning of motion parameters, this invention can detect early signs of faults (such as slow decay of swing amplitude, gradual extension of response time, and slight deterioration of synchronicity) before they have a substantial impact on equipment performance. This allows the equipment maintenance mode to be upgraded from passive "post-event maintenance" or periodic "preventive maintenance" to proactive "predictive maintenance," thereby greatly improving equipment reliability, reducing unplanned downtime, and lowering overall maintenance costs. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the working method of the present invention.

[0015] Figure 2 Comparison of displacement measurement based on vision and sensor at a 100 Hz operating frequency for the Type A needle selector.

[0016] Figure 3 A magnified view comparing the displacement measurements of the Type A needle selector based on vision and sensors at a working frequency of 100 Hz (A).

[0017] Figure 4 A magnified view comparing the displacement measurements of the Type A needle selector based on vision and sensors at a working frequency of 100 Hz (B). Detailed Implementation

[0018] The following is in conjunction with the appendix Figure 1 - Appendix Figure 4 The specific embodiments of the present invention will be further described in detail to make the technical solution of the present invention easier to understand and master. This embodiment aims to achieve synchronous, non-contact, and high-precision motion trajectory extraction of the high-speed oscillation behavior of multiple needle selection pieces in a circular knitting machine, and to support stable operation and anomaly identification under actual working conditions (such as a working frequency of 100 Hz).

[0019] In this embodiment, a dedicated motion trajectory measurement experimental device was constructed, which mainly consists of the following components: computer, PCO.dimax HS4 high-speed camera (resolution of 1008px×888px, sampling frequency of 4000 Hz, video output frame rate of 200 fps), needle selector driver, PDL-030-485 laser displacement sensor, and A-type needle selector (containing a total of 16 needle selector blades).

[0020] The method of this invention mainly includes five core steps: S1. Capture the dynamic process of the A-type needle selector under normal working conditions in real time using a high-speed industrial camera under low exposure conditions, and collect a continuous image sequence of the needle selector during its working process. S2. Perform grayscale conversion and mean filtering on the continuous image sequence obtained in step S1 to obtain the preprocessed image. For preprocessed images Sparse optical flow fusion and image difference processing were performed; the Shi-Tomasi corner detection algorithm was used to extract the initial feature point set. ,in The pyramid-shaped Lucas-Kanade sparse optical flow algorithm is employed. Track to the next frame Obtain the matching point set and state vector ; retain valid matching points ,based on Calculate the initial region of interest (ROI) 0， Its boundary coordinates are determined as follows: ; in This represents the minimum x-coordinate of all valid matching points. This represents the minimum value of the y-coordinates of all valid matching points. The maximum x-coordinate of all valid matching points. The maximum ordinate of all valid matching points, W and H are the image width and height, and 10 is the number of pixels for boundary expansion; Within the ROI0 region, Gaussian filtering is applied to the distribution of sub-images between two adjacent frames, with a size of 9x9 and a standard deviation of [missing value]. =0, thus obtaining and Calculate the absolute image difference D: ;

[0021] in, yes Sub-image after cropping ROI0 and applying Gaussian filtering; yes Sub-image after cropping ROI0 and applying Gaussian filtering; Binarize D and set a threshold. If D(x,y) > 17, then B(x,y) = 225; otherwise, it is 0. Then, perform closing and opening operations on B sequentially. The structuring element K is a 5x5 matrix of all 1s.

[0022] according to Spatial distribution of connected regions (e.g., concentrated on the right side) allows for the precise selection of the Region of Interest (ROI). S3. Perform Gaussian filtering on the ROI image obtained in step S2 to reduce noise interference. Binarize the ROI image using Otsu thresholding. Otsu thresholding (T) accurately calculates the threshold value based on the image's grayscale histogram, achieved by maximizing the inter-class variance. The inter-class variance is defined as:

[0023] in, This represents the percentage of pixels with a grayscale value ≤ k. This represents the percentage of pixels with a grayscale value greater than k. The average gray level of pixels with gray values ​​≤ k. The average gray level of pixels with a gray value greater than k.

[0024] Binarize the Region of Interest (ROI) image according to T. The optimal threshold T is determined by the following formula:

[0025] All possible target regions are extracted through connected component analysis, and effective candidate regions that meet the morphological characteristics of the needle selection piece are selected based on preset geometric constraints (such as area range, aspect ratio, and position distribution). S4. Fitting the candidate region with the minimum bounding rectangle helps to recover the region ignored by Otsu threshold segmentation in the previous step, further restoring the complete information of the feature point region of the A-type needle selector; the feature point coordinates of each needle selector are calculated based on the gray-level centroid method. Let the gray-level value of each pixel (i, j) in the image be g(i, j), then the gray-level centroid coordinates of the image are... Defined as: g(i,j) represents the gray value of the pixel located in the i-th row and j-th column; The numerator represents the weighted sum of column index j and row index i with the gray value; Denominator: The sum of all pixel grayscale values; S5. Construct an inter-frame feature matching and trajectory reconstruction algorithm, to The algorithm sorts the feature points by size and uses an array mapping method to store the feature points of each needle selection piece in an array in sequence, which is then used to build the trajectory curve. When the number of detected feature points in a frame is insufficient, the algorithm performs sorting correction or interpolation compensation based on historical trajectory trends and the arrangement order of the needle selection pieces to ensure the consistency of each trajectory number. The algorithm can accurately use each extracted feature point to construct a complete trajectory curve for subsequent fault diagnosis.

[0026] In this embodiment, a comparative experiment was conducted to verify the accuracy and practicality of the method described in this invention. Based on the structural conditions of the type A needle selector, and using the theoretical displacement curve generated under electrical signal drive as a reference, the visual measurement results obtained based on the method described in this invention were quantitatively compared with the measurement results of the laser displacement sensor, and its applicability at a 100Hz operating frequency was evaluated. The root mean square error (RMSE) and coefficient of determination (R²) were used as evaluation indicators to measure the consistency between the data obtained by each measurement method and the theoretical displacement curve.

[0027] The RMSE of the visual measurement results compared to the theoretical displacement curve was 0.042 mm, with an R² of 0.99; the RMSE of the laser displacement sensor measurement results compared to the theoretical displacement curve was 0.239 mm, with an R² of 0.96. These results demonstrate that the visual measurement results are in high agreement with the theoretical displacement curve, with a significantly lower error than the laser displacement sensor measurement results, and an R² close to 1, verifying the high accuracy and reliability of the method described in this invention. Furthermore, at a working frequency of 100 Hz, the dynamic change trend of the A-type needle selector captured by the method described in this invention remains highly consistent with the theoretical displacement curve, further proving that the method described in this invention has good applicability and robustness under actual working conditions.

[0028] In summary, this invention provides a non-contact, high-concurrency, and high-precision electromagnetic needle selector trajectory measurement solution, breaking through the technical bottleneck of traditional single-point measurement and possessing good engineering practicality and industrialization promotion value. The specific embodiments described above are merely preferred embodiments of this invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A method for measuring the trajectory of an electromagnetic needle selector based on machine vision, characterized in that, The method includes the following steps: S1. Acquire a continuous image sequence of the electromagnetic needle selector during its operation using a high-speed industrial camera under low-exposure conditions. ,in This represents the grayscale image of frame t; S2. Perform grayscale conversion and mean filtering preprocessing on the image sequence sequentially to obtain the preprocessed image. For preprocessed images Perform sparse optical flow fusion and image difference processing to extract motion feature points in the target region and automatically generate the region of interest (ROI). S3. Perform Gaussian filtering on the region of interest (ROI) and use Otsu thresholding and connected component analysis to extract candidate regions for multiple needle selection pieces. S4. For each candidate region Perform minimum bounding rectangle fitting, and calculate the feature point coordinates of each needle selection piece in its corresponding original grayscale sub-image based on the grayscale centroid method. ; S5. Construct an inter-frame feature matching and trajectory reconstruction algorithm to output the temporal motion trajectory of the multi-target needle selection piece.

2. The method according to claim 1, characterized in that, The sparse optical flow fusion process in step S2 involves extracting an initial feature point set using the Shi-Tomasi corner detection algorithm. ,in ; The pyramid-shaped Lucas-Kanade sparse optical flow algorithm is adopted. Track to the next frame Obtain the matching point set and state vector ; retain valid matching points ,based on The initial region of interest (ROI) 0 is calculated, and its boundary coordinates are determined as follows: ； in, This represents the minimum x-coordinate of all valid matching points. This represents the minimum value of the y-coordinates of all valid matching points. The maximum x-coordinate of all valid matching points. The maximum ordinate of all valid matching points, W and H are the image width and height, and 10 is the number of pixels for boundary expansion.

3. The method according to claim 1, characterized in that, The image difference processing in step S2 involves applying Gaussian filtering to the sub-image distributions of two adjacent frames within the ROI0 region. The sub-images are 9x9 in size and have a standard deviation of [missing value]. =0, thus obtaining and Calculate the absolute image difference D: ； in, yes Sub-image after cropping ROI0 and applying Gaussian filtering; yes Sub-image after ROI0 cropping and Gaussian filtering.

4. The method according to claim 1, characterized in that, In step S3, the optimal binarization threshold T is calculated using Otsu thresholding, where T is the gray level k that maximizes the inter-class variance. The formula for calculating the inter-class variance is: ; in, This represents the percentage of pixels with a grayscale value ≤ k. This represents the percentage of pixels with a grayscale value greater than k. The average gray level of pixels with gray values ​​≤ k. The average gray level of pixels with a gray value greater than k; Binarize the Region of Interest (ROI) image according to T. The optimal threshold T is determined by the following formula: .

5. The method according to claim 1, characterized in that, In step S3, all possible target regions are extracted through connected component analysis, and effective candidate regions that meet the morphological characteristics of the needle selection piece are selected based on preset geometric constraints.

6. The method according to claim 5, characterized in that, The geometric constraints are one or more of the following: area range, aspect ratio, and positional distribution.

7. The method according to claim 1, characterized in that, The feature point coordinates of each needle selection piece mentioned in step S4 The calculation formula is: in This represents the grayscale value of the pixel located in the i-th row and j-th column; The numerator represents the weighted sum of column index j and row index i with the grayscale value; Denominator: The sum of all pixel grayscale values.

8. The method according to claim 1, characterized in that, In step S5, for cases where feature points are missing between frames, sorting correction and interpolation compensation strategies are introduced to ensure the continuity and integrity of the trajectory.

9. The method described in any one of claims 1-8, or the time-series motion trajectory of the multi-target needle selector obtained therefrom, is used in the overall health status assessment, early fault warning, and / or intelligent operation and maintenance of electromagnetic needle selectors.

Images