Cloth stitch jumping and missing detection method

Abstract

The invention discloses a cloth stitch jumping and missing detection method, which comprises the following steps of: acquiring a stitch image in real time through an industrial camera, extracting all stitches in the stitch image, calculating a stitch length between the stitches, and identifying an image abnormal sewing type and a probability thereof through the stitch length; collecting a vibration signal on the needle head connecting rod through a vibration sensor, and inputting the vibration signal into a pre-trained vibration abnormality recognition model to obtain a vibration abnormality sewing type and a probability thereof; hash bucket division is carried out on the image abnormal sewing type and the probability thereof and the vibration abnormal sewing type and the probability thereof, DS theory fusion is carried out in the same Hash bucket to judge the current abnormal sewing type, and early warning is carried out according to the abnormal sewing type.

Description

Technical Field

[0001] This invention relates to the field of needle skipping detection technology, and in particular to a method for detecting needle skipping in fabric. Background Technology

[0002] With the increasing prevalence of passive safety systems in automobiles, the sewing quality of critical safety components such as airbags has a crucial impact on the overall vehicle safety performance. The main quality and safety hazards in the sewing process come from: Missing needles: In complex areas such as intersections / overlaps, due to the thickening of the fabric or mechanical malfunctions of the equipment, soft spots where the needles do not penetrate completely are very likely to occur, leading to localized tearing of the finished product under pressure; Broken threads and skipped needles: If the continuity of needles is interrupted or the needles are misaligned in densely packed workstations, stress will accumulate, the overall strength will be weakened, and it will easily evolve into a safety failure point.

[0003] Currently, the sewing quality inspection of critical components such as airbags mainly relies on manual visual inspection. A typical process includes: after the finished product comes off the line, quality inspectors observe the appearance of the stitches, manually check the integrity of the stitches, the uniformity of the stitches, and for any obvious broken or missing needles. Some companies have introduced preliminary machine vision inspection as a supplement, mainly using industrial cameras to capture sewing images and using traditional image processing methods to analyze the sewing trajectory and stitch characteristics.

[0004] Traditional machine vision solutions rely on single image features, making them susceptible to limitations in extracting stitch features from complex sewing patterns (such as multiple lines, intersections, and overlaps). They are also sensitive to environmental factors such as changing lighting conditions, material gloss, and background noise, resulting in insufficient stability and accuracy to meet the demands of high-safety-level products. Furthermore, relying solely on image information cannot detect hidden or dynamic anomalies caused by mechanical component malfunctions (such as needle bar jamming, structural loosening, or sudden needle breakage), leading to blind spots and potential missed or delayed alarms.

[0005] Furthermore, as production line equipment scales up, traditional detection systems relying on single servers or simple dual-channel data splitting struggle to guarantee real-time, independent, and continuous detection of multiple sewing machines under high-concurrency data streams. Centralizing all equipment data processing can easily lead to computational bottlenecks, data congestion, disordered equipment status information, and confusion regarding event attribution. On the other hand, simple polling and data splitting methods cannot guarantee the unique aggregation of the historical states of the same device at various times, affecting the accuracy of anomaly detection and behavior tracing.

[0006] The purpose of this invention is to design a fabric skipping needle detection method to address the problems existing in the prior art. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a method for detecting skipped needles in fabric, which can solve the above-mentioned problems.

[0008] This invention provides a method for detecting skipped needles in fabric, comprising: The sewing images are captured in real time by an industrial camera. All stitches in the sewing images are extracted, the stitch distance between stitches is calculated, and the abnormal sewing type and its probability are identified by the stitch distance. Vibration signals are collected from the needle connecting rod by a vibration sensor, and the vibration signals are input into a pre-trained vibration anomaly recognition model to obtain the type of vibration anomaly sewing and its probability. The image abnormal stitching type and its probability and the vibration abnormal stitching type and its probability are divided into hash buckets. Within the same hash bucket, DS theory fusion is performed to determine the current abnormal stitching type, and an early warning is issued based on the abnormal stitching type.

[0009] The beneficial effects of this invention are: First, by using industrial camera images and performing grayscale conversion, noise reduction, and morphological optimization, it can accurately separate the pin outline and geometric center under complex lighting and interference environments, greatly reducing the problems of misjudgment and missed detection in traditional manual inspection. Second, by fitting the main axis and sorting the pin centroids, it realizes an automatic statistical algorithm for adjacent pin distances. Third, it supports conditional judgment of anomalies such as skipped pins, missing pins, and broken pins.

[0010] Secondly, the vibration signal is highly sensitive and can capture physical phenomena such as needle bar jamming, loosening, and abnormal noises in mechanical parts. These can still be perceived normally when the image is occluded, blurred, or in extremely poor lighting, avoiding visual misjudgment and omission. Using historical synchronous image annotations as teachers, a multi-classification neural network for anomaly types is trained. The vibration signal input is the corresponding multi-type anomaly distribution, realizing comprehensive knowledge transfer and automatic annotation, and improving the ability to distinguish anomaly types.

[0011] Third, by utilizing DS theory to fuse image and vibration detection results, the system scientifically handles their respective uncertainties and conflicts. It can automatically quantify the risk when the two sources are highly inconsistent, accurately outputting credible anomaly types or uncertainties, far superior to a single weighted average strategy. This effectively avoids single-modal misjudgments caused by factors such as lighting interference and machine noise, ensuring that the system only issues an alarm when strong multimodal signals are consistent. Redundant continuous windows are automatically merged into single anomaly events, preventing duplicate alarms. The structured alarm information facilitates subsequent maintenance and quality tracking. The hash-based bucketing method improves system scalability and processing efficiency, ensuring real-time capabilities on large-scale production lines. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings required in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of the method in this embodiment. Detailed Implementation

[0014] To facilitate understanding by those skilled in the art, the structure of the present invention will now be described in further detail with reference to the accompanying drawings. It should be understood that, unless otherwise specified, the order of the steps mentioned in this embodiment can be adjusted according to actual needs, and they can even be executed simultaneously or partially simultaneously.

[0015] like Figure 1 As shown, this embodiment of the invention provides a method for detecting skipped needles in fabric, comprising: S1 acquires sewing images in real time using an industrial camera, extracts all stitches in the sewing images, calculates the stitch distance between stitches, and identifies abnormal sewing types and their probabilities based on the stitch distance. S101 performs grayscale processing and filtering noise reduction on the suture image to obtain a smooth grayscale image; In this step, the original industrial image suffers from uneven lighting, numerous noise points, and complex colors, directly affecting the robustness of subsequent binarization and contour detection. Binarization yields a grayscale image with clear texture, less noise, and prominent stitch features, laying the foundation for accurate stitch contour extraction and effectively improving the overall system detection accuracy and anti-interference capability.

[0016] S102 performs binarization on the grayscale image, followed by morphological operations to eliminate noise and breaks, resulting in an optimized binary image. S1021 converts the grayscale image into a binary image through threshold segmentation, performs morphological operations to eliminate noise and breaks, and obtains the optimized binary image. S1022 extracts the contours of all closed connected regions in the optimized binary image to obtain the contour of each candidate stitch. S1023 calculates the area and aspect ratio of each candidate contour, filters out contours whose area is within the area threshold and whose aspect ratio is within the aspect ratio threshold, and removes non-stitch areas.

[0017] In this step, simple binarization is susceptible to noise interference, leading to false contours, breaks, or poor connections, resulting in missed or false detections of pins. Additionally, stray objects or template patterns outside the pins are easily extracted incorrectly. Effective binarization of the grayscale image separates the pin features from the background; morphological processing removes isolated noise and repairs breaks; and all contour regions truly representing the pins are accurately extracted and selected.

[0018] S103 extracts the geometric center of each selected stitch outline and sorts them according to the suture direction, counts the distance between adjacent stitches, and performs stitch distance anomaly detection. S1031 calculates the centroid coordinates of each pin profile based on the image moments; S1032 uses the least squares method to fit the main axis of the stitch arrangement for the centroid coordinates of all stitches, projects all centroids onto the direction of the main axis, and sorts them in ascending order of projection value to ensure that the sorting result is consistent with the direction of the suture. S1033 calculates the Euclidean distance between each pair of adjacent centroids after sorting, and counts all stitch distances; S1034 compares each stitch distance with the standard stitch distance. If the difference between the stitch distance and the standard stitch distance is greater than the tolerance threshold, then the interval is determined to be an abnormal stitch distance area.

[0019] In this step, the multiple stitches are not strictly arranged in a straight line and are subject to distortion and jitter, making it difficult to automatically sort and measure the actual stitch distance through direct statistics. Inaccurate sorting can easily lead to disordered stitch distance statistics and misjudgment of anomalies. The accurate geometric center (centroid) of each stitch is calculated and obtained, and all stitches are sorted according to the overall direction of the suture to facilitate standardized statistical detection of adjacent stitch distances in subsequent steps.

[0020] S104 determines the sewing anomaly type based on the type and distribution of the abnormal interval. The sewing anomaly types include: skipped stitches, missed stitches, broken stitches, and uncertainties.

[0021] S1041 traverses the abnormal stitch spacing area. If n consecutive stitches have abnormal spacing or n consecutive centroids are missing, it is determined to be a missing stitch and a high probability of missing stitch is assigned. In this step, a missed needle occurs when multiple consecutive needles fail to form a proper loop, resulting in the absence of multiple adjacent needles or a significant increase in needle spacing. Multiple centroids may disappear in the local structure. The image will show a continuous area of ​​needles with abnormally continuous needle spacing or continuous disappearance of centroids. Typical causes include a malfunctioning needle selection mechanism, broken needles, or severe wear on the needle bed.

[0022] Where n can be 2 or 3, and is usually judged as a missed needle when there are ≥2 or ≥3 consecutive abnormal needle spacings.

[0023] S1042 calculates the area of ​​all pins that have not yet been marked as missing pins in the abnormal pin spacing area. If the area of ​​the pin area is less than 50% of the area threshold, it is determined to be a broken pin and a high probability of broken pin is assigned. In this step, a broken needle indicates localized structural damage at the needle's base, resulting in an incomplete needle shape. While the needle is not completely missing, its centroid can be detected in the image, but the overall area is significantly smaller. The image shows some needles at locations with abnormal needle spacing where the centroid is still present, but the structural area is less than the standard threshold (e.g., 50% of the normal value). Typical causes include localized needle tip damage, broken hook yarn, yarn breakage, and clogging by fly waste.

[0024] A single instance generally refers to a situation where only one adjacent stitch distance exceeds the standard threshold (e.g., ±20%), while the stitch distances before and after it are normal. Simultaneously, the number of stitches at that point has not substantially decreased, and the centroid is not missing. To avoid misjudging two consecutive skipped stitches as skipped stitches under extreme conditions, a maximum of two consecutive abnormal stitch distances can be set to still be considered skipped stitches; the upper limit is adjustable.

[0025] S1043 checks each of the remaining unmarked points. If an isolated single abnormally large stitch distance is found and the centroid is not missing, it is determined to be a skipped stitch and assigned a high probability of skipped stitch. Other types are assigned a low probability. In this step, skipped stitches refer to individual stitches that do not form a proper loop but are not missing, causing an abnormally large stitch length at that point. However, the center of mass of the stitch is present, the structure is relatively intact, and the abnormality only occurs at an isolated single point (or a very few consecutive points). The image shows an isolated abnormally large stitch length point, with the center of mass not missing, the area normal, and no abnormalities in the preceding and following intervals. Typical causes include occasional abnormal yarn tension, temporary foreign objects, and minor operational fluctuations.

[0026] Furthermore, missing needles require continuous abnormalities and / or multiple centroid missing points; a single point abnormality is unlikely to be classified as a missing needle. Broken needles are separated by the area characteristics; a single point is sufficient and those points already classified as missing needles should be excluded. Skipped needles focus on isolated single-point abnormalities with complete structure (normal area and centroid), requiring no abnormalities at the points before and after.

[0027] Locations identified as missing needles are automatically excluded from broken needle and skipped needle detection. This prevents a break point in a missing needle area from being repeatedly counted as a broken or skipped needle. Continuous defects (missing needles) have the largest impact and are prioritized for detection and exclusion to prevent a continuous string of missing needles from being broken into multiple lower-level faults by subsequent rules. Broken needle detection targets remaining abnormal points not identified as missing needles, excluding any covered special structures. Skipped needles, as remaining isolated anomalies, will not be misjudged due to the properties of certain points in the missing needle area. In this way, points output at each step are excluded from subsequent detections, achieving mutually exclusive and unique discrimination, resulting in accurate and transparent results.

[0028] S1044 If the image quality is poor or the area is blurry and cannot be determined, it is judged as uncertain and assigned a high probability. S1045 highlights all abnormal areas on the original image and distinguishes the abnormality type with different colors. It also outputs the abnormality location, type label, and confidence probability to facilitate subsequent manual review and traceability.

[0029] In this step, traditional inspection relies on manual labor, which is prone to missed detections, slow, and subjective; it cannot distinguish and label different types of anomalies, and it is difficult to trace specific locations. Furthermore, confusion in anomaly types (skipped stitches / missing stitches / broken stitches) easily leads to misjudgments. Based on the actual stitch length distribution, combined with data on missing stitches, area, and shape, this system automatically identifies various sewing anomalies (such as skipped stitches, missing stitches, broken stitches, etc.) and outputs the anomaly location and category, achieving anomaly visualization and source tracing. In each step, 0.85 probability is assigned to the judgment type, 0.15 to "uncertain," and the rest to 0. If a judgment cannot be made, all probabilities are assigned to "uncertain."

[0030] S2 collects vibration signals from the needle connecting rod using a vibration sensor, inputs the vibration signals into a pre-trained vibration anomaly recognition model, and obtains the type of vibration anomaly sewing and its probability. The vibration anomaly identification model is trained through the following steps: S201 synchronously acquires historical vibration signals and suture images, extracts the features of each vibration signal segment, and uses the image anomaly recognition results at the same time to label the abnormal sewing type, constructing labeled training samples, while the remaining unlabeled vibration signal features are used as unlabeled training samples. S202 is based on labeled training samples and uses a neural network to build a vibration anomaly recognition model. Vibration signal features are used as model input and abnormal sewing type is used as output. Supervised multi-class classification training is performed to obtain a supervised vibration anomaly recognition model. S203 is based on unlabeled training samples. The unlabeled vibration signal features are input into the supervised vibration anomaly identification model to obtain pseudo-labels and their confidence probabilities. Training samples and their pseudo-labels with confidence probabilities higher than the confidence probability threshold are selected and fused with labeled training samples to obtain fused training samples. S204 uses fused training samples to train a supervised vibration anomaly identification model. For labeled training samples, cross-entropy loss is used to update the model against the real labels, while for unlabeled training samples, weighted cross-entropy loss is used to update the model.

[0031] In this step, pure visual detection is limited by external environmental factors such as changes in lighting, occlusion, and fabric / thread color. While vibration signals can highly sensitively capture subtle anomalies in mechanical parts (such as needle bar jamming, needle breakage, and skipping steps), they lack intuitive annotation and data interpretation, making it difficult to utilize efficiently and systematically relying solely on human experience. Vibration data itself is difficult to directly distinguish anomaly types. Synchronous visual annotation maps vibration features to anomaly categories, providing a high-quality, supervised sample foundation for model training. The probability types output by the neural network model are combined with subsequent methods to achieve accurate identification of sewing anomaly types.

[0032] Furthermore, image recognition can only identify segments where sewing results have already deteriorated, but some vibration anomalies haven't yet caused visible defects in the image, and these crucial signals are all missed. Thus, the model can only issue an alert when problems have already appeared in the image, missing potential early warning signs of anomalies. Through this semi-supervised training, all vibration signals that weren't detected by the image are also included in the training. The model can learn more about normal equipment states and become more sensitive to subtle vibration changes that deviate from normal, achieving early warning of anomalies.

[0033] S3 divides the image abnormal stitching type and its probability and the vibration abnormal stitching type and its probability into hash buckets. Within the same hash bucket, DS theory fusion is performed to determine the current abnormal stitching type, and an early warning is issued based on the abnormal stitching type.

[0034] S301 divides the abnormal sewing types and their probabilities in the images obtained from each production line and the abnormal vibration sewing types and their probabilities into hash buckets based on the device ID; S3011 aggregates the image abnormal sewing types and their probabilities, vibration abnormal sewing types and their probabilities, window ID, device ID, and timestamp of each detection window to form a structured window task set; S3012 assigns each window task to the corresponding hash bucket based on the hash modulo of the device ID. The data in the bucket is then grouped by device ID, and each group is processed independently.

[0035] In this step, the airbag production line typically has dozens to hundreds of automatic sewing machines. Each machine is equipped with two types of sensors: image detection and vibration detection. Data is collected in real time every second, and each machine outputs a detection window of data. The entire factory generates dozens or hundreds of window data points every second, which are directly aggregated on a single server for processing. This results in insufficient speed, severe delays, and congestion.

[0036] Hash buckets can distribute data based on device IDs using hashing. For example, 50 sewing machines can be hashed into 10 buckets, with each bucket handling data from 5 machines. Each bucket can be processed independently and in parallel on different servers or threads, allowing the entire system to scale linearly with the number of machines.

[0037] S302 performs DS theory fusion of the image abnormal sewing type and its probability, and the vibration abnormal sewing type and its probability for each window within the same bucket, to obtain the fused abnormal sewing type; S3021 iterates through all windows under each device group and reads the probability distribution of image anomaly types, the probability distribution of vibration anomaly types, the window ID, the device ID, and the timestamp; S3022 confirms that the probability distribution of image anomaly types and the probability distribution of vibration anomaly types contain a complete and consistent set of anomaly types. All subset probabilities and uncertainty probabilities are normalized to 1. If any type is missing or invalid, it is marked as uncertain and an anomaly is recorded. S3023 uses the Dempster combination formula to calculate the fusion probability of the probability distributions of image anomaly types and vibration anomaly types. The calculation formula is as follows: , , , in, K represents the conflict coefficient, and K represents the normalization factor of the conflict coefficient, reflecting the degree of conflict between the two modes of observation. A For any subset of the complete set of abnormal sewing types, B,C These are subsets of the probability distribution of image anomaly types and subsets of the probability distribution of vibration anomaly types, respectively. Assign their respective basic probabilities; In this step, all possible combinations of assigning images to B and vibrations to C are iterated. As long as B and C have no overlap (no common recognized anomaly type), the product of these two assignment probabilities is accumulated. K This can be understood as the remaining proportion of information that can be trusted. For a subset A (e.g., A = {skipped needle}) of the entire set Θ (e.g., skipped needle, broken needle, missing needle, uncertain), you need to iterate through all combinations B and C such that B ∩ C = A, multiply their probabilities, and finally divide by K. The fusion probability distribution is to calculate an m(A) for each subset A.

[0038] S3024 if If the conflict threshold is set to 0, or K=0, the fusion result is unreliable, and the probability of uncertain abnormal sewing types is set to 1. In this step, if conflict exceeds a threshold (e.g., 0.7), it indicates a severe inconsistency between the "image" and "vibration," making the result unreliable; or, in the extreme case, K=0 (conflict=1), the denominator becomes meaningless. In this case, a forced assignment of m (uncertain) = 1 is made to avoid creating a false anomaly type.

[0039] S3025 if If the probability of an abnormal sewing type is less than or equal to the conflict threshold, the abnormal sewing type with the highest probability is selected as the output of the fused abnormal sewing type.

[0040] In this step, if the conflict is within a reasonable range, the merged m(A) is used. From all single-element subsets A (i.e., skipped stitches, broken stitches, missing stitches, and uncertain stitches), the one with the largest m(A) value is found and output as the merged abnormal sewing type.

[0041] Each device will generate two different sewing anomaly detection conclusions from two sensors within the same time window. These sewing anomaly detection conclusions may be the same or different. Anomalies from different devices should not and should not be confused. After regrouping within the bucket, for each window, all anomaly types and probabilities of that device at that moment are summarized. Furthermore, DS theory is used to fuse them into a more reliable fused anomaly conclusion, reducing the risk of misjudgment by a single sensor.

[0042] In practical applications, when the image is severely blurred but the vibration assessment is normal, the weighted average can only allocate probabilities among a few anomalies (such as broken needles, skipped needles, and missing needles). The algorithm cannot explicitly express what it doesn't know or cannot determine; it simply mechanically distributes the probabilities. In this situation, the anomaly probability is very low (distributed across various categories), leading the system to mistakenly believe the device is normal when it should actually be determined manually, resulting in missed detections. Furthermore, the two sensor viewpoints are highly conflicting—one asserting anomalies, the other asserting no anomalies—and the weighted result is merely an intermediate value, failing to reflect the significant disagreement.

[0043] When assigning probabilities in DS (Discrete Sensing), the entire confidence probability can be assigned to uncertainty, or to a subset (such as a broken needle or a skipped needle). If any sensor in a multi-sensor system has uncertainty or highly conflicting results, DS fusion will significantly increase the final probability of system uncertainty. All highly conflicting information will not be forcibly compromised; instead, it will be fused into the uncertainty, ensuring that the result is not over-amplified.

[0044] S303 merges the same type of abnormal stitching within the same bucket and issues an early warning based on the probability of the abnormal stitching type.

[0045] After grouping by device ID, S3031 sorts the windows of the same device by time. If multiple windows are determined to be of the same abnormal sewing type and the time interval between adjacent windows is less than the set threshold, they are merged into an abnormal event interval, and the start and end times and the probability of this type of all windows in the interval are recorded. S3032 If for each event interval, the maximum value of the corresponding probability of all windows is taken as its event probability. If its event probability is higher than the alarm threshold, an early warning is triggered; otherwise, it is marked as high risk and awaits review.

[0046] In this step, after an anomaly occurs, the equipment may continuously output abnormal results within a short period (because the anomaly has not been resolved). If every result triggers an alarm, factory personnel will be overwhelmed by invalid alarms. The purpose of merging is to group consecutive, similar anomaly windows into the same anomaly event segment, avoiding duplicate alarms. Only key information such as "anomaly start-end time, anomaly type, and extreme probability" is recorded, making alarms more accurate and facilitating subsequent tracing or shutdown procedures.

[0047] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0048] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0049] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0050] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0051] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The words first, second, and third, etc., do not indicate any order. These words can be interpreted as names.

[0052] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0053] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0054] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0055] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

Claims

1. A method for detecting skipped needles in fabric, characterized in that, include: S1 acquires sewing images in real time using an industrial camera, extracts all stitches in the sewing images, calculates the stitch distance between stitches, and identifies abnormal sewing types and their probabilities based on the stitch distance. S2 collects vibration signals from the needle connecting rod using a vibration sensor, inputs the vibration signals into a pre-trained vibration anomaly recognition model, and obtains the type of vibration anomaly sewing and its probability. S3 divides the image abnormal stitching type and its probability and the vibration abnormal stitching type and its probability into hash buckets. Within the same hash bucket, DS theory fusion is performed to determine the current abnormal stitching type, and an early warning is issued based on the abnormal stitching type.

2. The method for detecting skipped needles in fabric according to claim 1, characterized in that, The process of acquiring sewing images in real time using an industrial camera, extracting all stitches from the sewing images, calculating the stitch spacing between stitches, and identifying abnormal sewing types and their probabilities based on stitch spacing includes: S101 performs grayscale processing and filtering noise reduction on the suture image to obtain a smooth grayscale image; S102 performs binarization on the grayscale image, followed by morphological operations to eliminate noise and breaks, resulting in an optimized binary image. S103 extracts the geometric center of each selected stitch outline and sorts them according to the suture direction, counts the distance between adjacent stitches, and performs stitch distance anomaly detection. S104 determines the sewing anomaly type based on the type and distribution of the abnormal interval. The sewing anomaly types include: skipped stitches, missed stitches, broken stitches, and uncertainties.

3. The method for detecting skipped needles in fabric according to claim 2, characterized in that, The process of binarizing the grayscale image, followed by morphological operations to eliminate noise and breaks, to obtain an optimized binary image includes: S1021 converts the grayscale image into a binary image through threshold segmentation, performs morphological operations to eliminate noise and breaks, and obtains the optimized binary image. S1022 extracts the contours of all closed connected regions in the optimized binary image to obtain the contour of each candidate stitch. S1023 calculates the area and aspect ratio of each candidate contour, filters out contours whose area is within the area threshold and whose aspect ratio is within the aspect ratio threshold, and removes non-stitch areas.

4. The method for detecting skipped needles in fabric according to claim 2, characterized in that, The process of extracting the geometric center of each selected stitch outline and sorting them according to the suture direction, calculating the distance between adjacent stitches, and performing stitch distance anomaly detection includes: S1031 calculates the centroid coordinates of each pin profile based on the image moments; S1032 uses the least squares method to fit the main axis of the stitch arrangement for the centroid coordinates of all stitches, projects all centroids onto the direction of the main axis, and sorts them in ascending order of projection value to ensure that the sorting result is consistent with the direction of the suture. S1033 calculates the Euclidean distance between each pair of adjacent centroids after sorting, and counts all stitch distances; S1034 compares each stitch distance with the standard stitch distance. If the difference between the stitch distance and the standard stitch distance is greater than the tolerance threshold, then the interval is determined to be an abnormal stitch distance area.

5. The method for detecting skipped needles in fabric according to claim 2, characterized in that, The sewing anomaly type is determined based on the type and distribution of the abnormal interval. The sewing anomaly types include: skipped stitches, missed stitches, broken needles, and uncertainties. S1041 traverses the abnormal stitch spacing area. If n consecutive stitches have abnormal spacing or n consecutive centroids are missing, it is determined to be a missing stitch and a high probability of missing stitch is assigned. S1042 calculates the area of ​​all pins that have not yet been marked as missing pins in the abnormal pin spacing area. If the area of ​​the pin area is less than 50% of the area threshold, it is determined to be a broken pin and a high probability of broken pin is assigned. S1043 checks each of the remaining unmarked points. If an isolated single abnormally large stitch distance is found and the centroid is not missing, it is determined to be a skipped stitch and assigned a high probability of skipped stitch. Other types are assigned a low probability. S1044 If the image quality is poor or the area is blurry and cannot be determined, it is judged as uncertain and assigned a high probability. S1045 highlights all abnormal areas on the original image and distinguishes the abnormality type with different colors. It also outputs the abnormality location, type label, and confidence probability to facilitate subsequent manual review and traceability.

6. The method for detecting skipped needles in fabric according to claim 1, characterized in that, The vibration anomaly identification model is trained through the following steps: S201 synchronously acquires historical vibration signals and suture images, extracts the features of each vibration signal segment, and uses the image anomaly recognition results at the same time to label the abnormal sewing type, constructing labeled training samples, while the remaining unlabeled vibration signal features are used as unlabeled training samples. S201 synchronously acquires historical vibration signals and suture images, extracts the features of each vibration signal segment, and uses the image anomaly recognition results at the same time to label the abnormal sewing type, constructing labeled training samples, while the remaining unlabeled vibration signal features are used as unlabeled training samples. S202 is based on labeled training samples and uses a neural network to build a vibration anomaly recognition model. Vibration signal features are used as model input and abnormal sewing type is used as output. Supervised multi-class classification training is performed to obtain a supervised vibration anomaly recognition model. S203 is based on unlabeled training samples. The unlabeled vibration signal features are input into the supervised vibration anomaly identification model to obtain pseudo-labels and their confidence probabilities. Training samples and their pseudo-labels with confidence probabilities higher than the confidence probability threshold are selected and fused with labeled training samples to obtain fused training samples. S204 uses fused training samples to train a supervised vibration anomaly identification model. For labeled training samples, cross-entropy loss is used to update the model against the real labels, while for unlabeled training samples, weighted cross-entropy loss is used to update the model.

7. The method for detecting skipped needles in fabric according to claim 1, characterized in that, The process of dividing the image abnormal stitching type and its probability and the vibration abnormal stitching type and its probability into hash buckets, performing DS theory fusion within the same hash bucket to determine the current abnormal stitching type, and issuing an early warning based on the abnormal stitching type includes: S301 divides the abnormal sewing types and their probabilities in the images obtained from each production line and the abnormal vibration sewing types and their probabilities into hash buckets based on the device ID; S302 performs DS theory fusion of the image abnormal sewing type and its probability, and the vibration abnormal sewing type and its probability for each window within the same bucket, to obtain the fused abnormal sewing type; S303 merges the same type of abnormal stitching within the same bucket and issues an early warning based on the probability of the abnormal stitching type.

8. The method for detecting skipped needles in fabric according to claim 7, characterized in that, The step of dividing the abnormal sewing types and their probabilities from the images obtained from each production line and the abnormal vibration sewing types and their probabilities into hash buckets based on the device ID includes: S3011 aggregates the image abnormal sewing types and their probabilities, vibration abnormal sewing types and their probabilities, window ID, device ID, and timestamp of each detection window to form a structured window task set; S3012 assigns each window task to the corresponding hash bucket based on the hash modulo of the device ID. The data in the bucket is then grouped by device ID, and each group is processed independently.

9. The method for detecting skipped needles in fabric according to claim 7, characterized in that, The DS theory fusion of image anomaly stitching types and their probabilities, and vibration anomaly stitching types and their probabilities for each window within the same bucket yields fused anomaly stitching types including: S3021 iterates through all windows under each device group and reads the probability distribution of image anomaly types, the probability distribution of vibration anomaly types, the window ID, the device ID, and the timestamp; S3022 confirms that the probability distribution of image anomaly types and the probability distribution of vibration anomaly types contain a complete and consistent set of anomaly types. All subset probabilities and uncertainty probabilities are normalized to 1. If any type is missing or invalid, it is marked as uncertain and an anomaly is recorded. S3023 uses the Dempster combination formula to calculate the fusion probability of the probability distributions of image anomaly types and vibration anomaly types. The calculation formula is as follows: , , ， in, K represents the conflict coefficient, and K represents the normalization factor of the conflict coefficient, reflecting the degree of conflict between the two modes of observation. A For any subset of the complete set of abnormal sewing types, B,C These are subsets of the probability distribution of image anomaly types and subsets of the probability distribution of vibration anomaly types, respectively. Assign their respective basic probabilities; S3024 if If the conflict threshold is set to 0, or K=0, the fusion result is unreliable, and the probability of uncertain abnormal sewing types is set to 1. S3025 if If the probability of an abnormal sewing type is less than or equal to the conflict threshold, the abnormal sewing type with the highest probability is selected as the output of the fused abnormal sewing type.

10. A method for detecting skipped needles in fabric according to claim 7, characterized in that, The method of merging the same type of fusion-abnormal sewing within the same bucket, and issuing an early warning based on the probability of the fusion-abnormal sewing type, includes: After grouping by device ID, S3031 sorts the windows of the same device by time. If multiple windows are determined to be of the same abnormal sewing type and the time interval between adjacent windows is less than the set threshold, they are merged into an abnormal event interval, and the start and end times and the probability of this type of all windows in the interval are recorded. S3032 If for each event interval, the maximum value of the corresponding probability of all windows is taken as its event probability. If its event probability is higher than the alarm threshold, an early warning is triggered; otherwise, it is marked as high risk and awaits review.

Images