A garment sewing defect detection method and system based on visual recognition

Abstract

The application discloses a kind of clothing sewing defect detection method and system based on visual identification, it is related to visual detection technical field, obtain the sewing reference data of the fabric area to be detected, generate the sewing detection reference including standard sewing track, stitch unit reference, track offset allowed range and defect determination reference, the present application is by collecting the forward diffuse light illumination image and oblique incidence light illumination image of sewing area, based on two kinds of images generate seam line saliency map and pinhole saliency map, enhance the feature of seam line and pinhole, suppress fabric background interference, improve the defect recognition accuracy under complex fabric scene, simultaneously according to standard sewing track establish track coordinate system along path extension, seam line saliency map and pinhole saliency map are mapped to track coordinate system, extract stitch unit in track coordinate system and calculate the stitch length, seam line width and other characteristics of each unit, realize the accurate detection to single stitch scale defect.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, specifically to a method and system for detecting defects in garment sewing based on visual recognition. Background Technology

[0002] In garment sewing production, stitch quality usually needs to meet requirements such as consistent stitch length, continuous stitch, stable seam distance, and uniform stitch appearance. Defects such as skipped stitches, broken threads, missed stitches, and thread deviation often only appear within local stitch length units, and the defects are small in size and short in duration. If they cannot be identified in time during the sewing process, they will affect the structural strength and appearance consistency of the garment, and increase the risk of subsequent rework and scrap.

[0003] Existing methods for detecting sewing defects mainly include manual visual inspection, single-image threshold segmentation, and simple visual inspection based on the contour of the seam area. Manual visual inspection relies on the operator's experience and is difficult to reliably detect short-distance thread breaks, skipped stitches, and localized missed stitches under high-speed sewing cycles. Single-image threshold segmentation usually relies on the grayscale or color difference between the seam and the fabric. When the fabric has texture, dark patterns, reflections, wrinkles, or the seam color is close to the fabric color, the seam area is easily confused with the fabric texture, leading to false detections or missed detections. Detection methods based solely on the overall contour of the seam are difficult to distinguish between abnormal stitch spacing (where the stitch length increases but the seam remains continuous), abnormal thread breaks (where the connection between adjacent stitches is interrupted), and missed stitches (where there is no effective seam response within the standard trajectory area). Therefore, this paper proposes a visual recognition-based method and system for detecting garment sewing defects to address the aforementioned problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this paper provides a visual recognition-based method and system for detecting garment sewing defects. This solution overcomes the limitations of existing sewing defect detection methods, which primarily rely on manual visual inspection, single-image threshold segmentation, and simple visual inspection based on the contour of the sewing area. Manual visual inspection depends on operator experience and is difficult to reliably detect short-distance thread breaks, skipped stitches, and localized missed stitches under high-speed sewing. Single-image threshold segmentation typically depends on the grayscale or color difference between the sewing thread and the fabric. When the fabric has texture, dark patterns, reflections, wrinkles, or the sewing thread color is close to the fabric color, the sewing area can easily be confused with the fabric texture, leading to false detections or missed detections. Furthermore, detection methods based solely on the overall contour of the sewing thread struggle to distinguish between abnormal stitch spacing (where the stitch length increases but the sewing thread remains continuous), abnormal thread breaks (where adjacent stitches are interrupted), and missed stitches (where there is no effective sewing thread response within the standard trajectory area).

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A visual recognition-based method for detecting garment sewing defects includes:

[0007] Obtain sewing reference data for the fabric area to be inspected, and generate a sewing inspection reference that includes a standard sewing trajectory, a stitch distance unit reference, a trajectory offset allowable range, and a defect judgment reference. The sewing reference data includes the sewing path, standard stitch distance, standard seam distance, stitch type, and allowable deviation of sewing process parameters.

[0008] During the sewing process, the first image of the fabric area to be detected under normal diffuse lighting and the second image of the oblique incident lighting are simultaneously acquired to generate a seam saliency map and a pinhole saliency map, respectively.

[0009] Establish a trajectory coordinate system and map the suture salience map and the pinhole salience map to the trajectory coordinate system;

[0010] Extract the stitch spacing unit in the trajectory coordinate system, and calculate the stitch spacing length, suture width, needle hole response value, trajectory lateral offset, and suture saliency response value in the corresponding area of ​​the standard sewing trajectory.

[0011] Using the stitch spacing unit as a node, edges are established between adjacent stitch spacing units along the sewing path in the trajectory coordinate system to construct a stitch spacing-level stitch topology graph, and the stitch continuity result is calculated based on the stitch spacing-level stitch topology graph.

[0012] The stitch length, suture width, needle hole response value, trajectory lateral offset, stitch continuity result, and suture saliency response value in the corresponding area of ​​the standard sewing trajectory are matched with the sewing inspection benchmark. Combined with the defect judgment benchmark, the sewing anomaly judgment result is obtained.

[0013] Based on the sewing anomaly assessment results, determine the type, location, and severity of the sewing defects, output the detection results, and generate sewing process feedback information based on the detection results.

[0014] Furthermore, a vision-based garment sewing defect detection system is proposed to implement the vision-based garment sewing defect detection method described above, including:

[0015] A reference processing module is used to acquire sewing reference data of the fabric area to be inspected and generate a sewing inspection reference that includes a standard sewing trajectory, a stitch distance unit reference, a trajectory offset allowable range, and a defect judgment reference. The reference processing module includes:

[0016] The reference data acquisition unit is used to acquire sewing reference data for the fabric area to be inspected.

[0017] The inspection benchmark generation unit is used to generate a sewing inspection benchmark that includes a standard sewing trajectory, a stitch distance unit benchmark, a trajectory offset allowable range, and a defect judgment benchmark based on the sewing benchmark data.

[0018] An image acquisition and preprocessing module is used to simultaneously acquire a first image of the fabric area to be inspected under forward diffuse lighting and a second image of the fabric under oblique incident lighting during the sewing process, and to perform synchronous compensation and feature enhancement processing on the images. The image acquisition and preprocessing module includes:

[0019] The synchronous acquisition control unit is used to acquire the sewing machine needle bar position signal and fabric feed displacement signal, calculate the acquisition time and image displacement, synchronously acquire the first image under forward diffuse lighting and the second image under oblique incident lighting and perform synchronous compensation.

[0020] The image enhancement processing unit is used to perform pixel registration on the first image under forward diffuse lighting and the second image under oblique incident lighting, calculate the color normalization result and the shadow gradient result, and generate the salience map of the seam and the salience map of the pinhole respectively.

[0021] The feature extraction and analysis module is used to establish a trajectory coordinate system and map a saliency map, extract stitch spacing units and calculate stitch feature parameters, construct a stitch spacing-level stitch topology map and calculate stitch continuity results. The feature extraction and analysis module includes:

[0022] The coordinate transformation and mapping unit is used to establish a trajectory coordinate system based on the standard sewing trajectory and to map the salient lines and needle holes to the trajectory coordinate system.

[0023] The stitch feature calculation unit is used to extract the stitch distance unit in the trajectory coordinate system, calculate the stitch distance length, suture width, needle hole response value, trajectory lateral offset, and suture saliency map response value in the corresponding area of ​​the standard sewing trajectory, construct the stitch distance level stitch topology map, and calculate the stitch continuity result.

[0024] An anomaly detection module is used to match the extracted stitch features with the sewing inspection benchmark, and combine the results with the defect detection benchmark to obtain a sewing anomaly detection result. The anomaly detection module includes:

[0025] The feature matching and determination unit is used to match the stitch length, suture width, needle hole response value, trajectory lateral offset, stitch continuity result, and suture saliency map response value in the corresponding area of ​​the standard sewing trajectory with the sewing detection benchmark to obtain the stitch length abnormality determination result, suture width abnormality determination result, needle hole abnormality determination result, trajectory offset abnormality determination result, stitch continuity abnormality determination result, and missing stitch abnormality determination result.

[0026] The defect information generation unit is used to determine the type, location, and severity of sewing defects based on the sewing anomaly judgment results;

[0027] The results output and feedback module is used to output the sewing defect detection results and generate sewing process feedback information based on the detection results.

[0028] The beneficial effects of this invention compared to the prior art are:

[0029] This solution acquires a first image of the fabric area under forward diffuse lighting and a second image of the fabric under oblique incident lighting. Based on these two types of images, it generates a seam saliency map and a pinhole saliency map, respectively, enhancing the features of seams and pinholes and suppressing background interference, thereby improving the accuracy of defect identification in complex fabric scenarios. At the same time, it establishes a trajectory coordinate system that extends along the path based on the standard sewing trajectory, mapping the seam saliency map and the pinhole saliency map to the trajectory coordinate system. In the trajectory coordinate system, it extracts the stitch distance unit and calculates the stitch distance length, seam width, pinhole response value, trajectory lateral offset, and other features of each unit, achieving accurate detection of defects at the single stitch distance scale and solving the problem of missed detection of small-scale, short-duration defects.

[0030] This solution constructs a stitch-level stitch topology map using stitch spacing units as nodes and the connection relationships between adjacent stitch spacing units as edges. Node features reflect local anomalies in a single stitch spacing, while edge stitch continuity values ​​reflect the connection status of adjacent stitch spacings. This accurately distinguishes different types of sewing defects such as skipped stitches, broken threads, thread offsets, and missed stitches, avoiding confusion between defect types. Simultaneously, it matches stitch spacing length, thread width, needle hole response value, trajectory lateral offset, stitch continuity results, and the saliency map response value of the corresponding area of ​​the standard sewing trajectory with the sewing inspection benchmark. Combined with the defect judgment benchmark, it obtains comprehensive sewing anomaly judgment results, simultaneously covering all common sewing defect types such as stitch spacing anomalies, thread width anomalies, needle hole anomalies, trajectory offset anomalies, stitch continuity anomalies, and missed stitch anomalies. Attached Figure Description

[0031] Figure 1 This is a flowchart of a visual recognition-based method for detecting defects in garment sewing proposed in this invention;

[0032] Figure 2 This is a flowchart illustrating the generation of suture salience maps and pinhole salience maps in this invention;

[0033] Figure 3 This is a flowchart illustrating the mapping between the trajectory coordinate system and the suture salience map and the pinhole salience map in this invention;

[0034] Figure 4 This is a flowchart of the topology graph construction and continuity result acquisition in this invention;

[0035] Figure 5 This is a flowchart of the anomaly determination process in this invention;

[0036] Figure 6 This is a structural block diagram of a vision recognition-based garment sewing defect detection system proposed in this invention; Detailed Implementation

[0037] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0038] Reference Figures 1-5 As shown, a method for detecting garment sewing defects based on visual recognition includes:

[0039] S1. Obtain sewing reference data for the fabric area to be inspected, and generate a sewing inspection reference that includes a standard sewing trajectory, a stitch distance unit reference, a trajectory offset allowable range, and a defect judgment reference. The sewing reference data includes the sewing path, standard stitch distance, standard seam distance, stitch type, and allowable deviation of sewing process parameters.

[0040] Specifically, sewing reference data serves as the basis for subsequent visual inspection. The sewing path is set according to the sewing locations and stitch paths specified in the garment process documents and the corresponding path coordinates in the pattern data, and is written into the sewing equipment control program. Standard stitch length, standard seam distance, stitch type, and allowable deviations in sewing process parameters are derived from the garment process documents and the sewing equipment control program. The conversion relationship between image coordinates and actual length is determined by the camera calibration ratio. The sewing equipment positioning error is determined by the positioning accuracy parameters of the sewing equipment. The sewing path is the geometric position the thread should pass through in the area of ​​the fabric to be inspected. The standard stitch length is the reference distance between adjacent needle holes or adjacent stitch units. The standard seam distance is the target distance of the standard sewing trajectory relative to the fabric edge. The stitch type includes the needle hole arrangement, thread connection form, and reference range for thread width. The allowable deviations in sewing process parameters are the allowable deviations from the above process parameters, including allowable deviations in stitch length, seam distance, and thread width.

[0041] Furthermore, the step of generating a sewing inspection benchmark based on the sewing benchmark data specifically includes:

[0042] S11. Generate a standard sewing trajectory based on the sewing path;

[0043] Specifically, the sewing path is based on the sewing parts and stitching paths specified in the garment process document. The pattern data provides the corresponding path coordinates, and the sewing equipment control program reads these coordinates as the path to be processed. When generating the standard sewing trajectory, the path coordinates are first sorted according to the sewing direction to obtain an ordered sequence of path points. Then, the distance and direction changes between adjacent path points are checked. Path points with repeated coordinates or whose adjacent distance is less than half the trajectory sampling arc length interval are merged. The trajectory sampling arc length interval is determined based on the standard stitch length and camera calibration resolution. It is not greater than half the standard stitch length and is greater than or equal to the actual length corresponding to the camera calibration resolution. For path points that are not marked as turning points in the garment process document and that prevent the preceding and following path segments from being continuously connected along the sewing direction, the path point is deleted or corrected according to the position of its preceding and following adjacent path points. Then, the path turning points are transitioned according to the preceding and following directions of adjacent path points to ensure continuous direction changes between adjacent path segments. Finally, the cumulative length is calculated along the processed sewing path, and trajectory sampling points are selected according to equal arc length intervals to obtain a standard sewing trajectory composed of multiple trajectory sampling points.

[0044] S12. Generate a stitch spacing unit reference based on the standard stitch spacing and stitch type;

[0045] Specifically, the needle spacing unit reference includes the standard needle spacing. Needle spacing lower limit threshold Needle spacing upper limit threshold The expected connection relationship between the pinhole arrangement template and adjacent pin spacing units. and According to standard stitch length The allowable deviation ΔP of the needle spacing is calculated.

[0046] in, ;

[0047] ;

[0048] It should be noted that the needle hole arrangement template represents the number of needle holes corresponding to the stitch type, the needle hole arrangement direction, and the reference interval between adjacent needle holes. The expected connection relationship between adjacent stitch spacing units represents the stitch connection state that should be formed between the previous stitch spacing unit and the next stitch spacing unit arranged along the sewing direction.

[0049] S13. Generate the allowable range of trajectory offset based on the standard seam distance and the allowable deviation of sewing process parameters;

[0050] Specifically, the allowable range of trajectory deviation Used to limit the maximum permissible offset of the seam centerline relative to the standard sewing path in the normal direction. This includes the process tolerance corresponding to the standard seam distance, the sewing compensation amount corresponding to the fabric thickness, and the error allowance corresponding to the positioning accuracy of the sewing equipment. The process tolerance comes from the allowable deviation of the sewing process parameters, the sewing compensation amount corresponding to the fabric thickness comes from the sewing requirements for the corresponding fabric thickness in the garment process document, and the error allowance is set according to the positioning accuracy parameters of the sewing equipment.

[0051] S14. Generate a defect judgment benchmark based on the standard stitch length, stitch type, and allowable deviation of sewing process parameters. The defect judgment benchmark includes a needle hole response threshold, a stitch length upper limit threshold, a stitch length lower limit threshold, a suture width threshold, a stitch continuity threshold, a suture significant response threshold, and a shadow gradient threshold.

[0052] Specifically, the defect determination criteria include the pinhole response threshold. Needle spacing upper limit threshold Needle spacing lower limit threshold Thread width threshold Line continuity threshold Significant response threshold of suture and shadow gradient threshold ;

[0053] Needle spacing upper limit threshold and needle spacing lower limit threshold According to standard stitch length The allowable deviation ΔP of the needle spacing is calculated.

[0054] Pinhole response threshold Determined based on the lower quantile statistical value of the pinhole response value in qualified sewing samples of the same stitch type;

[0055] suture width threshold The upper limit of the width is the sum of the allowable deviation of the stitch width and the width of the process stitch corresponding to the stitch type.

[0056] Line continuity threshold Determined based on the lower quantile statistical value of the thread continuity value in the connecting area of ​​adjacent stitch units in qualified sewing samples of the same stitch type;

[0057] Significant response threshold of suture Determined based on the lower quantile statistical value of the saliency map response value of the corresponding area of ​​the standard sewing trajectory in qualified sewing samples of the same stitch type;

[0058] Shadow gradient threshold The upper quantile statistical value is determined based on the shadow gradient results of the seam area under oblique incident lighting conditions in qualified sewing samples of the same stitch type.

[0059] It should be noted that the pinhole response threshold Line continuity threshold and suture significant response threshold Taking the 5th percentile values ​​of pinhole response, suture continuity, and suture saliency response from qualified samples can reduce the impact of individual low-response noise points on the threshold. (Shadow gradient threshold) Taking the 95th percentile value of the shadow gradient result in qualified samples can reduce the impact of individual bright reflective points, local wrinkles or isolated noise points on the threshold.

[0060] S2. During the sewing process, simultaneously acquire the first image of the fabric area to be detected under forward diffuse lighting and the second image of the fabric area under oblique incident lighting, and generate a seam salience map and a pinhole salience map respectively;

[0061] Specifically, the first image under forward diffused lighting is used to represent the color or grayscale difference between the seam and the fabric, while the second image under oblique incident lighting is used to represent the shadow gradient differences formed by the needle holes, stitch discontinuities, and seam edges under oblique incident lighting.

[0062] It should be noted that the first image under forward diffuse lighting and the second image under oblique incident lighting are acquired synchronously by the vision acquisition module under the corresponding lighting conditions, and the two images correspond to the same fabric area to be detected, so as to ensure that the subsequent saliency map calculation and trajectory coordinate mapping have a consistent image basis.

[0063] Furthermore, the step of simultaneously acquiring a first image of the fabric area to be inspected under forward diffuse lighting and a second image of the fabric area under oblique incident lighting during the sewing process, and generating a seam saliency map and a pinhole saliency map respectively, specifically includes:

[0064] S21. Obtain the position signal of the sewing machine needle bar and the fabric feed displacement signal;

[0065] Specifically, the needle bar position signal can be output by the sewing machine encoder, photoelectric sensor, Hall sensor or equipment controller, which indicates the position phase of the needle within one stitch period. The fabric feed displacement signal can be output by the fabric feed motor encoder, fabric displacement sensor or equipment controller, which indicates the displacement of the fabric area to be detected relative to the camera field of view.

[0066] S22. Determine the acquisition time of the needle spacing cycle based on the needle bar position signal, and calculate the displacement of the fabric area to be detected in the image based on the fabric feeding displacement signal;

[0067] Specifically, the acquisition time is set after the needle leaves the fabric and the edge of the presser foot does not overlap with the detection area corresponding to the standard sewing trajectory. This triggers the acquisition of the first image under forward diffused light and the second image under oblique incident light, in order to reduce the impact of needle obstruction and instantaneous vibration of the sewing mechanism on the image.

[0068] The displacement of the fabric area to be detected in the image is obtained by converting the fabric feed displacement signal and the camera calibration ratio. The camera calibration ratio represents the conversion relationship between the actual length and the image pixels. The fabric feed displacement signal represents the actual displacement of the fabric area to be detected caused by the fabric feed mechanism between adjacent acquisition times. The actual displacement is output by the fabric feed motor encoder, fabric displacement sensor or device controller. The camera calibration ratio represents the correspondence between the actual length and the image pixels. It is calculated from the pixel length of the calibration line segment with known actual length in the image, or determined by the correspondence between the actual size of the camera field of view and the size of the image pixels. By removing the actual bit corresponding to the fabric feed displacement signal and using the camera calibration ratio, the pixel displacement of the fabric area to be detected in the image coordinate system can be obtained.

[0069] S23. At the acquisition time, the first image of the forward diffuse illumination and the second image of the oblique incident illumination are acquired synchronously, and the first image of the forward diffuse illumination and the second image of the oblique incident illumination are compensated synchronously according to the displacement.

[0070] Specifically, at the acquisition time of the stitch period, a first image is acquired under normal diffuse lighting conditions and a second image is acquired under oblique incident lighting conditions. When there is an acquisition time interval between the first image under normal diffuse lighting and the second image under oblique incident lighting, the actual displacement of the fabric area to be detected along the fabric feeding direction during this time interval is obtained according to the fabric feeding displacement signal. Combined with the conversion relationship between image coordinates and actual length, the actual displacement is converted into a displacement in the image coordinate system. The position of the fabric area to be detected in the first image under normal diffuse lighting and the second image under oblique incident lighting is compensated according to this displacement, so that the image content corresponding to the same stitch period and the same standard sewing trajectory area in the two images is aligned in the image coordinate system.

[0071] S24. Perform pixel registration on the first image of forward diffuse lighting and the second image of oblique incident lighting to obtain the registered first image of forward diffuse lighting and the second image of oblique incident lighting;

[0072] Specifically, based on the correspondence between feature points, region boundaries, or pinhole response positions of the same fabric region to be detected in the first image under forward diffuse lighting and the second image under oblique incident lighting, pixel registration is performed on the first image under forward diffuse lighting and the second image under oblique incident lighting so that the same physical location in the two images corresponds to the same image coordinates. After registration, it is checked whether the boundary position or pinhole position of the same detection region in the two images is consistent. When the deviation between the corresponding positions exceeds the set pixel error boundary, image registration is re-executed.

[0073] S25. Calculate the color normalization result of the fabric region to be detected based on the first image of the registered forward diffused light, and calculate the shadow gradient result of the fabric region to be detected based on the second image of the registered oblique incident light.

[0074] Specifically, the first image under registered forward diffuse lighting is converted to a color space for color feature extraction;

[0075] As a preferred embodiment of this solution, the HSV color space is adopted. The hue component (H), saturation component (S), and luminance component (V) of each pixel in the fabric area to be detected are extracted. Based on the color statistics of the standard sewing trajectory area and the fabric texture area in the current fabric area to be detected, the reference values ​​for the seam color and the fabric background color are determined. The color distance between each pixel and the seam color calibration value is calculated respectively. and the color distance between each pixel and the fabric background color calibration value. And calculate the color difference response based on the color distance difference. ;

[0076] in, ε is a correction factor to prevent the denominator from being zero;

[0077] ε is the normalization correction coefficient. ε is a very small positive number greater than 0, used to prevent the denominator from being zero. ε is determined according to the data precision of the image response value. Its value is less than the minimum effective resolution of the color difference response, shadow gradient magnitude or discrimination calculation result. In a preferred embodiment of this scheme, ε is 10^-6.

[0078] It is understandable that the color difference response... The larger the value, the closer the corresponding pixel is to the stitch color feature;

[0079] Furthermore, the response to color differences within the area of ​​the fabric to be tested. Normalization is performed to obtain the color normalization result. ,in, , and The color difference response is respectively within the area of ​​the fabric to be tested. The maximum and minimum values.

[0080] Furthermore, the registered second image is converted into a grayscale image I2, and a gradient operator is used to calculate the grayscale gradient within the fabric region to be detected.

[0081] As a preferred embodiment of this solution, the Sobel gradient operator is used to calculate the gradient components of the grayscale image I2 in the horizontal direction. and the gradient component in the vertical direction and according to and Calculate the shadow gradient magnitude G, where, ;

[0082] Furthermore, the shadow gradient amplitude G in the fabric area to be detected is normalized to obtain the shadow gradient result. ,in, , and These represent the maximum and minimum values ​​of the shadow gradient amplitude G within the fabric area to be detected, respectively, and the shadow gradient results. Used to indicate the degree of local light and shadow variation formed by pinholes, suture edges, stitch discontinuities, and local uneven areas under oblique incident light conditions.

[0083] S26. Based on the color normalization results and shadow gradient results, enhance the seam area in the fabric area to be detected and suppress the fabric texture area to obtain the seam saliency map. Based on the stitch discontinuities and local dark spot responses in the seam saliency map, obtain the pinhole saliency map.

[0084] Specifically, the result of color normalization and shadow gradient results Weighted fusion is performed to obtain the initial suture significant response M, where, , For color weights, For the shadow gradient weights, and + =1;

[0085] Color weight and shadow gradient weights The weights are determined based on the statistical results of the image response within the current fabric region to be detected. It is assumed that the absolute difference between the average value of the standard sewing trajectory region and the average value of the fabric texture region in the color normalization result is the first type of difference value A, and the absolute difference between the average value of the seam edge and the needle hole neighborhood and the average value of the fabric texture region in the shadow gradient result is the second type of difference value B. The fabric texture region is the region in the current fabric region to be detected that deviates from the standard sewing trajectory region and does not belong to the needle hole response position. The color weight is determined according to A / (A+B), and the shadow gradient weight is determined according to B / (A+B).

[0086] Where A and B are both non-negative numbers, the color weight and the shadow gradient weight are both values ​​in the range of 0 to 1, and their sum is 1. When A+B is zero, the color weight and the shadow gradient weight are both 0.5.

[0087] Furthermore, the initial salient response M of the seam is screened for continuity along the standard sewing trajectory direction. High response regions that are continuously distributed along the standard sewing trajectory direction and whose response values ​​meet the characteristics of the seam region are retained, while fabric texture responses that are inconsistent with the standard sewing trajectory direction and are discretely distributed are suppressed to obtain the seam salient map.

[0088] When generating the pinhole saliency map, the locations where local response decreases along the standard sewing trajectory are extracted from the stitch saliency map as stitch discontinuities. Local dark spot responses with lower brightness than the surrounding area are extracted from the corresponding region in the second image, and these are combined with the shadow gradient results. Extract local extrema of the shadow gradient. When the same location simultaneously satisfies the conditions of line discontinuity, enhanced local dark spot response, and local extrema of the shadow gradient, mark the location as the pinhole response location. Calculate the pinhole response value based on the local dark spot response value and the shadow gradient result to form a pinhole saliency map.

[0089] S3. Establish a trajectory coordinate system based on the standard sewing trajectory, and map the salient lines and needle holes to the trajectory coordinate system;

[0090] Furthermore, the step of establishing a trajectory coordinate system based on a standard sewing trajectory and mapping the seam salience map and needle hole salience map to the trajectory coordinate system specifically includes:

[0091] S31. Discretize the standard sewing trajectory into multiple trajectory sampling points, and calculate the tangential direction and normal direction at each trajectory sampling point;

[0092] Specifically, the standard sewing trajectory is sampled at preset arc length intervals to obtain a set of trajectory sampling points. The preset arc length interval is no greater than half of the standard stitch length, so that each stitch length unit contains at least two trajectory sampling points, and the sampled standard sewing trajectory can continuously cover the sewing path. For each trajectory sampling point, the tangential direction is calculated based on the coordinate difference between its preceding and following trajectory sampling points. The tangential direction is rotated 90 degrees to obtain the normal direction. The tangential direction is used to represent the direction of extension along the sewing path, and the normal direction is used to represent the lateral direction of deviation from the standard sewing trajectory.

[0093] S32. Using the tangential direction as the first coordinate direction and the normal direction as the second coordinate direction, establish a trajectory coordinate system that extends along the standard sewing trajectory. The first coordinate direction represents the length position along the sewing path, and the second coordinate direction represents the lateral offset position of the sewing thread relative to the standard sewing trajectory.

[0094] Specifically, the trajectory coordinate system uses arc length coordinates s and lateral offset coordinates d. The starting point of s is the starting point of the standard sewing trajectory. s gradually increases along the sewing direction. d takes the standard sewing trajectory as its zero point. Points located on one side of the normal direction of the standard sewing trajectory take positive values, and points located on the opposite side take negative values. The unit of s is consistent with the unit of the sewing path length, and the unit of d is consistent with the actual length unit after image calibration, and also consistent with the unit of the allowable range of trajectory offset. Through this trajectory coordinate system, the stitch pixel points and needle hole response pixel points in the image can be converted into the length position along the sewing path and the lateral offset position relative to the standard sewing trajectory.

[0095] S33. Calculate the coordinate transformation relationship between the image coordinate system and the trajectory coordinate system based on the position, tangential direction and normal direction of each trajectory sampling point in the image coordinate system;

[0096] Specifically, for a pixel q in the image coordinate system, find its nearest projection point p on the standard sewing trajectory, take the arc length corresponding to the projection point p as the length coordinate s of the pixel q, and take the projection value of the vector pq in the normal direction at the projection point p as the horizontal offset coordinate d.

[0097] It should be noted that by converting the pixel distance into the actual length through the camera calibration ratio, the lateral offset of the trajectory is made to be on the same dimension as the standard seam distance and the allowable deviation of the sewing process parameters.

[0098] S34. According to the coordinate transformation relationship, map the suture pixel points in the suture saliency map to the suture trajectory points in the trajectory coordinate system, and obtain the length coordinates and lateral offset coordinates corresponding to the suture trajectory points;

[0099] Specifically, in the saliency map, pixels that are continuously distributed along the standard sewing trajectory and whose saliency map response value is higher than that of the adjacent non-seam region are taken as saliency pixels. Then, according to the coordinate transformation relationship between the image coordinate system and the trajectory coordinate system, the saliency pixels are mapped to the trajectory coordinate system to obtain the saliency trajectory point set. The saliency trajectory point set includes the length coordinates, lateral offset coordinates, and saliency map response value of each saliency pixel.

[0100] S35. According to the coordinate transformation relationship, map the pinhole response pixels in the pinhole saliency map to the pinhole response points in the trajectory coordinate system, and obtain the length coordinates and lateral offset coordinates corresponding to the pinhole response points;

[0101] Specifically, in the pinhole saliency map, pixels with pinhole response values ​​higher than their surrounding neighboring pixels are taken as initial response points. When multiple initial response points are adjacent and form the same response region, the pixel with the largest pinhole response value in the response region is selected as the pinhole response point. Then, according to the coordinate transformation relationship between the image coordinate system and the trajectory coordinate system, the pinhole response points are mapped to the trajectory coordinate system to obtain the pinhole response point set. The pinhole response point set includes the length coordinate, lateral offset coordinate, and pinhole response value corresponding to each pinhole response point.

[0102] S36. Based on the mapped suture trajectory points and the mapped pinhole response points, generate the suture saliency distribution results and pinhole saliency distribution results in the trajectory coordinate system.

[0103] Specifically, the suture saliency distribution result is the suture saliency response distribution indexed by the length coordinate s and the lateral offset coordinate d, and the pinhole saliency distribution result is the pinhole saliency response distribution indexed by the length coordinate s and the lateral offset coordinate d.

[0104] It should be noted that both types of distribution results retain the correspondence with the original image coordinates, so that the defect results can be calculated from the trajectory coordinates back to the image area or the actual sewing position.

[0105] S4. Extract the stitch spacing unit in the trajectory coordinate system, and calculate the stitch spacing length, stitch width, needle hole response value, trajectory lateral offset, and stitch saliency response value in the corresponding area of ​​the standard sewing trajectory for the stitch spacing unit.

[0106] Furthermore, the step of extracting stitch spacing units in the trajectory coordinate system and calculating the stitch spacing length, seam width, needle hole response value, trajectory lateral offset, and seam saliency map response value within the corresponding area of ​​the standard sewing trajectory specifically includes:

[0107] S41. Extract candidate pinhole points from the pinhole significant distribution results;

[0108] Specifically, in the results of pinhole significant distribution, the point whose pinhole response value is higher than the response value of its surrounding adjacent position is taken as the initial response point. The pinhole response value is generated by the local dark spot response and shadow gradient results. When multiple initial response points are adjacent to form the same response area, the point with the largest pinhole response value in the response area is selected as the pinhole candidate point. The pinhole candidate point includes the length coordinate, the lateral offset coordinate and the pinhole response value.

[0109] S42. Perform sequence matching on the candidate pinhole points according to the standard pin spacing to obtain the effective pinhole point sequence;

[0110] Specifically, the candidate pinhole points are sorted in ascending order according to their length coordinate s, and the length distance between adjacent candidate points is calculated.

[0111] If the length spacing between adjacent candidate pinhole points is within the lower limit threshold of pin distance. Up to the upper limit of the needle spacing threshold Between, and the pinhole response value is not lower than the pinhole response threshold. If so, the corresponding candidate pinhole point will be retained as a valid pinhole point;

[0112] If the length spacing between adjacent pinhole candidate points is less than If the pinhole response value is higher, then the candidate pinhole points with higher pinhole response values ​​will be retained.

[0113] If the length spacing between adjacent pinhole candidate points is greater than If the stitch spacing is abnormal, the mark will be retained for subsequent judgment of stitch spacing abnormalities or skipped stitch defects.

[0114] S43. Divide the area between two adjacent valid pinhole points into a pin spacing unit;

[0115] Specifically, the length coordinate interval between the i-th effective pinhole point and the (i+1)-th effective pinhole point is taken as the i-th stitch distance unit, and the horizontal range of the stitch distance unit is the area corresponding to the standard sewing trajectory.

[0116] It should be noted that each pin spacing unit has a unique pin spacing unit number, which is used for subsequent topology map node and defect location.

[0117] S44. Calculate the center line of the suture within the stitch spacing unit based on the results of the significant suture distribution;

[0118] Specifically, within the length coordinate interval corresponding to the stitch spacing unit, the statistical results of the significant distribution of sutures along the normal direction are analyzed. The weighted center of the significant suture response or the ridge line with the maximum response is used as the center line of the suture within the stitch spacing unit. When there is no effective significant suture response in a local area, the area is reserved as a low response area for subsequent judgment of abnormal suture continuity or missing sutures.

[0119] S45. Based on the effective needle point sequence and the suture centerline, calculate the needle length, suture width, needle response value, trajectory lateral offset, and suture saliency response value within the corresponding area of ​​the standard sewing trajectory.

[0120] Specifically, the stitch length is the difference in length coordinates between two adjacent effective needle holes; the stitch width is the width of the salient response area of ​​the stitch within the stitch unit in the normal direction; the width is the distance between the two boundaries of the salient response area of ​​the stitch in the normal direction; the needle hole response value is the average value of the salient response of the needle holes at the location of the effective needle hole and in its adjacent area; the lateral offset of the trajectory is the normal offset of the center line of the stitch within the stitch unit relative to the standard sewing trajectory, and the normal offset is expressed by the lateral offset coordinate in the trajectory coordinate system; the salient response value of the stitch within the corresponding area of ​​the standard sewing trajectory is the average value of the salient response values ​​of each stitch within the corresponding area of ​​the standard sewing trajectory of the stitch unit.

[0121] S5. Using the stitch spacing unit as a node, establish edges between adjacent stitch spacing units along the sewing path in the trajectory coordinate system, construct a stitch spacing level stitch topology graph, and calculate the stitch continuity result based on the stitch spacing level stitch topology graph.

[0122] Specifically, the stitch-level topology graph is used to unify the local features of each stitch unit and the connection status between adjacent stitch units into the same data structure. Nodes reflect whether there are abnormalities in stitch spacing, width, needle holes or offset in a single stitch unit, and edges reflect whether the stitches between adjacent stitch units are continuous. This avoids confusion caused by relying solely on single-point thresholds for broken threads, skipped stitches and missed stitches.

[0123] Furthermore, the step of establishing edges between adjacent stitch spacing units along the sewing path in the trajectory coordinate system, using stitch spacing units as nodes, to construct a stitch spacing-level stitch topology graph, and calculating the stitch continuity result based on the stitch spacing-level stitch topology graph, specifically includes:

[0124] S51. Set each stitch unit as a node in the stitch-level stitch topology graph;

[0125] Specifically, nodes For the i-th stitch spacing unit, the node attributes include stitch spacing unit number, starting length coordinates, ending length coordinates, stitch spacing length, suture width, needle hole response value, trajectory lateral offset, and suture saliency response value within the corresponding area of ​​the standard sewing trajectory.

[0126] S52. Generate the adjacency relationship between two adjacent pin spacing units according to the arrangement order of the effective pinhole point sequence;

[0127] Specifically, according to the length coordinate order of the effective needle point sequence, the needle spacing units adjacent to each other along the sewing path are determined as adjacent needle spacing units, and the adjacency relationship between two adjacent needle spacing units is generated.

[0128] S53. Establish an edge between two adjacent stitch spacing units, the edge being used to indicate the suture connection state between two adjacent stitch spacing units;

[0129] Specifically, edge The connection area between the i-th stitch unit and the (i+1)-th stitch unit is located between the end position of the i-th stitch unit and the start position of the (i+1)-th stitch unit. It extends forward and backward along the standard sewing path to form the standard sewing path connection area, with an extension length less than the standard stitch length. Furthermore, it matches the connection length of adjacent stitch units corresponding to the stitch type, and is used to cover the area where there should be a suture connection between adjacent stitch units. The edge attributes include the connection area length range, the average value of the suture saliency map response within the connection area, the continuous response length, the response discontinuity length, and the suture continuity value. The connection area length range is used to record the start and end positions of the edge on the standard sewing trajectory. The average value of the suture saliency map response is used to reflect the strength of the suture response within the connection area. The continuous response length is used to record the length of continuous suture response within the connection area. The response discontinuity length is used to record the length of interruption of suture response within the connection area. The suture continuity value is used for subsequent comparison with the stitch continuity threshold.

[0130] S54. Write the stitch length, suture width, needle hole response value, and trajectory lateral offset to the corresponding node;

[0131] Specifically, after writing the stitch length, suture width, needle hole response value, and trajectory lateral offset into the corresponding node, the stitch length is used to assist in the judgment of stitch length abnormality and skipped stitch defects, the suture width is used to judge suture width abnormality, the needle hole response value is used to assist in the judgment of needle hole abnormality and skipped stitch defects, and the trajectory lateral offset is used to judge trajectory offset abnormality and suture offset defects.

[0132] S55. Calculate the suture continuity value of the edge based on the suture saliency response value between two adjacent stitch spacing units;

[0133] Specifically, within the connected region corresponding to the edge, the saliency map response value of the seam is statistically analyzed along the standard sewing trajectory direction to obtain the continuous response length within the connected region. Total length of the connecting area and the mean of significant response of the suture in the normalized connection area. and in accordance with Calculate the suture continuity value C.

[0134] Where α is the continuous response length weight and β is the suture significant response weight. Both α and β are values ​​in the range of 0 to 1, and α+β=1.

[0135] α and β are determined based on the statistical results of samples corresponding to the current stitch type. By obtaining qualified stitch samples and defective stitch samples of the same type as the current stitch, the ratio of continuous response lengths within the sample connection region is calculated respectively. / Significant response mean of sutures The first discrimination index is determined based on the average difference between qualified and defective trace samples in the ratio of continuous response length. The second discrimination index was determined based on the average difference between qualified and defective suture samples in terms of the mean of significant suture response. and in accordance with , Determine α and β, where ε is a correction factor to prevent the denominator from being zero.

[0136] When the ratio of continuous response lengths has a greater ability to distinguish between acceptable and defective stitches than the mean of significant suture responses, α is greater than β.

[0137] When the mean of the suture significant response has a greater distinguishing effect on acceptable and defective sutures than the distinguishing effect of the ratio of continuous response lengths, β is greater than α.

[0138] when and When both are zero or no defective stitch samples are obtained, α and β are both 0.5, and C ranges from 0 to 1. The larger C is, the more continuous the suture connection between adjacent stitch units is.

[0139] S56. Calculate the stitch continuity result based on the comparison between the suture continuity value and the stitch continuity threshold.

[0140] Specifically, when the suture continuity value C is lower than the suture continuity threshold... When this occurs, generate an anomaly result for the continuity of the trace corresponding to that edge;

[0141] When the stitch continuity values ​​of multiple consecutive edges are all below the stitch continuity threshold At that time, the connected areas corresponding to multiple abnormal edges are merged into continuous abnormal segments. The stitch continuity results include the abnormal edge number, the coordinates of the start and end lengths of the abnormal segment, the abnormal duration length, and the minimum stitch continuity value, which are used for subsequent determination of the location and severity of suture defects.

[0142] S6. Match the stitch length, stitch width, needle hole response value, trajectory lateral offset, stitch continuity result, and the saliency map response value of the stitch in the corresponding area of ​​the standard sewing trajectory with the sewing inspection benchmark, and combine with the defect judgment benchmark to obtain the sewing anomaly judgment result;

[0143] Furthermore, the process of matching the stitch length, thread width, needle hole response value, trajectory lateral offset, stitch continuity result, and the saliency map response value of the corresponding area of ​​the standard sewing trajectory with the sewing inspection benchmark, and combining this with the defect judgment benchmark, to obtain the sewing anomaly judgment result, specifically includes:

[0144] S61. Compare the stitch length with the stitch unit reference, the upper limit threshold of the stitch length, and the lower limit threshold of the stitch length to obtain the stitch length abnormality judgment result;

[0145] Specifically, let the stitch length of the i-th stitch unit be... The upper limit threshold for needle spacing is The lower limit threshold for needle spacing is If the stitch length Greater than the upper limit threshold of needle spacing If the stitch spacing is too large, then it is determined that the i-th stitch spacing unit has an abnormality. Less than the lower limit threshold of needle distance If the stitch spacing is too small, it is determined that there is an abnormality. In to If the distance is within a certain range, the stitch spacing is considered normal.

[0146] It should be noted that excessively large needle spacing indicates an abnormal quantity. according to Dimensionless transformation, when Less than When the needle spacing is too small, the abnormal quantity is... according to Dimensionless transformation, when In to In between, and All values ​​are set to 0. When performing defect scoring later, the abnormal pin spacing is considered... Pick and When determining the feed speed trend using the larger value in the range, the continuous stitch length unit is called respectively. Results of the changes.

[0147] S62. Compare the suture width with the suture width threshold to obtain the suture width anomaly determination result;

[0148] Specifically, let the suture width of the i-th stitch unit be... The suture width threshold includes the lower limit of width. and width limit suture width threshold Maximum width ,like Greater than or less If the suture width is abnormal in the i-th stitch length unit, then the abnormal suture width amount is determined. Used to indicate the degree to which the suture width deviates from the allowable range;

[0149] when Greater than hour, according to calculate;

[0150] when Less than hour, according to calculate;

[0151] when In to In between, Take 0.

[0152] S63. Compare the pinhole response value with the pinhole response threshold to obtain the pinhole anomaly determination result;

[0153] Specifically, let the pinhole response value of the i-th pin spacing unit be... If the pinhole response value Below the pinhole response threshold If the corresponding pinhole or pin spacing unit is found to have insufficient pinhole response, it is determined that there is an abnormality.

[0154] It should be noted that the pinhole response is insufficient and abnormal. according to Perform dimensionless transformation.

[0155] S64. Compare the lateral offset of the trajectory with the allowable range of trajectory offset to obtain the trajectory offset anomaly judgment result;

[0156] Specifically, let the lateral offset of the trajectory of the i-th needle spacing unit be... If the trajectory lateral offset The absolute value is greater than the allowable range of trajectory offset. If so, it is determined that the i-th needle spacing unit has an abnormal trajectory offset.

[0157] It should be noted that the trajectory deviation anomaly You can follow Perform dimensionless transformation.

[0158] S65. Compare the line continuity result with the line continuity threshold to obtain the line continuity anomaly judgment result;

[0159] Specifically, if the edge The suture continuity value C is lower than the stitch continuity threshold. If the line continuity is abnormal between the i-th stitch unit and the (i+1)-th stitch unit, then it is determined that there is an anomaly in the line continuity.

[0160] It should be noted that continuous outliers You can follow After dimensionless transformation, the results of the line continuity anomaly determination include the anomaly edge number, the anomaly connection area, the continuity anomaly quantity, and the anomaly duration.

[0161] S66. Compare the saliency response value of the seam saliency map within the area corresponding to the standard sewing trajectory with the saliency response threshold of the seam to obtain the result of the defective seam;

[0162] Specifically, let the saliency response value of the seam within the area corresponding to the standard sewing trajectory be... If the seam thread response value within the area corresponding to the standard sewing trajectory is significant Below the suture significant response threshold If there is no valid needle hole sequence that meets the stitch length unit reference within the area corresponding to the standard sewing trajectory, then the area is determined to have a missed stitch anomaly, and the missed stitch anomaly amount is... You can follow =max(0, The calculation is then used to score the severity of the leak defect.

[0163] S67. Generate sewing anomaly judgment results based on the results of the abnormal stitch distance judgment, abnormal stitch width judgment, abnormal needle hole judgment, abnormal trajectory deviation judgment, abnormal stitch continuity judgment, and abnormal missing stitch judgment.

[0164] Specifically, the sewing anomaly judgment results are indexed by the stitch distance unit number and the edge number, and the results of the stitch distance anomaly judgment, the sewing width anomaly judgment, the needle hole anomaly judgment, the trajectory offset anomaly judgment, the stitch continuity anomaly judgment, and the missing stitch anomaly judgment are recorded. The corresponding anomaly length coordinate range, lateral offset coordinate range, image coordinate range, and dimensionless anomaly quantity are also recorded.

[0165] When multiple abnormal conditions are met simultaneously in the same area, the corresponding abnormal judgment results are output respectively.

[0166] S7. Determine the type, location, and severity of the sewing defect based on the sewing anomaly judgment result, output the detection result of the sewing defect, and generate sewing process feedback information based on the detection result.

[0167] Furthermore, determining the type, location, and severity of the sewing defect based on the sewing anomaly assessment results specifically includes:

[0168] S711. When the pinhole response value is lower than the pinhole response threshold and the corresponding stitch length is greater than the upper limit of the stitch length, mark the location of the corresponding stitch length unit as the skipped stitch defect location and mark the type of sewing defect as skipped stitch defect;

[0169] Specifically, skipped stitch defects are triggered by both insufficient needle hole response and excessive stitch length. The location of the skipped stitch defect is determined by the length coordinate range of the corresponding stitch length unit, and this length coordinate range is then converted into the image coordinate area and the actual sewing path position.

[0170] S712. When the stitch continuity result is lower than the stitch continuity threshold, mark the position between adjacent stitch units as the location of the thread breakage defect, and mark the type of sewing defect as the thread breakage defect;

[0171] Specifically, the thread breakage defect is triggered by abnormal suture continuity on the edge; if the suture continuity value of a single edge is lower than... If so, the corresponding connected area of ​​that edge is marked as a candidate location for a broken line;

[0172] If multiple consecutive edges are lower than The multiple connected areas are then merged into a single broken thread defect segment, and the starting length coordinates, ending length coordinates, minimum suture continuity value, and number of abnormal edges of this segment are used as the basis for calculating the location and severity of the broken thread defect.

[0173] S713. When the lateral offset of the track of the adjacent stitch units along the sewing path exceeds the allowable range of track offset, the location of the adjacent stitch units is marked as the location of the sewing offset defect, and the type of sewing defect is marked as the sewing offset defect.

[0174] Specifically, the lateral offset of the trajectory of adjacent stitch units is compared sequentially according to the stitch unit number. When the absolute value of the lateral offset of the trajectory of adjacent stitch units along the sewing path is greater than the allowable range of trajectory offset, the group of adjacent stitch units is determined as a continuous offset segment.

[0175] It should be noted that the starting length coordinates, ending length coordinates, and lateral offset of the trajectory of the continuous offset segment are used to determine the location of the seam offset defect and to generate fabric guide position feedback information.

[0176] S714. When the saliency response value of the seam in the area corresponding to the standard sewing trajectory is lower than the saliency response threshold and there is no valid needle hole sequence that meets the needle distance unit reference in the area corresponding to the standard sewing trajectory, the corresponding area is marked as the location of the missing seam defect, and the type of the sewing defect is marked as a missing seam defect.

[0177] Specifically, when the saliency response value of the seam within the area corresponding to the standard sewing trajectory is lower than the saliency response threshold of the seam, When the length position is recorded as a low response position, the continuity of adjacent low response positions is judged along the standard sewing trajectory direction. When the length interval between two adjacent low response positions is not greater than the trajectory sampling interval, they are classified into the same continuous low response region. The length coordinate corresponding to the first low response position in this continuous low response region is used as the starting length coordinate of the leak defect, and the length coordinate corresponding to the last low response position is used as the ending length coordinate of the leak defect. Based on the starting length coordinate and the ending length coordinate, the length range corresponding to the leak defect is determined in the trajectory coordinate system. Combined with the horizontal range of the corresponding area of ​​the standard sewing trajectory, the region of the leak defect in the trajectory coordinate system is obtained. Then, according to the coordinate transformation relationship between the trajectory coordinate system and the image coordinate system, the region is converted back to the image coordinate system to obtain the image region corresponding to the leak defect.

[0178] S715. Calculate the defect score based on the number of stitch units covered by the defect, the duration of the defect on the standard sewing track, the lateral offset of the track, and the stitch continuity results.

[0179] Specifically, before calculating the defect score, each input quantity is dimensionless to cover abnormal quantities in the number of pin spacing units. The number of defect-covered pin spacing units n and the threshold for the number of severely defective pins. The ratio is truncated to obtain the result;

[0180] Duration anomaly Defect duration L and duration threshold The critical defect needle count threshold is obtained by truncating the ratio. and duration threshold Determined by the quality requirements and allowable deviations of sewing process parameters in the garment manufacturing documents;

[0181] Trajectory deviation anomaly Obtained from S64;

[0182] Continuous outliers Obtained from S65;

[0183] Insufficient pinhole response abnormality and abnormal amount of leaks These are obtained from S63 and S66 respectively.

[0184] For different defect types, select the corresponding outlier to calculate the defect score. For example, skipped needle defect scoring can be done by , , and Weighted average;

[0185] The scoring of broken wire defects can be done by , and Weighted average;

[0186] The suture misalignment defect score can be determined by , and Weighted average;

[0187] Leakage defect scoring can be done by , and The weighted average is obtained.

[0188] It should be noted that each weight is read from the garment process file. When the garment process file does not record the corresponding weight, each weight in the same defect scoring formula takes the same value.

[0189] S716. Determine the severity of sewing defects based on defect scores.

[0190] Specifically, the defect score is matched with a severity threshold range to generate a severity level of mild, moderate, or severe. The defect score is a dimensionless value ranging from 0 to 1, and the severity threshold range includes the first severity threshold. Second severity threshold , where 0≤ < ≤1;

[0191] First severity threshold Second severity threshold Based on the defect score sample distribution corresponding to the defect type, the first severity threshold is determined by acquiring minor defect samples, general defect samples, and severe defect samples of the same type as the current defect, and obtaining the defect score for each sample according to the defect scoring calculation method. The first severity threshold is determined based on the upper quantile of the defect score for minor defect samples and the lower quantile of the defect score for general defect samples. The second severity threshold is determined based on the upper quantile of the defect score for general defect samples and the lower quantile of the defect score for severe defect samples. ;

[0192] When no defect score sample for the corresponding defect type is obtained, the first severity threshold is determined based on the allowable rework boundary and downtime judgment boundary for the corresponding defect type in the garment process document. Second severity threshold Among them, the defect score corresponding to the allowable rework boundary is used as the first severity threshold T1, and the defect score corresponding to the downtime determination boundary is used as the second severity threshold. When the garment manufacturing process documents do not specify the permissible rework boundaries and the stoppage decision boundaries, Take 1 / 3, and Take 2 / 3.

[0193] When the defect score is less than When this happens, the severity of the corresponding defect is marked as minor;

[0194] When the defect score is greater than or equal to and less than When this happens, the severity of the corresponding defect is marked as "moderate".

[0195] When the defect score is greater than or equal to When the severity of the corresponding defect is marked as severe, the severity level of the defect will be marked as severe.

[0196] When multiple defect types exist in the same area, the defect score corresponding to each defect type is calculated separately, and the scores are compared according to their severity levels. The highest severity level is used as the output of the severity of the area.

[0197] Furthermore, the output of sewing defect detection results and the generation of sewing process feedback information based on the detection results specifically include:

[0198] S721. Output the type, location, severity, and corresponding stitch length unit number of the sewing defect;

[0199] Specifically, the detection results include defect type, defect location, severity, defect score, corresponding stitch distance unit number, and trajectory coordinates and image coordinates used for rework positioning. The detection results are output to a display terminal, quality management system, sewing equipment controller, or rework positioning system.

[0200] S722. When the stitch length of adjacent stitch units along the sewing path increases sequentially, feedback information is generated to reduce the fabric feed speed;

[0201] Specifically, the stitch lengths of adjacent stitch units are compared sequentially according to their stitch unit numbers. When the stitch length of the previous stitch unit is less than the stitch length of the next stitch unit, and this increasing relationship occurs continuously along the sewing path, feedback information to reduce the fabric feed speed is generated.

[0202] It should be noted that the increasing relationship is obtained by directly comparing the stitch lengths of adjacent stitch units, and the comparison objects are adjacent stitch units in the same effective pinhole point sequence.

[0203] S723. When the stitch length of adjacent stitch units along the sewing path decreases one by one, feedback information is generated to increase the fabric feed speed;

[0204] Specifically, the stitch lengths of adjacent stitch units are compared sequentially according to their stitch unit numbers. When the stitch length of the previous stitch unit is greater than the stitch length of the next stitch unit, and this decreasing relationship occurs continuously along the sewing path, feedback information to increase the fabric feeding speed is generated.

[0205] S724. When the lateral offset of the stitch length units adjacent to each other along the sewing path exceeds the allowable range of the trajectory offset, feedback information is generated to adjust the fabric guide position.

[0206] Specifically, the adjustment direction is determined by the sign of the lateral offset of the trajectory within the continuous offset segment. When the lateral offset of the trajectory is positive, the feedback information indicates that the fabric guide position should be adjusted in the negative normal direction. When the lateral offset of the trajectory is negative, the feedback information indicates that the fabric guide position should be adjusted in the positive normal direction.

[0207] The adjustment range is the average absolute value of the lateral offset of the trajectory of each stitch unit within the continuous offset segment, used to indicate the adjustment amount of the fabric guide position relative to the current guide position;

[0208] Based on the feedback information mentioned above, the direction of the fabric guide position adjustment is opposite to the direction of the sewing thread offset, so as to reduce the lateral offset of the subsequent sewing thread relative to the standard sewing trajectory.

[0209] S725. When the suture width is greater than the suture width threshold and the shadow gradient result of the corresponding area of ​​the stitch spacing unit in the second image is greater than the shadow gradient threshold, feedback information for adjusting the thread tension is generated;

[0210] Specifically, this feedback condition requires both abnormal seam width and enhanced shadow gradient in the oblique incidence image to avoid false triggering caused solely by fabric texture or lighting fluctuations. When the seam width exceeds a seam width threshold... When the shadow gradient result of the corresponding area of ​​the stitch spacing unit in the second image is greater than the shadow gradient threshold, feedback information for adjusting the thread tension is generated. When only the suture width abnormality or only the shadow gradient abnormality is met, no feedback information for adjusting the thread tension is generated.

[0211] S726. When there are defects such as skipped needles, broken threads, or leaks, generate alarm information, shutdown information, and rework location information according to the type, location, and severity of the corresponding defects.

[0212] Specifically, when skipped stitches, broken threads, or missing stitches are present, the result output and feedback module generates alarm information, shutdown information, and rework location information based on the type, location, and severity of the corresponding defect. The alarm information indicates that the corresponding type of defect exists in the current fabric area to be inspected. The shutdown information instructs the sewing equipment to stop the current sewing action based on the severity of the corresponding defect. The rework location information includes the stitch length unit number, starting length coordinates, ending length coordinates, and corresponding image area of ​​the corresponding defect.

[0213] Reference Figure 6 As shown, a vision-based garment sewing defect detection system is proposed to implement the vision-based garment sewing defect detection method described in any of the above embodiments, including:

[0214] The reference processing module is used to acquire sewing reference data of the fabric area to be inspected and generate a sewing inspection reference that includes standard sewing trajectory, stitch distance unit reference, trajectory offset allowable range and defect judgment reference.

[0215] Furthermore, the benchmark processing module includes:

[0216] The reference data acquisition unit is used to acquire sewing reference data from garment process documents, pattern data, sewing equipment control programs, camera imaging parameters, and sewing equipment positioning accuracy parameters;

[0217] The inspection benchmark generation unit is used to generate a sewing inspection benchmark that includes a standard sewing trajectory, a stitch distance unit benchmark, a trajectory offset allowable range, and a defect judgment benchmark based on the sewing benchmark data.

[0218] The image acquisition and preprocessing module is used to simultaneously acquire a first image of the fabric area to be detected under forward diffuse lighting and a second image of the oblique incident lighting during the sewing process, and to perform synchronous compensation and feature enhancement processing on the first image of the forward diffuse lighting and the second image of the oblique incident lighting.

[0219] Furthermore, the image acquisition and preprocessing module includes:

[0220] The synchronous acquisition control unit is used to acquire the sewing machine needle bar position signal and fabric feed displacement signal, determine the acquisition time of the needle pitch cycle and the displacement of the fabric area to be detected in the image, synchronously acquire the first image of forward diffuse lighting and the second image of oblique incident lighting at the acquisition time, and synchronously compensate the first image of forward diffuse lighting and the second image of oblique incident lighting according to the displacement.

[0221] The image enhancement processing unit is used to perform pixel registration on the first image under forward diffuse lighting and the second image under oblique incident lighting, calculate the color normalization result and the shadow gradient result, and generate a seam saliency map and a pinhole saliency map based on the color normalization result and the shadow gradient result.

[0222] The feature extraction and analysis module is used to establish a trajectory coordinate system and map a saliency map, extract the stitch spacing unit and calculate the line feature parameters, construct a stitch spacing level line topology map and calculate the line continuity result;

[0223] Furthermore, the feature extraction and analysis module includes:

[0224] The coordinate transformation and mapping unit is used to establish a trajectory coordinate system based on the standard sewing trajectory and to map the salient lines and needle holes to the trajectory coordinate system.

[0225] The stitch feature calculation unit is used to extract the stitch distance unit in the trajectory coordinate system, calculate the stitch distance length, suture width, needle hole response value, trajectory lateral offset, and suture saliency response value in the corresponding area of ​​the standard sewing trajectory. It also constructs a stitch distance-level stitch topology graph and calculates the stitch continuity result by using the stitch distance unit as a node and the connection relationship between adjacent stitch distance units along the sewing path as an edge.

[0226] The anomaly detection module is used to match the extracted stitch features with the sewing inspection benchmark, and combine the defect detection benchmark to obtain the sewing anomaly detection result.

[0227] Furthermore, the anomaly detection module includes:

[0228] The feature matching and determination unit is used to match the needle length, suture width, needle hole response value, trajectory lateral offset, stitch continuity result, and suture saliency map response value in the corresponding area of ​​the standard sewing trajectory with the needle length unit reference, the trajectory offset allowable range, and the defect determination reference, respectively, to obtain the needle length abnormality determination result, suture width abnormality determination result, needle hole abnormality determination result, trajectory offset abnormality determination result, stitch continuity abnormality determination result, and missing stitch abnormality determination result;

[0229] The defect information generation unit is used to determine the type, location, and severity of sewing defects based on the sewing anomaly judgment results;

[0230] The results output and feedback module is used to output the sewing defect detection results and generate sewing process feedback information based on the detection results.

[0231] This solution acquires a first image of the fabric area under forward diffuse lighting and a second image of the fabric under oblique incident lighting. Based on these two types of images, it generates a seam saliency map and a pinhole saliency map, respectively, enhancing the features of seams and pinholes and suppressing background interference, thereby improving the accuracy of defect identification in complex fabric scenarios. At the same time, it establishes a trajectory coordinate system that extends along the path based on the standard sewing trajectory, mapping the seam saliency map and the pinhole saliency map to the trajectory coordinate system. In the trajectory coordinate system, it extracts the stitch distance unit and calculates the stitch distance length, seam width, pinhole response value, trajectory lateral offset, and other features of each unit, achieving accurate detection of defects at the single stitch distance scale and solving the problem of missed detection of small-scale, short-duration defects.

[0232] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for detecting garment sewing defects based on visual recognition, characterized in that, include: Obtain sewing reference data for the fabric area to be inspected, and generate a sewing inspection reference that includes a standard sewing trajectory, a stitch distance unit reference, a trajectory offset allowable range, and a defect judgment reference. The sewing reference data includes the sewing path, standard stitch distance, standard seam distance, stitch type, and allowable deviation of sewing process parameters. During the sewing process, the first image of the fabric area to be detected under normal diffuse lighting and the second image of the oblique incident lighting are simultaneously acquired to generate a seam saliency map and a pinhole saliency map, respectively. Establish a trajectory coordinate system and map the suture salience map and the pinhole salience map to the trajectory coordinate system; Extract the stitch spacing unit in the trajectory coordinate system, and calculate the stitch spacing length, suture width, needle hole response value, trajectory lateral offset, and suture saliency response value in the corresponding area of ​​the standard sewing trajectory. Using the stitch spacing unit as a node, edges are established between adjacent stitch spacing units along the sewing path in the trajectory coordinate system to construct a stitch spacing-level stitch topology graph, and the stitch continuity result is calculated based on the stitch spacing-level stitch topology graph. The stitch length, suture width, needle hole response value, trajectory lateral offset, stitch continuity result, and suture saliency response value in the corresponding area of ​​the standard sewing trajectory are matched with the sewing inspection benchmark. Combined with the defect judgment benchmark, the sewing anomaly judgment result is obtained. Based on the sewing anomaly assessment results, determine the type, location, and severity of the sewing defects, output the detection results, and generate sewing process feedback information based on the detection results.

2. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that: The generation of the sewing inspection benchmark, which includes a standard sewing trajectory, a stitch distance unit reference, a permissible trajectory offset range, and a defect judgment benchmark, specifically includes: Generate a standard sewing trajectory based on the sewing path; Generate stitch spacing unit references based on standard stitch spacing and stitch type; The allowable range of trajectory offset is generated based on the standard seam distance and the allowable deviation of sewing process parameters; Defect judgment criteria are generated based on the standard stitch length, stitch type, and allowable deviations of sewing process parameters. The defect judgment criteria include the needle hole response threshold, the upper limit threshold of stitch length, the lower limit threshold of stitch length, the suture width threshold, the stitch continuity threshold, the suture significant response threshold, and the shadow gradient threshold.

3. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that: The process of simultaneously acquiring a first image of the fabric area under forward diffuse lighting and a second image of the fabric under oblique incident lighting during the sewing process, and generating a seam saliency map and a pinhole saliency map respectively, specifically includes: Acquire the position signal of the sewing machine needle bar and the fabric feed displacement signal; The acquisition time of the needle pitch cycle is determined based on the needle bar position signal, and the displacement of the fabric area to be detected in the image is calculated based on the fabric feed displacement signal. At the acquisition time, the first image of the forward diffuse illumination and the second image of the oblique incident illumination are acquired simultaneously, and the first image of the forward diffuse illumination and the second image of the oblique incident illumination are compensated synchronously according to the displacement. Pixel registration is performed on the first image under forward diffuse lighting and the second image under oblique incident lighting to obtain the registered first image under forward diffuse lighting and the second image under oblique incident lighting. The color normalization result of the fabric region to be detected is calculated based on the first image of the registered forward diffused light, and the shadow gradient result of the fabric region to be detected is calculated based on the second image of the registered oblique incident light. Based on the color normalization and shadow gradient results, the seam area in the fabric region to be detected is enhanced and the fabric texture area is suppressed to obtain the seam saliency map. Based on the stitch discontinuities and local dark spot responses in the seam saliency map, the pinhole saliency map is obtained.

4. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that: The establishment of the trajectory coordinate system, which maps the suture salience map and the pinhole salience map to the trajectory coordinate system, specifically includes: The standard sewing trajectory is discretized into multiple trajectory sampling points, and the tangential and normal directions at each trajectory sampling point are calculated. Using the tangential direction as the first coordinate direction and the normal direction as the second coordinate direction, a trajectory coordinate system is established that extends along the standard sewing trajectory. The first coordinate direction represents the length position along the sewing path, and the second coordinate direction represents the lateral offset position of the sewing thread relative to the standard sewing trajectory. Based on the position, tangential direction, and normal direction of each trajectory sampling point in the image coordinate system, calculate the coordinate transformation relationship between the image coordinate system and the trajectory coordinate system; According to the coordinate transformation relationship, the suture pixels in the suture saliency map are mapped to the suture trajectory points in the trajectory coordinate system, and the length coordinates and lateral offset coordinates corresponding to the suture trajectory points are obtained. According to the coordinate transformation relationship, the pinhole response pixels in the pinhole saliency map are mapped to pinhole response points in the trajectory coordinate system, and the length coordinates and lateral offset coordinates corresponding to the pinhole response points are obtained. Based on the mapped suture trajectory points and the mapped pinhole response points, generate salient distribution results of sutures and pinholes in the trajectory coordinate system.

5. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that: The process of extracting stitch spacing units in the trajectory coordinate system and calculating the stitch spacing length, seam width, needle hole response value, trajectory lateral offset, and seam saliency map response value within the corresponding area of ​​the standard sewing trajectory specifically includes: Extract candidate pinhole points from the results of significant pinhole distribution; Based on the standard pin spacing, sequence matching is performed on candidate pinhole points to obtain a valid pinhole point sequence; Divide the area between two adjacent valid pinhole points into a pin spacing unit; Calculate the suture centerline within the stitch spacing unit based on the suture distribution results; Based on the effective needle point sequence and the suture centerline, calculate the needle length, suture width, needle response value, trajectory lateral offset, and suture saliency response value within the corresponding area of ​​the standard sewing trajectory.

6. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that, The process involves establishing edges between adjacent stitch spacing units along the sewing path in the trajectory coordinate system, using stitch spacing units as nodes, constructing a stitch spacing-level stitch topology graph, and calculating stitch continuity results based on the stitch spacing-level stitch topology graph. Specifically, this includes: Each stitch unit is set as a node in the stitch-level trace topology graph; Generate the adjacency relationship between two adjacent pin spacing units according to the arrangement order of the effective pinhole point sequence; An edge is established between two adjacent stitch spacing units, the edge being used to indicate the suture connection state between two adjacent stitch spacing units; Write the stitch length, suture width, needle hole response value, and trajectory lateral offset to the corresponding node; The suture continuity value of the edge is calculated based on the suture saliency response value between two adjacent stitch spacing units. The stitch continuity result is calculated based on the comparison between the suture continuity value and the stitch continuity threshold.

7. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that, The process involves matching the stitch length, suture width, needle hole response value, lateral offset of the sewing trajectory, stitch continuity results, and the saliency map response value of the suture within the corresponding area of ​​the standard sewing trajectory with the sewing inspection benchmark. Combined with the defect judgment benchmark, this yields the sewing anomaly judgment result, specifically including: The needle spacing length is compared with the needle spacing unit reference, the upper limit threshold of needle spacing, and the lower limit threshold of needle spacing to obtain the needle spacing abnormality judgment result; The suture width is compared with the suture width threshold to obtain the suture width anomaly determination result; The pinhole response value is compared with the pinhole response threshold to obtain the pinhole anomaly determination result; The lateral offset of the trajectory is compared with the allowable range of trajectory offset to obtain the trajectory offset anomaly judgment result; The line continuity result is compared with the line continuity threshold to obtain the line continuity anomaly judgment result; The sewing saliency response value of the sewing line saliency map within the area corresponding to the standard sewing trajectory is compared with the sewing saliency response threshold to obtain the result of the missing sewing anomaly judgment; Based on the results of abnormal stitch length, abnormal stitch width, abnormal needle hole, abnormal trajectory deviation, abnormal stitch continuity, and abnormal missed stitch, a sewing abnormality judgment result is generated.

8. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that, The determination of the type, location, and severity of sewing defects based on the sewing anomaly assessment results specifically includes: When the pinhole response value is lower than the pinhole response threshold and the corresponding stitch length is greater than the upper limit of the stitch length, the location of the corresponding stitch length unit is marked as the skipped stitch defect location, and the type of sewing defect is marked as skipped stitch defect. When the stitch continuity result is lower than the stitch continuity threshold, the position between adjacent stitch units is marked as the location of the thread breakage defect, and the type of sewing defect is marked as the thread breakage defect. When the lateral offset of the track of each of the adjacent stitch units along the sewing path exceeds the allowable range of track offset, the location of the adjacent stitch units is marked as the location of the sewing offset defect, and the type of sewing defect is marked as the sewing offset defect. When the saliency response value of the seam in the area corresponding to the standard sewing trajectory is lower than the saliency response threshold and there is no valid needle hole sequence that meets the needle distance unit reference in the area corresponding to the standard sewing trajectory, the corresponding area is marked as the location of the missing seam defect, and the type of the sewing defect is marked as a missing seam defect. The defect score is calculated based on the number of stitch units covered by the defect, the duration of the defect on the standard sewing track, the lateral offset of the track, and the stitch continuity results. The severity of sewing defects is determined based on the defect score.

9. The method for detecting garment sewing defects based on visual recognition according to claim 1, characterized in that, The process involves determining the type, location, and severity of sewing defects based on the sewing anomaly assessment results, outputting the detection results, and generating sewing process feedback information based on the detection results. Specifically, this includes: Output the type, location, severity, and corresponding stitch length unit number of the sewing defect; When the stitch length of adjacent stitch units along the sewing path increases one by one, feedback information is generated to reduce the fabric feed speed. When the stitch length of adjacent stitch units along the sewing path decreases one by one, feedback information is generated to increase the fabric feeding speed. When the lateral offset of the stitch spacing units adjacent to each other along the sewing path exceeds the allowable range of trajectory offset, feedback information is generated to adjust the fabric guide position. When the suture width is greater than the suture width threshold and the shadow gradient result of the corresponding area of ​​the stitch spacing unit in the second image is greater than the shadow gradient threshold, feedback information for adjusting the suture tension is generated. When defects such as skipped needles, broken threads, or leaks are present, alarm information, shutdown information, and rework location information are generated based on the type, location, and severity of the corresponding defects.

10. A vision-based garment sewing defect detection system, used to implement the vision-based garment sewing defect detection method as described in any one of claims 1-9, characterized in that, include: A reference processing module is used to acquire sewing reference data of the fabric area to be inspected and generate a sewing inspection reference that includes a standard sewing trajectory, a stitch distance unit reference, a trajectory offset allowable range, and a defect judgment reference. The reference processing module includes: The reference data acquisition unit is used to acquire sewing reference data for the fabric area to be inspected. The inspection benchmark generation unit is used to generate a sewing inspection benchmark that includes a standard sewing trajectory, a stitch distance unit benchmark, a trajectory offset allowable range, and a defect judgment benchmark based on the sewing benchmark data. An image acquisition and preprocessing module is used to simultaneously acquire a first image of the fabric area to be inspected under forward diffuse lighting and a second image of the fabric under oblique incident lighting during the sewing process, and to perform synchronous compensation and feature enhancement processing on the images. The image acquisition and preprocessing module includes: The synchronous acquisition control unit is used to acquire the sewing machine needle bar position signal and fabric feed displacement signal, calculate the acquisition time and image displacement, synchronously acquire the first image under forward diffuse lighting and the second image under oblique incident lighting and perform synchronous compensation. The image enhancement processing unit is used to perform pixel registration on the first image under forward diffuse lighting and the second image under oblique incident lighting, calculate the color normalization result and the shadow gradient result, and generate the salience map of the seam and the salience map of the pinhole respectively. The feature extraction and analysis module is used to establish a trajectory coordinate system and map a saliency map, extract stitch spacing units and calculate stitch feature parameters, construct a stitch spacing-level stitch topology map and calculate stitch continuity results. The feature extraction and analysis module includes: The coordinate transformation and mapping unit is used to establish a trajectory coordinate system based on the standard sewing trajectory and to map the salient lines and needle holes to the trajectory coordinate system. The stitch feature calculation unit is used to extract the stitch distance unit in the trajectory coordinate system, calculate the stitch distance length, suture width, needle hole response value, trajectory lateral offset, and suture saliency map response value in the corresponding area of ​​the standard sewing trajectory, construct the stitch distance level stitch topology map, and calculate the stitch continuity result. An anomaly detection module is used to match the extracted stitch features with the sewing inspection benchmark, and combine the results with the defect detection benchmark to obtain a sewing anomaly detection result. The anomaly detection module includes: The feature matching and determination unit is used to match the stitch length, suture width, needle hole response value, trajectory lateral offset, stitch continuity result, and suture saliency map response value in the corresponding area of ​​the standard sewing trajectory with the sewing detection benchmark to obtain the stitch length abnormality determination result, suture width abnormality determination result, needle hole abnormality determination result, trajectory offset abnormality determination result, stitch continuity abnormality determination result, and missing stitch abnormality determination result. The defect information generation unit is used to determine the type, location, and severity of sewing defects based on the sewing anomaly judgment results; The results output and feedback module is used to output the sewing defect detection results and generate sewing process feedback information based on the detection results.

Images