Fabric hole defect detection method and system based on multi-frame fusion

Abstract

The application discloses a fabric hole defect detection system based on multi-frame fusion. The system collects continuous image frames in the process of fabric continuous conveying, combines displacement data to complete image registration, and constructs a space-time alignment image sequence under a unified coordinate system; texture constraint parameters are formed by counting the main direction and periodic scale of the warp and weft of the fabric; the structure filter is oriented and scaled according to the texture constraint, and a structure response result is generated; time consistency mapping is established between multiple frames, and abnormal response points that continuously appear are extracted; the abnormal points are converted into tensor representation and subjected to neighborhood voting accumulation to form a structure tensor field; the hole area is determined through eigenvalue analysis and direction consistency discrimination, and defect space coordinates and boundary contour data are output, so that stable detection of the fabric hole is realized.

Description

Technical Field

[0001] This invention relates to the field of textile quality inspection technology, and in particular to a method and system for detecting fabric holes and defects based on multi-frame fusion. Background Technology

[0002] With the improvement of automation and intelligent manufacturing in the textile industry, online quality inspection of fabrics during continuous high-speed conveying has become an important part of weaving enterprises to achieve stable production and quality control. At present, industrial cameras are generally used on production lines to image the fabric surface in real time, and image processing algorithms are used to identify and alarm for defects such as yarn breaks, stains, and holes.

[0003] Existing fabric hole detection technologies still have significant shortcomings under complex texture backgrounds and continuous motion conditions. On the one hand, traditional single-frame detection methods mainly rely on grayscale thresholding, edge detection, or simple filtering operators for defect extraction, without incorporating the fabric's warp and weft periodic texture features for constraint. This makes it easy to misjudge normal texture undulations as defects or miss real holes in dense texture areas, making it difficult to balance detection sensitivity and stability. On the other hand, conventional methods lack cross-frame temporal consistency analysis mechanisms and cannot effectively utilize the spatial correspondence between consecutive images. Under fabric shaking, lighting fluctuations, or random noise interference, instantaneous false detection points are easily generated, leading to large fluctuations in detection results.

[0004] Existing technologies mostly adopt direct discrimination methods based on pixel gradients or local response amplitudes, lacking comprehensive analysis of structural continuity and directional consistency. They fail to model the structural morphology at the tensor level, making it difficult to accurately depict the defect boundary morphology when faced with irregular hole edges and large size variations, easily leading to problems such as boundary breakage or inaccurate regional expansion.

[0005] Therefore, how to provide a method and system for detecting fabric holes and defects based on multi-frame fusion is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] This invention proposes a method and system for detecting fabric holes and defects based on multi-frame fusion. By continuously acquiring multiple frames of images during the fabric transport process and combining them with displacement data to achieve sub-pixel level registration, a spatiotemporally aligned image sequence is constructed. Texture constraint parameters are formed by statistically analyzing the main warp and weft directions and periodic scale of the fabric. The texture constraints are used to orient and scale a structure-selective filter to obtain a stable structural response. Then, reliable anomaly point sets are extracted through cross-frame trajectory association and continuous frame counting. A structural tensor field is generated using tensor voting, and the hole area markers, defect coordinates, and boundary contour data are output by combining feature spectrum and orientation consistency discrimination. This method has the advantages of strong anti-texture interference capability, strong cross-frame noise reduction capability, and high accuracy in defect localization and boundary extraction.

[0007] The fabric hole defect detection method and system based on multi-frame fusion according to embodiments of the present invention includes the following steps: The image acquisition and displacement calibration module acquires continuous image frames during the continuous fabric conveying process, and performs sub-pixel-level spatial registration of adjacent image frames based on the displacement data output by the conveying encoder to construct a spatiotemporally aligned image sequence. The fabric principal direction statistics module calculates the warp and weft principal direction vectors and corresponding periodic scale parameters in the spatiotemporally aligned image sequence, and generates a set of fabric periodic texture constraint parameters. The COSFIRE filtering module is improved by configuring the support direction set, scale parameter set, and response weight of the COSFIRE filter in a structured manner based on the fabric periodic texture constraint parameter set, and calculating the structure-selective response map on each frame image. The matrix construction module establishes a pixel-level time index matrix in the structure selection response layer, performs cross-frame trajectory association and continuous frame counting on response points, and outputs a set of structural anomaly points that meet the continuous frame threshold condition. The tensor voting computation module converts the set of structural anomalies into a second-order symmetric tensor representation, performs tensor voting operations within a limited neighborhood, and accumulates the structural tensor field. The tensor feature spectrum discrimination module performs eigenvalue decomposition on the structural tensor field, calculates the eigenvalue ratio and the rate of change of the principal direction, generates the marking results of the fabric hole area based on the eigenvalue spectrum change, and outputs the spatial coordinates and boundary contour data of the defect.

[0008] Optionally, modules can be integrated using the following methods: S1. Collect continuous image frames of the same area during the continuous fabric conveying process, and perform spatial registration of the continuous image frames based on the displacement data output by the conveying encoder to construct a spatiotemporally aligned image sequence. S2. Calculate the main direction vectors of the warp and weft directions of the fabric and the corresponding periodic scale parameters in the spatiotemporally aligned image sequence to generate a set of fabric periodic texture constraint parameters. S3. Based on the fabric periodic texture constraint parameter set, constrain the support direction set and scale parameter set of the COSFIRE filter, and calculate the structure-selective response map on each frame image. S4. Establish a time consistency mapping matrix in the structure selection response layer, perform cross-frame trajectory association and continuous frame counting on response points, and extract the set of structural anomalies that meet the continuous frame threshold conditions. S5. Convert the set of structural anomalies into a second-order symmetric tensor representation, perform tensor voting operation within a limited neighborhood, and generate a structural tensor field. S6. Perform eigenvalue decomposition on the structural tensor field, calculate the eigenvalue ratio and the rate of change of the principal direction, generate the marking results of the fabric hole area based on the eigenvalue spectrum change, and output the spatial coordinates and boundary contour data of the defect.

[0009] Optionally, S2 includes: S21. Establish a fixed-size two-dimensional analysis grid within the common overlapping rectangular region of the spatiotemporally aligned image sequence. Perform mean removal processing on the pixel grayscale values ​​within each grid cell, and calculate the gradient components in the horizontal and vertical directions to construct the structure tensor matrix of each grid cell: ; in, and These are the gradient components of the pixel in the horizontal and vertical directions, respectively; S22. Perform eigenvalue decomposition on the structural tensor matrix, take the eigenvector direction corresponding to the largest eigenvalue as the main structural direction of the grid unit, and perform direction clustering statistics on the main structural directions of each grid unit in all frames, selecting the two orthogonal directions with the highest frequency as the main warp direction angles of the fabric. With the principal direction angle of the latitudinal direction ; S23, respectively with and Assuming the projection direction, equally spaced sampling sequences are extracted along the corresponding direction within each grid cell. A one-dimensional fast Fourier transform is then performed on the sampling sequences, and the frequency with the largest amplitude in the amplitude spectrum, excluding zero frequency, is selected. As the dominant frequency component, its corresponding period scale is calculated. ; S24. Within all frames and all grid cells, perform median statistics on the meridional and zonal periodic scales respectively, and remove outliers that deviate from the median by more than a preset proportion to obtain a stable meridional periodic scale. Compared with zonal periodic scale ; S25, with the main direction angle of the meridian. Latitudinal direction angle Meridional periodic scale and zonal periodic scale Construct a set of parameters to constrain the periodic texture of the fabric.

[0010] Optionally, S3 includes: S31. Read the warp principal direction angle from the fabric periodic texture constraint parameter set. Latitudinal direction angle Meridional periodic scale latitudinal periodic scale Within the common overlapping rectangular area, the reference frame of the COSFIRE structure template is determined, and the template construction area is delineated within the reference frame; S32. Scan candidate center points within the template construction area at fixed pixel intervals, extract a local window of fixed size for each candidate center point, calculate the average gradient magnitude within the local window, select the candidate center point with the largest average gradient magnitude as the template center point, and determine the local window corresponding to the template center point as the reference structure segment. S33. Within the reference structure segment, generate a set of sampling positions using the "concentric rings - fixed angle step size" method. Specifically, generate multiple layers of concentric rings with the template center point as the origin, and the radius of each concentric ring is taken as... and The smaller values ​​are integer multiples and incremented by layer number; a candidate sampling direction angle list is generated on each concentric ring with a fixed angular step size, and each sampling position is recorded as a polar coordinate binary tuple of "ring radius, direction angle"; S34. Perform deterministic elimination on the candidate sampling direction angle list. The elimination rule is: when a candidate direction angle is... When the included angle is less than a preset direction exclusion angle threshold, the candidate direction angle is discarded; when the included angle is less than a preset direction exclusion angle threshold, the candidate direction angle is discarded. When the included angle is less than the preset direction exclusion angle threshold, the candidate direction angle is eliminated, the remaining direction angles are retained to form a support direction set, and the support direction set is applied to each layer of concentric rings to obtain a set of sampling positions consisting only of the retained direction angles; S35. Construct local primitive responses within the reference structural segment, specifically: perform multi-scale differential Gaussian filtering on the reference structural segment to obtain a set of primitive response maps arranged by scale number; for each sampling position in the sampling position set, read the response value at the sampling position in the primitive response maps of all scales, select the scale with the largest response value as the scale label of the sampling position, and write "ring radius, orientation angle, and scale label" into the structural template entry list; S36. Calculate the periodic phase deviation and solidify the weight for each item in the structural template item list, specifically as follows: S361. Project the displacement vector of the sampling position of the item relative to the center point of the template onto the longitudinal and latitudinal directions respectively to obtain the longitudinal projection distance and the latitudinal projection distance; S362, Project the meridional distance to The remainder is used to obtain the meridional phase margin, and the zonal projected distance is then used to... The remainder is used to obtain the latitudinal phase margin; S363. Calculate the distances from the meridional phase margin to the nearest period boundary and the zonal phase margin to the nearest period boundary respectively, and add them together to obtain the periodic phase deviation. ; S364, with Calculate the fixed weights of the entries and write them into the weight table. The fixed weights satisfy the following: ; The weight table assigns a unique weight value to each structural template entry; S37. Perform structural template matching on each frame of the spatiotemporally aligned image sequence, specifically: take any pixel as a candidate center point, convert the ring radius and orientation angle of each entry in the structural template entry list into the sampling coordinates of the candidate center point, read the response value at the sampling coordinates on the primitive response map of the corresponding scale label, and accumulate the response value according to the weight table to obtain the structural response value of the candidate center point. S38. Repeatedly perform structure template matching on all pixels within the common overlapping rectangular area of ​​each frame image, arrange all structure response values ​​according to pixel spatial position to obtain the structure selection response map of the frame, and bind the structure selection response map with the frame index and output it.

[0011] Optionally, determining the reference frame for the COSFIRE structural template and delineating the template construction area within the reference frame includes: Within the common overlapping rectangular region, grayscale images of the region are extracted frame by frame from the spatiotemporally aligned image sequence. Two types of quantities are calculated sequentially: the variance of pixel grayscale values ​​within the region is used as a texture stability index, and the average gradient magnitude within the region is used as an edge information index. The texture stability index and edge information index corresponding to each frame are arranged into a candidate list in chronological order. After removing frames whose edge information index is lower than the median of the candidate list, the frame with the smallest texture stability index is selected from the remaining frames as the reference frame for constructing the COSFIRE structure template. Within the common overlapping rectangular region of the reference frame, multiple sub-regions are divided according to a fixed grid. The average gradient magnitude of each sub-region is calculated and sorted. The sub-region with the highest sorting value is selected as the template construction region, and the geometric center of the sub-region is used as the scanning start position for template construction.

[0012] Optionally, S4 includes: S41, Regarding the first image in the spatiotemporally aligned image sequence The structure-selective response map of the frame is segmented by amplitude thresholding within the common overlapping rectangular region to obtain a binary response map. Connectivity labeling is then performed on the binary response map to obtain the first... Set of frame response points; S42, Record-based The cumulative vertical pixel displacement value of the frame will be the first The coordinates of each response point in the frame response point set are mapped to a unified reference coordinate system, and the mapped response state is written into a three-dimensional time consistency mapping matrix. ;in: Represents the horizontal pixel coordinates within the shared overlapping rectangular area; Represents the vertical pixel coordinates within the commonly overlapping rectangular area; Indicates time index; when coordinates In the When a response point exists in the frame, let Otherwise ; S43, within a time window length of Within the sliding time interval, for each pixel coordinate Calculate the maximum number of consecutive occurrences along the time dimension The maximum number of consecutive occurrences is determined as follows: ; in, Represents pixel coordinates In length The maximum number of consecutive responses within a continuous time window; Indicates the starting time index of the sliding time window; This represents the time offset within the window, with a value range of [value missing]. to ; Indicates the preset number of frames for the sliding window; S44. Set the consecutive frame threshold as follows: When a certain pixel coordinate satisfy When this happens, the pixel is marked as a continuously responding pixel; S45. Merge all persistent response pixels according to the 8-neighborhood connectivity rule to obtain a candidate persistent response region set, and calculate the centroid trajectory of each candidate region within the time window; when the centroid displacement of the same region between adjacent frames is less than the preset trajectory tolerance threshold, determine the coordinates of all pixels in the region as a set of structural anomaly points. S46. Output the set of structural anomalies, where the set of structural anomalies is defined as points that repeatedly occur across consecutive frames in a unified reference coordinate system and satisfy a consecutive frame threshold. The set of pixels that respond to the condition.

[0013] Optionally, S5 includes: S51. Read the set of structural anomalies and record each structural anomaly as... And read the response amplitude at the outlier point in the structure-selective response plot. ; Calculate the horizontal and vertical gradients of the structure-selective response map at the anomaly points to determine the local principal direction unit vector at the anomaly points. ,in Obtained by normalizing the gradient direction; S52. Convert each structural outlier into a second-order symmetric tensor seed. The second-order symmetric tensor seed is constructed as follows: The outer product forms a matrix and with response amplitude The matrix is ​​weighted by magnitude to obtain ,in Indicates transpose; S53, Based on the main meridional direction angle Latitudinal direction angle Meridional periodic scale Compared with zonal periodic scale For each structural anomaly point Construct the defined neighborhood using the following process : S531, with and Establish two-dimensional orthogonal basis vectors , ,in Direction angle The corresponding unit vector, Direction angle The corresponding unit vector; S532, select any candidate pixel relatively displacement vector Projected to and , obtain the projected distance and ; S533, Set the meridional neighborhood radius to The latitudinal neighborhood radius is Only retain those that meet the requirements. and Candidate pixels are used as a directionally restricted candidate set; S534. Calculate the phase margin for candidate pixels within the orientation-restricted candidate set. and Calculate the phase distance to the nearest periodic boundary. , Only retain those that meet the requirements. and Candidate pixels are used as the final defined neighborhood. ,in This represents the modulo operation; S54. For each structural anomaly point and its defined neighborhood Any pixel within Calculate displacement distance The propagation direction unit vector is obtained by normalizing the displacement vector. ; S55, for each The voting tensor is generated as follows: : S551. Calculate the unit vector of the propagation direction. orthogonal basis vectors in the meridional direction The included angle ,as well as orthogonal basis vectors in the latitudinal direction The included angle ; S552, Set the direction gating conditions as follows When the direction gating condition is not met, the command It is a zero matrix; S553. When the direction gating condition is met, with Constructing a two-dimensional rotation matrix Tensor seed according to Perform orientation mapping and apply Gaussian decay to the voting intensity with respect to displacement distance to obtain the voting tensor. ; S56. For any pixel within the common overlapping rectangular region All conditions must be met upon accumulation. The structural outlier voting tensor is used to obtain the pixel. Structure tensor The structure tensor is determined as follows: ; in, Represents a set of structural outliers. Indicates structural outliers The defined neighborhood, Represents pixels Relative structural anomalies displacement distance, This represents the voting decay scale parameter. Represents the unit vector of the propagation direction. The constructed two-dimensional rotation matrix, Indicates structural outliers The second-order symmetric tensor seed, Indicates transpose; S57. Calculate the structure tensor for all pixels within the common overlapping rectangular region, and output all of them according to pixel coordinates. Generate the structure tensor field.

[0014] Optionally, S6 includes: S61. Read the structural tensor field and assign coordinates to each pixel within the common overlapping rectangular region. Structure tensor Perform eigenvalue decomposition to obtain the largest eigenvalue. Minimum eigenvalue and the unit eigenvector corresponding to the largest eigenvalue and will The corresponding direction is used as pixel coordinates The main direction of the location; S62, For each pixel coordinate Calculate the ratio of eigenvalues The eigenvalue ratio is defined as and The ratio of eigenvalues ​​is used as the first component of the structural continuity measure. S63. Establish a fixed-radius neighborhood centered on pixel coordinates (p). In the neighborhood Read the principal direction vector point by point inside The principal direction vector and the meridional principal direction angle are intersected. and the principal latitudinal direction angle The included angle is calculated to obtain the warp deviation angle and the weft deviation angle respectively, and the smaller value of the two is taken as the fabric orientation consistency deviation of the point in the neighborhood; S64, in the neighborhood The deviation of the fabric orientation consistency is statistically analyzed, and the proportion of pixels in the neighborhood with a deviation greater than a preset orientation deviation threshold is calculated to obtain the pixel coordinates. directional consistency fracture index Furthermore, the directional consistency fracture index is used as the second component of the structural continuity measure; S65. Set the eigenvalue ratio threshold. Fracture index threshold with direction consistency Within the common overlapping rectangular area, it will simultaneously satisfy and The pixel is marked as the hole seed pixel, and region growth is performed starting from the hole seed pixel. During the region growth process, only adjacent pixels that meet the condition that "the feature value ratio is not less than the lower bound of the feature value ratio of the hole seed pixel and the direction consistency fracture index is not less than the lower bound of the fracture index of the hole seed pixel" are allowed to be incorporated into the growth region to obtain a set of hole candidate regions. S66. Perform connected component labeling and area filtering on the set of candidate hole regions, delete candidate regions with areas smaller than the preset area threshold, and retain the remaining regions to form the fabric hole region labeling result. The fabric hole region labeling result is the set of hole regions after warp-direction joint discrimination and region growth. S67. Perform boundary extraction on each hole area in the fabric hole area marking result, output the defect space coordinates as the set of all pixel coordinates in the hole area, and output the boundary contour data as the ordered pixel coordinate sequence of the outer boundary of the hole area.

[0015] The beneficial effects of this invention are: During the structure selection stage, the COSFIRE filter is oriented and scaled by introducing the main warp and weft directions and periodic scale constraints. The main direction is eliminated and the periodic phase consistency weight is solidified for the support direction set. This ensures that the filter response matches the periodic texture structure of the fabric, effectively suppressing the high response interference generated by regular warp and weft textures, highlighting the response of abnormal structures, improving the sensitivity to small holes and edge fracture structures, and enhancing the adaptive capability under different fabric specifications.

[0016] In the structural aggregation stage, by converting abnormal response points into second-order symmetric tensors and performing tensor voting operations within the neighborhood constrained by both the warp and weft directions and the periodic phase, a gating mechanism consistent with the orthogonal basis of the propagation direction and warp and weft is introduced. This ensures that the voting process accumulates only in directions that conform to the fabric structure rules, which can strengthen continuous structural information, suppress the diffusion of isolated noise points, and improve the ability to characterize the continuity of hole edges and structural fracture features, thereby improving the stability and boundary integrity of defect region extraction. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 The flowchart shows the fabric hole and defect detection method and system based on multi-frame fusion proposed in this invention. Figure 2 This is a schematic diagram illustrating the construction and response generation of the COSFIRE filter structure template proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figure 1 - Figure 2 A method and system for detecting fabric holes and defects based on multi-frame fusion, comprising the following steps: The image acquisition and displacement calibration module acquires continuous image frames during the continuous fabric conveying process, and performs sub-pixel-level spatial registration of adjacent image frames based on the displacement data output by the conveying encoder to construct a spatiotemporally aligned image sequence. The fabric principal direction statistics module calculates the warp and weft principal direction vectors and corresponding periodic scale parameters in the spatiotemporally aligned image sequence, and generates a set of fabric periodic texture constraint parameters. The COSFIRE filtering module is improved by configuring the support direction set, scale parameter set, and response weight of the COSFIRE filter in a structured manner based on the fabric periodic texture constraint parameter set, and calculating the structure-selective response map on each frame image. The cross-frame structural consistency matrix construction module establishes a pixel-level time index matrix in the structural selection response layer, performs cross-frame trajectory association and continuous frame counting on response points, and outputs a set of structural anomalies that meet the continuous frame threshold conditions. The tensor voting computation module converts the set of structural anomalies into a second-order symmetric tensor representation, performs tensor voting operations within a limited neighborhood, and accumulates the structural tensor field. The tensor feature spectrum discrimination module performs eigenvalue decomposition on the structural tensor field, calculates the eigenvalue ratio and the rate of change of the principal direction, generates the marking results of the fabric hole area based on the eigenvalue spectrum change, and outputs the spatial coordinates and boundary contour data of the defect.

[0020] In this embodiment, the modules are interconnected using the following method: S1. Collect continuous image frames of the same area during the continuous fabric conveying process, and perform spatial registration of the continuous image frames based on the displacement data output by the conveying encoder to construct a spatiotemporally aligned image sequence. S2. Calculate the main direction vectors of the warp and weft directions of the fabric and the corresponding periodic scale parameters in the spatiotemporally aligned image sequence to generate a set of fabric periodic texture constraint parameters. S3. Based on the fabric periodic texture constraint parameter set, constrain the support direction set and scale parameter set of the COSFIRE filter, and calculate the structure-selective response map on each frame image. S4. Establish a time consistency mapping matrix in the structure selection response layer, perform cross-frame trajectory association and continuous frame counting on response points, and extract the set of structural anomalies that meet the continuous frame threshold conditions. S5. Convert the set of structural anomalies into a second-order symmetric tensor representation, perform tensor voting operation within a limited neighborhood, and generate a structural tensor field. S6. Perform eigenvalue decomposition on the structural tensor field, calculate the eigenvalue ratio and the rate of change of the principal direction, generate the marking results of the fabric hole area based on the eigenvalue spectrum change, and output the spatial coordinates and boundary contour data of the defect.

[0021] In this embodiment, S1 includes: S11. Fix an area array industrial camera and an incremental rotary encoder on the fabric conveying path, complete the camera intrinsic parameter calibration and pixel physical size calibration, measure the actual length value corresponding to a unit pixel, and record the actual displacement distance corresponding to a single encoder pulse. S12. When the camera triggers image acquisition each time, the current count value of the encoder is read synchronously, the difference between the current count value and the count value when the first frame was acquired is calculated, and the difference is multiplied by the actual displacement distance of a single pulse to obtain the cumulative physical displacement distance of the current frame relative to the first frame. S13. Divide the cumulative physical displacement distance by the actual length value corresponding to the unit pixel to obtain the cumulative vertical pixel displacement of the current frame relative to the first frame, and retain the pixel displacement as a floating point. S14. Using the first frame image as the reference coordinate system, perform a vertical translation transformation on the current frame image according to the cumulative vertical pixel displacement. During the translation process, bilinear interpolation is used to resample the non-integer pixel displacements. S15. Perform boundary calculation on the overlapping area between the image frame after translation transformation and the reference frame, determine the common overlapping rectangular area of ​​all registered image frames, and crop each frame image according to the common overlapping area. S16. Store the cropped image frames into a fixed-length sliding window buffer according to the acquisition time sequence, and record the corresponding cumulative vertical pixel displacement value and timestamp for each frame to construct a spatiotemporally aligned image sequence arranged in the same pixel coordinate system.

[0022] In this embodiment, S2 includes: S21. Establish a fixed-size two-dimensional analysis grid within the common overlapping rectangular region of the spatiotemporally aligned image sequence. Perform mean removal processing on the pixel grayscale values ​​within each grid cell, and calculate the gradient components in the horizontal and vertical directions to construct the structure tensor matrix of each grid cell: ; in, and These are the gradient components of the pixel in the horizontal and vertical directions, respectively; S22. Perform eigenvalue decomposition on the structural tensor matrix, take the eigenvector direction corresponding to the largest eigenvalue as the main structural direction of the grid cell, and perform direction clustering statistics on the main structural directions of each grid cell in all frames, selecting the two orthogonal directions with the highest frequency as the main warp direction angles of the fabric. With the principal latitudinal direction angle ; S23, respectively with and Assuming the projection direction, equally spaced sampling sequences are extracted along the corresponding direction within each grid cell. A one-dimensional fast Fourier transform is then performed on the sampling sequences, and the frequency with the largest amplitude in the amplitude spectrum, excluding zero frequency, is selected. As the dominant frequency component, its corresponding period scale is calculated. ; S24. Within all frames and all grid cells, perform median statistics on the meridional and zonal periodic scales respectively, and remove outliers that deviate from the median by more than a preset proportion to obtain a stable meridional periodic scale. Compared with zonal periodic scale ; S25, with the main direction angle of the meridian. Latitudinal direction angle Meridional periodic scale and zonal periodic scale Construct a set of fabric periodic texture constraint parameters, and use the set of parameters as input parameters for subsequent structural filtering constraint configuration.

[0023] In this embodiment, S3 includes: S31. Read the warp principal direction angle from the fabric periodic texture constraint parameter set. latitudinal main direction angle Meridional periodic scale latitudinal periodic scale Within the common overlapping rectangular area, a reference frame for constructing the COSFIRE structure template is determined, and the template construction area is delineated within the reference frame; S32. Scan candidate center points within the template construction area at fixed pixel intervals, extract a local window of fixed size for each candidate center point, calculate the average gradient magnitude within the local window, select the candidate center point with the largest average gradient magnitude as the template center point, and determine the local window corresponding to the template center point as the reference structure segment. S33. Within the reference structure segment, generate a set of sampling positions using the "concentric rings - fixed angle step size" method. Specifically, generate multiple layers of concentric rings with the template center point as the origin, and the radius of each concentric ring is taken as... and The smaller values ​​are integer multiples and incremented by layer number; a candidate sampling direction angle list is generated on each concentric ring with a fixed angular step size, and each sampling position is recorded as a polar coordinate binary tuple of "ring radius, direction angle"; S34. Perform deterministic elimination on the candidate sampling direction angle list. The elimination rule is: when a candidate direction angle is... When the included angle is less than a preset direction exclusion angle threshold, the candidate direction angle is discarded; when the included angle is less than a preset direction exclusion angle threshold, the candidate direction angle is discarded. When the included angle is less than the preset direction exclusion angle threshold, the candidate direction angle is eliminated, and the remaining direction angles are retained to form a support direction set. The support direction set is then applied to each layer of concentric rings to obtain a set of sampling positions consisting only of the retained direction angles. S35. Construct local primitive responses within the reference structural segment, specifically: perform multi-scale differential Gaussian filtering on the reference structural segment to obtain a set of primitive response maps arranged by scale number; for each sampling position in the sampling position set, read the response value at the sampling position in the primitive response maps of all scales, select the scale with the largest response value as the scale label of the sampling position, and write "ring radius, orientation angle, and scale label" into the structural template entry list; S36. Calculate the periodic phase deviation and solidify the weight for each item in the structural template item list, specifically as follows: S361. Project the displacement vector of the sampling position of the item relative to the center point of the template onto the longitudinal and latitudinal directions respectively to obtain the longitudinal projection distance and the latitudinal projection distance. S362, Project the meridional distance to The remainder is used to obtain the meridional phase margin, and the zonal projected distance is then used to... The remainder is used to obtain the latitudinal phase margin; S363. Calculate the distances from the meridional phase margin to the nearest period boundary and the zonal phase margin to the nearest period boundary respectively, and add them together to obtain the periodic phase deviation. ; S364, with Calculate the fixed weights of the entries and write them into the weight table. The fixed weights satisfy the following: ; The weight table assigns a unique weight value to each structural template entry; S37. Perform structural template matching on each frame of the spatiotemporally aligned image sequence, specifically: take any pixel as a candidate center point, convert the ring radius and orientation angle of each entry in the structural template entry list into the sampling coordinates of the candidate center point, read the response value at the sampling coordinates on the primitive response map of the corresponding scale label, and accumulate the response value according to the weight table to obtain the structural response value of the candidate center point. S38. Repeat step S37 for all pixels within the common overlapping rectangular area of ​​each frame image, arrange all structural response values ​​according to pixel spatial position to obtain the structural selection response map of the frame, and bind the structural selection response map with the frame index for output.

[0024] In this embodiment, determining the reference frame for constructing the COSFIRE structure template within a common overlapping rectangular region and delineating the template construction region within the reference frame includes: within the common overlapping rectangular region, firstly, extracting the grayscale image of the region frame by frame from the spatiotemporally aligned image sequence, and sequentially calculating two types of quantities: one is the variance of pixel grayscale within the region as a texture stability index, and the other is the average gradient magnitude within the region as an edge information index; forming a candidate list of the "texture stability index" and "edge information index" corresponding to each frame in chronological order, removing frames whose edge information index is lower than the median of the candidate list, and selecting the frame with the smallest texture stability index from the remaining frames as the reference frame for constructing the COSFIRE structure template; dividing the common overlapping rectangular region of the reference frame into multiple sub-regions according to a fixed grid, calculating and sorting the average gradient magnitude of each sub-region, selecting the sub-region with the highest sorting value as the template construction region, and using the geometric center of the sub-region as the scanning start position for template construction.

[0025] In this embodiment, S4 includes: S41, Regarding the first image in the spatiotemporally aligned image sequence The structure-selective response map of the frame is segmented by amplitude thresholding within the common overlapping rectangular region to obtain a binary response map. Connectivity labeling is then performed on the binary response map to obtain the first... The set of frame response points; where the first The frame represents the first in a spatiotemporally aligned image sequence. One time index frame; S42, Record-based The cumulative vertical pixel displacement value of the frame will be the first The coordinates of each response point in the frame response point set are mapped to a unified reference coordinate system, and the mapped response state is written into a three-dimensional time consistency mapping matrix. ;in Represents the horizontal pixel coordinates within the shared overlapping rectangular area; Represents the vertical pixel coordinates within the commonly overlapping rectangular area; Indicates time index; when coordinates In the When a response point exists in the frame, let Otherwise ; S43, within a time window length of Within the sliding time interval, for each pixel coordinate Calculate the maximum number of consecutive occurrences along the time dimension The maximum number of consecutive occurrences is determined as follows: ; in, Represents pixel coordinates In length of The maximum number of consecutive responses within a continuous time window; Indicates the starting time index of the sliding time window; This represents the time offset within the window, with a value range of [value missing]. to ; Indicates the preset number of frames for the sliding window; S44. Set the consecutive frame threshold as follows: When a certain pixel coordinate satisfy When this happens, the pixel is marked as a continuously responding pixel; S45. Merge all persistent response pixels according to the 8-neighborhood connectivity rule to obtain a candidate persistent response region set, and calculate the centroid trajectory of each candidate region within the time window; when the centroid displacement of the same region between adjacent frames is less than the preset trajectory tolerance threshold, determine the coordinates of all pixels in the region as a set of structural anomaly points. S46. Output the set of structural anomalies, where the set of structural anomalies is defined as points that repeatedly occur across consecutive frames in a unified reference coordinate system and satisfy a consecutive frame threshold. The set of pixels that respond to the condition.

[0026] In this embodiment, S5 includes: S51. Read the set of structural anomalies and record each structural anomaly as... And read the response amplitude at the outlier point in the structure-selective response plot. ; Calculate the horizontal and vertical gradients of the structure-selective response map at the anomaly points to determine the local principal direction unit vector at the anomaly points. ,in Obtained by normalizing the gradient direction; S52. Convert each structural outlier into a second-order symmetric tensor seed. The second-order symmetric tensor seed is constructed as follows: outer product matrix and with response amplitude The matrix is ​​weighted by magnitude to obtain ,in Indicates transpose; S53. Based on the meridional principal direction angle obtained in step S2 latitudinal main direction angle Meridional periodic scale latitudinal periodic scale For each structural anomaly point Construct the defined neighborhood using the following process : S531, with and Establish two-dimensional orthogonal basis vectors , ,in Direction angle The corresponding unit vector, Direction angle The corresponding unit vector; S532, select any candidate pixel relatively displacement vector Projected to and , obtain the projected distance and ; S533, Set the meridional neighborhood radius to The latitudinal neighborhood radius is Only retain those that meet the requirements. and Candidate pixels are used as a directionally restricted candidate set; S534. Calculate the phase margin for candidate pixels within the orientation-restricted candidate set. and Calculate the phase distance to the nearest periodic boundary. , Only retain those that meet the requirements. and Candidate pixels are used as the final defined neighborhood. ,in This represents the modulo operation; S54. For each structural anomaly point and its defined neighborhood Any pixel within Calculate displacement distance The propagation direction unit vector is obtained by normalizing the displacement vector. ; S55, for each The voting tensor is generated as follows: : S551. Calculate the unit vector of the propagation direction. orthogonal basis vectors in the meridional direction The included angle ,as well as orthogonal basis vectors in the latitudinal direction The included angle ; S552, Set the direction gating conditions as follows When the direction gating condition is not met, the command It is a zero matrix; S553. When the direction gating condition is met, with Constructing a two-dimensional rotation matrix Tensor seed according to Perform orientation mapping and apply Gaussian decay to the voting intensity with respect to displacement distance to obtain the voting tensor. ; S56. For any pixel within the common overlapping rectangular region All conditions must be met upon accumulation. The structural outlier voting tensor is used to obtain the pixel. Structure tensor The structure tensor is determined as follows: ; in, Represents a set of structural outliers. Indicates structural outliers The defined neighborhood, Represents pixels Relative structural anomalies displacement distance, This represents the voting decay scale parameter. Represents the unit vector of the propagation direction. The constructed two-dimensional rotation matrix, Indicates structural outliers The second-order symmetric tensor seed, Indicates transpose; S57. Repeatedly calculate the structure tensor for all pixels within the common overlapping rectangular region, and output all of them according to pixel coordinates. Generate and output the structure tensor field.

[0027] In this embodiment, S6 includes: S61. Read the structure tensor field and calculate the coordinates of each pixel within the common overlapping rectangular region. Structure tensor Perform eigenvalue decomposition to obtain the largest eigenvalue. Minimum eigenvalue and the unit eigenvector corresponding to the largest eigenvalue and will The corresponding direction is used as pixel coordinates The main direction of the location; S62, For each pixel coordinate Calculate the ratio of eigenvalues The eigenvalue ratio is defined as and The ratio of eigenvalues ​​is used as the first component of the structural continuity measure. S63, using pixel coordinates Establish a fixed radius neighborhood around the center In the neighborhood Read the principal direction vector point by point inside The principal direction vector and the meridional principal direction angle are intersected. and the principal latitudinal direction angle The included angle is calculated to obtain the warp deviation angle and the weft deviation angle respectively, and the smaller value of the two is taken as the fabric orientation consistency deviation of the point in the neighborhood; S64, in the neighborhood The deviation of the fabric orientation consistency is statistically analyzed, and the proportion of pixels in the neighborhood with a deviation greater than a preset orientation deviation threshold is calculated to obtain the pixel coordinates. directional consistency fracture index Furthermore, the directional consistency fracture index is used as the second component of the structural continuity measure; S65. Set the eigenvalue ratio threshold. Fracture index threshold with direction consistency Within the common overlapping rectangular area, it will simultaneously satisfy and The pixel is marked as the hole seed pixel, and region growth is performed starting from the hole seed pixel. During the region growth process, only adjacent pixels that meet the condition that "the feature value ratio is not less than the lower bound of the feature value ratio of the hole seed pixel and the direction consistency fracture index is not less than the lower bound of the fracture index of the hole seed pixel" are allowed to be incorporated into the growth region to obtain a set of hole candidate regions. S66. Perform connected component labeling and area filtering on the set of candidate hole regions, delete candidate regions with areas smaller than the preset area threshold, and retain the remaining regions to form the fabric hole region labeling result. The fabric hole region labeling result is the set of hole regions after warp-direction joint discrimination and region growth. S67. Perform boundary extraction on each hole area in the fabric hole area marking result, output the defect space coordinates as the set of all pixel coordinates in the hole area, and output the boundary contour data as the ordered pixel coordinate sequence of the outer boundary of the hole area.

[0028] Example: A large textile enterprise has long faced problems with high false alarm rates, high missed detection rates, and inaccurate defect boundary positioning in its continuous production of high-density woven fabrics. The fabrics produced by this enterprise are mainly high-warp-density plain and twill weaves, with regular and distinctly periodic warp and weft yarn arrangements, exhibiting a stable periodic texture on the surface under high-speed conveying conditions. During loom operation, due to yarn breakage, tension fluctuations, or mechanical friction, localized hole-like defects can form in the fabric. If these defects are not identified in time, they will be amplified in subsequent dyeing and finishing processes, leading to the downgrading or even scrapping of the entire roll.

[0029] The company's original detection system used single-frame image acquisition and a fixed threshold segmentation algorithm. In each frame, it extracted suspected abnormal areas through grayscale contrast and gradient enhancement, and then filtered and output alarm results using area thresholds. In actual operation, it was found that regular warp and weft textures exhibit strong gradient responses under fluctuating lighting or slight fabric vibrations, making it difficult for the system to distinguish normal texture structures from actual hole edges, resulting in frequent false alarms. Furthermore, because precise displacement correction was not performed between adjacent image frames, the slight displacement of the fabric during high-speed movement caused the defect location to shift across different frames. Single-frame judgment lacked temporal consistency constraints, and random noise points were easily misidentified as defects. For small holes with irregular or torn boundaries, traditional methods only relied on grayscale differences for region expansion, often resulting in broken or incomplete boundary extraction, failing to provide accurate defect contour data.

[0030] To address the aforementioned issues, the fabric hole and defect detection method and system based on multi-frame fusion described in this invention are deployed on the production line. The system features a fixed area array industrial camera mounted above the fabric conveyor path, and establishes a synchronous triggering mechanism with an incremental encoder linked to the loom spindle. The camera acquires images of the fabric surface at a fixed frame rate, synchronously reading the encoder's current count value each time acquisition is triggered. The system calculates the cumulative physical displacement of the current frame relative to the first frame based on the physical displacement distance corresponding to a single encoder pulse, and converts this into a longitudinal pixel displacement based on the pixel physical size calibration results. For non-integer pixel displacements, bilinear interpolation is used to resample the images, and longitudinal translation transformation is performed on consecutive image frames to achieve sub-pixel-level spatial registration of images acquired at different times within a unified pixel coordinate system. After cropping common overlapping areas, the images are stored in a sliding window buffer in chronological order to construct a spatiotemporally aligned image sequence.

[0031] Based on the spatiotemporally aligned image sequence, the system establishes a fixed-size two-dimensional analysis grid within the common area. Mean removal is performed on the pixel grayscale values ​​within each grid cell, and the horizontal and vertical gradient components are calculated to construct a structural tensor matrix. By performing eigenvalue decomposition on the structural tensor matrix, the eigenvector direction corresponding to the largest eigenvalue is extracted as the local principal structural direction. Directional clustering statistics are performed across all frames to determine the warp and weft principal direction angles of the fabric. The system extracts equally spaced sampling sequences along the warp and weft directions, performs frequency domain transformation to obtain the principal frequency components, calculates the corresponding periodic scale, and removes outliers through median statistics to form a stable set of fabric periodic texture constraint parameters.

[0032] In the structural response extraction stage, the system performs structured configuration of the COSFIRE filter based on periodic texture constraint parameters. During template construction, frames with high texture stability and moderate edge information are selected as reference frames. Template construction regions are delineated within their common areas, and the candidate center point with the largest gradient magnitude is used as the template center. A set of sampling positions is generated according to concentric rings and a fixed angular step size. Candidate sampling directions are eliminated, removing directions with an angle less than a set threshold with the main latitude and longitude directions, retaining only the set of support directions that deviate from the main latitude and longitude directions. In the multi-scale differential filtering response map, a scale label is determined for each sampling position, and the periodic phase deviation is calculated based on the projection distance of the sampling position in the latitude and longitude directions. Weight values ​​are solidified according to the phase deviation to form a list of structural template entries. This structural template slides and matches on each frame of the spatiotemporally aligned image sequence, accumulating response values ​​according to weights to generate a structure-selective response map.

[0033] To enhance detection stability, the system establishes a three-dimensional temporal consistency mapping matrix on the structure-selective response layer, performing cross-frame trajectory association and continuous frame counting for response points. Within a set sliding time window, a pixel is marked as a continuous response pixel only when the number of consecutive responses at a certain pixel location reaches a threshold condition. Subsequently, connected component merging is performed on the continuous response pixels, and the centroid trajectory of the region is analyzed. When the centroid displacement of a region between adjacent frames is less than a preset tolerance threshold, it is identified as a set of structural anomalies.

[0034] The set of structural anomalies is converted into a second-order symmetric tensor seed. The system reads the response amplitude at each anomaly and calculates the local principal direction unit vector. A tensor matrix is ​​constructed through outer product and weighted by the response amplitude. A constrained neighborhood is built based on the principal latitude and longitude directions and the periodic scale. Only pixels satisfying the constraints of directional projection distance and periodic phase are retained for voting. During the voting process, the angle between the propagation direction and the orthogonal basis is calculated, and directional gating conditions are set. Voting tensors satisfying the conditions are rotated, Gaussian decayed according to distance, and accumulated to the target pixel, forming a structural tensor field.

[0035] The system performs pixel-by-pixel eigenvalue decomposition on the structural tensor field, calculates the ratio of the maximum to the minimum eigenvalue as a structural continuity index, and statistically analyzes the proportion of deviation of the principal direction from the principal latitude and longitude directions within a fixed-radius neighborhood to obtain the directional consistency fracture index. Only when both the eigenvalue ratio and the fracture index exceed a threshold are the pixel marked as a seed defect, and a complete fracture region is generated through restricted region growth. The system ultimately outputs a set of defect spatial coordinates and a boundary contour sequence, and feeds the detection results back to the production control system.

[0036] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fabric hole defect detection system based on multi-frame fusion, characterized in that, include: The image acquisition and displacement calibration module acquires continuous image frames during the continuous fabric conveying process, and performs sub-pixel-level spatial registration of adjacent image frames based on the displacement data output by the conveying encoder to construct a spatiotemporally aligned image sequence. The fabric principal direction statistics module calculates the warp and weft principal direction vectors and corresponding periodic scale parameters in the spatiotemporally aligned image sequence, and generates a set of fabric periodic texture constraint parameters. The COSFIRE filtering module is improved by configuring the support direction set, scale parameter set, and response weight of the COSFIRE filter in a structured manner based on the fabric periodic texture constraint parameter set, and calculating the structure-selective response map on each frame image. The matrix construction module establishes a pixel-level time index matrix in the structure selection response layer, performs cross-frame trajectory association and continuous frame counting on response points, and outputs a set of structural anomaly points that meet the continuous frame threshold condition. The tensor voting computation module converts the set of structural anomalies into a second-order symmetric tensor representation, performs tensor voting operations within a limited neighborhood, and accumulates the structural tensor field. The tensor feature spectrum discrimination module performs eigenvalue decomposition on the structural tensor field, calculates the eigenvalue ratio and the rate of change of the principal direction, generates the marking results of the fabric hole area based on the eigenvalue spectrum change, and outputs the spatial coordinates and boundary contour data of the defect.

2. A method for detecting fabric holes and defects based on multi-frame fusion, characterized in that, The modules are connected in the following way: S1. Collect continuous image frames of the same area during the continuous fabric conveying process, and perform spatial registration of the continuous image frames based on the displacement data output by the conveying encoder to construct a spatiotemporally aligned image sequence. S2. Calculate the main direction vectors of the warp and weft directions of the fabric and the corresponding periodic scale parameters in the spatiotemporally aligned image sequence to generate a set of fabric periodic texture constraint parameters. S3. Based on the fabric periodic texture constraint parameter set, constrain the support direction set and scale parameter set of the COSFIRE filter, and calculate the structure-selective response map on each frame image. S4. Establish a time consistency mapping matrix in the structure selection response layer, perform cross-frame trajectory association and continuous frame counting on response points, and extract the set of structural anomalies that meet the continuous frame threshold conditions. S5. Convert the set of structural anomalies into a second-order symmetric tensor representation, perform tensor voting operation within a limited neighborhood, and generate a structural tensor field. S6. Perform eigenvalue decomposition on the structural tensor field, calculate the eigenvalue ratio and the rate of change of the principal direction, generate the marking results of the fabric hole area based on the eigenvalue spectrum change, and output the spatial coordinates and boundary contour data of the defect.

3. The fabric hole defect detection method based on multi-frame fusion according to claim 2, characterized in that, S2 includes: S21. Establish a fixed-size two-dimensional analysis grid within the common overlapping rectangular region of the spatiotemporally aligned image sequence. Perform mean removal processing on the pixel grayscale values ​​within each grid cell, and calculate the gradient components in the horizontal and vertical directions to construct the structure tensor matrix of each grid cell: ； in, and These are the gradient components of the pixel in the horizontal and vertical directions, respectively; S22. Perform eigenvalue decomposition on the structural tensor matrix, take the eigenvector direction corresponding to the largest eigenvalue as the main structural direction of the grid cell, and perform direction clustering statistics on the main structural directions of each grid cell in all frames, selecting the two orthogonal directions with the highest frequency as the main warp direction angles of the fabric. With the principal latitudinal direction angle ; S23, respectively with and Assuming the projection direction, equally spaced sampling sequences are extracted along the corresponding direction within each grid cell. A one-dimensional fast Fourier transform is then performed on the sampling sequences, and the frequency with the largest amplitude in the amplitude spectrum, excluding zero frequency, is selected. As the dominant frequency component, its corresponding period scale is calculated. ; S24. Within all frames and all grid cells, perform median statistics on the meridional and zonal periodic scales respectively, and remove outliers that deviate from the median by more than a preset proportion to obtain a stable meridional periodic scale. Compared with zonal periodic scale ; S25, with the main direction angle of the meridian. Latitudinal direction angle Meridional periodic scale and zonal periodic scale Construct a set of parameters to constrain the periodic texture of the fabric.

4. The fabric hole defect detection method based on multi-frame fusion according to claim 2, characterized in that, S3 includes: S31. Read the warp principal direction angle from the fabric periodic texture constraint parameter set. Latitudinal direction angle Meridional periodic scale latitudinal periodic scale Within the common overlapping rectangular area, the reference frame of the COSFIRE structure template is determined, and the template construction area is delineated within the reference frame; S32. Scan candidate center points within the template construction area at fixed pixel intervals, extract a local window of fixed size for each candidate center point, calculate the average gradient magnitude within the local window, select the candidate center point with the largest average gradient magnitude as the template center point, and determine the local window corresponding to the template center point as the reference structure segment. S33. Generate a set of sampling positions within the reference structure segment using a "concentric ring - fixed angle step" method. Specifically, generate multiple layers of concentric rings with the template center point as the origin, and the radius of each concentric ring is taken as... and The smaller values ​​are integer multiples and incremented by layer number; a candidate sampling direction angle list is generated on each concentric ring with a fixed angular step size, and each sampling position is recorded as a polar coordinate binary of "ring radius, direction angle"; S34. Perform deterministic elimination on the candidate sampling direction angle list. The elimination rule is: when a candidate direction angle is... When the included angle is less than a preset direction exclusion angle threshold, the candidate direction angle is discarded; when the included angle is less than a preset direction exclusion angle threshold, the candidate direction angle is discarded. When the included angle is less than the preset direction exclusion angle threshold, the candidate direction angle is eliminated, and the remaining direction angles are retained to form a support direction set. The support direction set is then applied to each layer of concentric rings to obtain a set of sampling positions consisting only of the retained direction angles. S35. Construct local primitive responses within the reference structural segment, specifically: perform multi-scale differential Gaussian filtering on the reference structural segment to obtain a set of primitive response maps arranged by scale number; for each sampling position in the sampling position set, read the response value at the sampling position in the primitive response maps of all scales, select the scale with the largest response value as the scale label of the sampling position, and write "ring radius, orientation angle, scale label" into the structural template entry list; S36. Calculate the periodic phase deviation and solidify the weight for each item in the structural template item list, specifically as follows: S361. Project the displacement vector of the sampling position of the item relative to the center point of the template onto the longitudinal and latitudinal directions respectively to obtain the longitudinal projection distance and the latitudinal projection distance. S362, Project the meridional distance to The remainder is used to obtain the meridional phase margin, and the zonal projected distance is then used to... The remainder is used to obtain the latitudinal phase margin; S363. Calculate the distances from the meridional phase margin to the nearest period boundary and the zonal phase margin to the nearest period boundary respectively, and add them together to obtain the periodic phase deviation. ; S364, with Calculate the fixed weights of the entries and write them into the weight table. The fixed weights satisfy the following: ； The weight table assigns a unique weight value to each structural template entry; S37. Perform structural template matching on each frame of the spatiotemporally aligned image sequence, specifically: take any pixel as a candidate center point, convert the ring radius and orientation angle of each entry in the structural template entry list into the sampling coordinates of the candidate center point, read the response value at the sampling coordinates on the primitive response map of the corresponding scale label, and accumulate the response value according to the weight table to obtain the structural response value of the candidate center point. S38. Repeatedly perform structure template matching on all pixels within the common overlapping rectangular area of ​​each frame image, arrange all structure response values ​​according to pixel spatial position to obtain the structure selection response map of the frame, and bind the structure selection response map with the frame index and output it.

5. The fabric hole defect detection method based on multi-frame fusion according to claim 4, characterized in that, The process of determining the reference frame for the COSFIRE structure template and delineating the template construction area within the reference frame includes: Within the common overlapping rectangular region, grayscale images of the region are extracted frame by frame from the spatiotemporally aligned image sequence. Two types of quantities are calculated sequentially: the variance of pixel grayscale values ​​within the region is used as a texture stability index, and the average gradient magnitude within the region is used as an edge information index. The texture stability index and edge information index corresponding to each frame are arranged into a candidate list in chronological order. After removing frames whose edge information index is lower than the median of the candidate list, the frame with the smallest texture stability index is selected from the remaining frames as the reference frame for constructing the COSFIRE structure template. Within the common overlapping rectangular region of the reference frame, multiple sub-regions are divided according to a fixed grid. The average gradient magnitude of each sub-region is calculated and sorted. The sub-region with the highest sorted value is selected as the template construction region, and the geometric center of the sub-region is used as the scanning start position for template construction.

6. The fabric hole defect detection method based on multi-frame fusion according to claim 2, characterized in that, S4 includes: S41, Regarding the first image in the spatiotemporally aligned image sequence The structure-selective response map of the frame is segmented by amplitude thresholding within the common overlapping rectangular region to obtain a binary response map. Connectivity labeling is then performed on the binary response map to obtain the first... Set of frame response points; S42, Record-based The cumulative vertical pixel displacement value of the frame will be the first The coordinates of each response point in the frame response point set are mapped to a unified reference coordinate system, and the mapped response state is written into a three-dimensional time consistency mapping matrix. ;in Represents the horizontal pixel coordinates within the shared overlapping rectangular area; Represents the vertical pixel coordinates within the commonly overlapping rectangular area; Indicates time index; when coordinates In the When a response point exists in the frame, let Otherwise ; S43, within a time window length of Within the sliding time interval, for each pixel coordinate Calculate the maximum number of consecutive occurrences along the time dimension The maximum number of consecutive occurrences is determined as follows: ； in, Represents pixel coordinates In length The maximum number of consecutive responses within a continuous time window; Indicates the starting time index of the sliding time window; This represents the time offset within the window, with a value range of [value missing]. to ; Indicates the preset number of frames for the sliding window; S44. Set the consecutive frame threshold as follows: When a certain pixel coordinate satisfy When this happens, the pixel is marked as a continuously responding pixel; S45. Merge all persistent response pixels according to the 8-neighborhood connectivity rule to obtain a candidate persistent response region set, and calculate the centroid trajectory of each candidate region within the time window; when the centroid displacement of the same region between adjacent frames is less than the preset trajectory tolerance threshold, determine the coordinates of all pixels in the region as a set of structural anomaly points. S46. Output the set of structural anomalies, where the set of structural anomalies is defined as points that repeatedly occur across consecutive frames in a unified reference coordinate system and satisfy a consecutive frame threshold. The set of pixels that respond to the condition.

7. The fabric hole defect detection method based on multi-frame fusion according to claim 2, characterized in that, S5 includes: S51. Read the set of structural anomalies and record each structural anomaly as... And read the response amplitude at the outlier point in the structure-selective response plot. ; Calculate the horizontal and vertical gradients of the structure-selective response map at the anomaly points to determine the local principal direction unit vector at the anomaly points. ,in Obtained by normalizing the gradient direction; S52. Convert each structural outlier into a second-order symmetric tensor seed. The second-order symmetric tensor seed is constructed as follows: The outer product forms a matrix and with response amplitude The matrix is ​​weighted by magnitude to obtain ,in Indicates transpose; S53, Based on the main meridional direction angle Latitudinal direction angle Meridional periodic scale Compared with zonal periodic scale For each structural anomaly point Construct the defined neighborhood using the following process : S531, with and Establish two-dimensional orthogonal basis vectors , ,in Direction angle The corresponding unit vector, Direction angle The corresponding unit vector; S532, select any candidate pixel relatively displacement vector Projected to and , obtain the projected distance and ; S533, Set the meridional neighborhood radius to The latitudinal neighborhood radius is Only retain those that meet the requirements. and Candidate pixels are used as a directionally restricted candidate set; S534. Calculate the phase margin for candidate pixels within the orientation-restricted candidate set. and Calculate the phase distance to the nearest periodic boundary. , Only retain those that meet the requirements. and Candidate pixels are used as the final defined neighborhood. ,in This represents the modulo operation; S54. For each structural anomaly point and its defined neighborhood Any pixel within Calculate displacement distance The propagation direction unit vector is obtained by normalizing the displacement vector. ; S55, for each The voting tensor is generated as follows: : S551. Calculate the unit vector of the propagation direction. orthogonal basis vectors in the meridional direction The included angle ,as well as orthogonal basis vectors in the latitudinal direction The included angle ; S552, Set the direction gating conditions as follows When the direction gating condition is not met, the command It is a zero matrix; S553. When the direction gating condition is met, with Constructing a two-dimensional rotation matrix Tensor seed according to Perform orientation mapping and apply Gaussian decay to the voting intensity with respect to displacement distance to obtain the voting tensor. ; S56. For any pixel within the common overlapping rectangular region All conditions must be met upon accumulation. The structural outlier voting tensor is used to obtain the pixel. Structure tensor The structure tensor is determined as follows: ； in, Represents a set of structural outliers. Indicates structural outliers The defined neighborhood, Represents pixels Relative structural anomalies displacement distance, This represents the voting decay scale parameter. Represents the unit vector of the propagation direction. The constructed two-dimensional rotation matrix, Indicates structural outliers The second-order symmetric tensor seed, Indicates transpose; S57. Calculate the structure tensor for all pixels within the common overlapping rectangular region, and output all of them according to pixel coordinates. Generate the structure tensor field.

8. The fabric hole defect detection method based on multi-frame fusion according to claim 2, characterized in that, S6 includes: S61. Read the structural tensor field and assign coordinates to each pixel within the common overlapping rectangular region. Structure tensor Perform eigenvalue decomposition to obtain the largest eigenvalue. Minimum eigenvalue and the unit eigenvector corresponding to the largest eigenvalue and will The corresponding direction is used as pixel coordinates The main direction of the location; S62, For each pixel coordinate Calculate the ratio of eigenvalues The eigenvalue ratio is defined as and The ratio of eigenvalues ​​is used as the first component of the structural continuity measure. S63. Establish a fixed-radius neighborhood centered on pixel coordinates (p). In the neighborhood Read the principal direction vector point by point inside The principal direction vector and the meridional principal direction angle are intersected. and the principal latitudinal direction angle The included angle is calculated to obtain the warp deviation angle and the weft deviation angle respectively, and the smaller value of the two is taken as the fabric orientation consistency deviation of the point in the neighborhood; S64, in the neighborhood The deviation of the fabric orientation consistency is statistically analyzed, and the proportion of pixels in the neighborhood with a deviation greater than a preset orientation deviation threshold is calculated to obtain the pixel coordinates. directional consistency fracture index Furthermore, the directional consistency fracture index is used as the second component of the structural continuity measure; S65. Set the eigenvalue ratio threshold. Fracture index threshold with direction consistency Within the common overlapping rectangular area, it will simultaneously satisfy and The pixel is marked as the hole seed pixel, and region growth is performed starting from the hole seed pixel. During the region growth process, only adjacent pixels that meet the condition that "the feature value ratio is not less than the lower bound of the feature value ratio of the hole seed pixel and the direction consistency fracture index is not less than the lower bound of the fracture index of the hole seed pixel" are allowed to be incorporated into the growth region to obtain a set of hole candidate regions. S66. Perform connected component labeling and area filtering on the set of candidate hole regions, delete candidate regions with areas smaller than the preset area threshold, and retain the remaining regions to form the fabric hole region labeling result. The fabric hole region labeling result is the set of hole regions after warp-direction joint discrimination and region growth. S67. Perform boundary extraction on each hole area in the fabric hole area marking result, output the defect space coordinates as the set of all pixel coordinates in the hole area, and output the boundary contour data as the ordered pixel coordinate sequence of the outer boundary of the hole area.

Images