A fabric defect detection method based on spatio-temporal feature fusion enhancement

Abstract

The application discloses a fabric defect detection method based on space-time feature fusion enhancement, comprising the following steps: acquiring at least one fabric moving image sequence data on a cloth inspection machine; inputting the fabric moving image sequence data into a space-time feature fusion defect detection model to obtain a defect detection result of the fabric moving image sequence data output by the space-time feature fusion defect detection model, wherein the defect detection result comprises a defect type and corresponding fabric physical position information of the defect; wherein the space-time feature fusion defect detection model comprises a fabric image space alignment module, a fabric image grid division module, a pre-trained image encoder module, a defect feature enhancement module, a multi-scale multi-frame consistency time convolution filtering module and a classification head. The application solves the technical problem of low accuracy and robustness of fabric defect detection caused by only using static image processing and not fully utilizing dynamic information of a video stream in the related art.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image recognition, and particularly relates to a fabric defect detection method based on video stream data. It is suitable for fabric quality detection and defect positioning in the textile industry, and is particularly suitable for high-precision defect detection in a high-speed moving fabric environment. BACKGROUND

[0002] In modern textile industry, fabric defect detection has always been one of the key issues. The existence of defects will affect the appearance and quality of the fabric, and early detection and treatment of defects is crucial for textile production.

[0003] Fabric defect detection is an important part of quality control in the textile industry. Manual detection method relies on the human eye to observe and judge whether there are defects on the fabric surface. This method is simple to operate, but has the problems of low detection efficiency, low accuracy and high cost, especially when dealing with large-scale high-speed production tasks, it performs particularly poorly. The cloth inspection machine is a machine device used for detecting fabric defects. It is mainly used on the textile production line to check the surface defects of the fabric, such as holes, poor stitching lines, etc., to ensure the quality of the textile products. The cloth inspection machine uses high-speed photography and image processing technology to quickly and accurately identify defects on the fabric, replacing the traditional manual detection method, greatly improving the detection efficiency and accuracy. At the same time, through the cloth inspection machine, for complex fabric structure or delicate texture fabric, it can better detect and judge, effectively avoid missing detection and misjudgment, etc. The cloth inspection machine is one of the important equipment in modern textile production, which is of great significance to ensure the quality of textile products, improve production efficiency and reduce cost. In the prior art, patent 1 with publication number CN103456021B detects defects in single frame images based on morphological analysis. Patent 2 with publication number CN113433137 detects defects on the surface of single frame fabric using YOLO network; patent 3 with publication number CN109934802B extracts defects in single frame images by combining Fourier transform and image morphology. However, the above prior art has the disadvantage of being based on static images, using feature extraction, classification algorithm or deep learning model to realize automatic detection of fabric defects, and not fully utilizing the dynamic information of video stream. Although these methods have high precision in low-speed or static fabric scenes, for high-speed moving fabric, due to the limited information of single frame image, the accuracy and robustness of detection are significantly reduced. In addition, traditional deep learning methods mainly focus on single frame static features, while ignoring the time series information between consecutive frames.

[0004] In view of the above technical problems, no effective solution has been proposed so far. SUMMARY

[0005] The embodiment of the application provides a fabric defect detection method based on spatio-temporal feature fusion enhancement, which can greatly save labor cost, reduce defect rate, improve production efficiency and product quality, and bring economic benefits to textile enterprises.

[0006] The application achieves the purpose by the following technical scheme:

[0007] A fabric defect detection method based on spatio-temporal feature fusion enhancement comprises the following steps:

[0008] Obtain fabric movement image sequence data on at least one cloth inspection machine;

[0009] Input the fabric movement image sequence data into a spatio-temporal feature fusion defect detection model to obtain a defect detection result of the fabric movement image sequence data output by the spatio-temporal feature fusion defect detection model, wherein the defect detection result comprises a defect type and fabric physical position information corresponding to the defect.

[0010] The spatio-temporal feature fusion defect detection model comprises a fabric image space alignment module, a fabric image grid division module, a pre-trained image encoder module, a defect feature enhancement module, a multi-scale multi-frame consistency time convolution filtering module, and a classification head; the fabric image space alignment module performs spatial alignment on the fabric movement image sequence along the fabric movement direction, and each frame of image obtains a region segment corresponding to the same fabric physical position; the fabric image grid division module divides the region segment in each frame of image of the spatially aligned image sequence into a plurality of grid segments in the same way along the direction perpendicular to the fabric movement, takes out the grid segments corresponding to the same fabric physical position in each frame of image, and generates a grid segment image sequence of each fabric physical position at different time points; the pre-trained image encoder module is used for extracting features of the grid segment image sequence and generating a grid segment feature vector sequence; the defect feature enhancement module performs feature enhancement on the grid segment feature vector sequence by introducing a learnable background feature vector in the feature space to obtain a grid segment enhanced feature vector sequence; the multi-scale multi-frame consistency time convolution filtering module adopts a multi-scale convolution strategy to process the grid segment enhanced feature vector sequence, and performs parallel convolution in different time windows, so that defects with different durations are better modeled; and the classification head is processed through feature flattening, a fully connected layer, and a softmax activation function in sequence to obtain a classification result. For the grid segment classified as a defect, fabric physical position information corresponding to the grid segment is obtained according to a meter counter.

[0011] Further, the spatio-temporal feature fusion defect detection model is obtained by the following steps:

[0012] S1: Construct a fabric image sequence defect detection dataset, pre-process the image sequence data and divide it into a training set, a validation set and a test set;

[0013] S2: Input the fabric image sequence data into a fabric image space alignment module to obtain a region segment image sequence, wherein each region segment image corresponds to sampling of the same fabric physical position at different times;

[0014] S3: Input the region segment image sequence into a fabric image grid division module to obtain a plurality of groups of grid segment image sequences, wherein each group of grid segment image sequences corresponds to sampling of the same fabric physical position at different times;

[0015] S4: Input each group of grid segment image sequences into a pre-trained image encoder module to obtain a grid segment feature vector sequence;

[0016] S5: Input each group of grid segment feature vector sequences into a defect feature enhancement module to obtain a grid segment enhanced feature vector sequence;

[0017] S6: Input each group of grid segment enhanced feature vector sequences into a multi-scale multi-frame consistency temporal convolution filtering module to obtain a grid segment filtered feature vector sequence;

[0018] S7: Input the grid segment filtered feature vector sequence into a classification head to obtain a grid segment classification result;

[0019] S8: Calculate the loss according to the model output result and the manual annotation result, perform gradient backpropagation, and update the model parameters;

[0020] S9: Repeat steps S2 to S8, iteratively train until the model converges, and the training phase ends;

[0021] S10: Use the validation set to select the optimal hyperparameters for the model;

[0022] S11: Evaluate the generalization performance of the model using the test set.

[0023] Further, in the step S1, the system acquires an image sequence of the fabric surface in the form of continuous frames through a camera fixedly installed on the cloth inspection machine. represents the fabric image taken by the camera at time t, H, W, and C represent the height, width, and channel number of the image, respectively. The direction from left to right of the collected fabric image is set as the x-axis direction, and the direction from top to bottom is set as the y-axis direction. The top-left corner coordinate of the image is (0, 0), and the bottom-right corner coordinate is (W-1, H-1);

[0024] Specifically, the image collected by the camera is an RGB image, so the channel number C = 3;

[0025] Specifically, the fabric moves along the x-axis direction at a known constant speed v (pixel / frame), and the sampling time interval between two adjacent frames of images is Δt (second);

[0026] Specifically, the fabric image sequence corresponding to any time T0 contains T frames of images collected continuously,

[0027] Only the first frame of image in each fabric image sequence needs to be labeled during manual labeling;

[0028] Further, in the step S2, for the fabric image sequence The rectangular region with the upper left corner coordinates ((k-1)v, 0) and the lower right corner coordinates ((k-1)v+w-1, H-1) in the fabric image obtained at the time T0+kΔt (k ∈ {1, 2, …, T}) forms the kth region segment, and the region segment image sequence composed of T region segments is formed

[0029] Further, in the step S3, for the region segment image sequence composed of T region segments Grid division is performed along the y-axis direction, and a total of M grid segments are divided, and the height of each grid segment is h = H / M. In the fabric image obtained at the time T0+kΔt (k ∈ {1, 2, …, T}), the mth grid segment m ∈ {1, 2, …, M} The upper left corner coordinates are ((k-1)v, (m-1)h), and the lower right corner coordinates are ((k-1)v+w-1, (m-1)h+h-1). The region segment image sequence The mth grid segment of each region segment in the region segment image sequence forms a grid segment image sequence composed of T grid segments

[0030] Further, in the step S4, the pre-trained image encoder module can be represented as:

[0031]

[0032] Where θ represents the parameters of the pre-trained network, and d is the output feature dimension. The pre-trained image encoder module is used to convert each grid segment image into a d-dimensional feature vector:

[0033]

[0034] The feature vectors at T times are collected to obtain a grid segment feature vector sequence:

[0035]

[0036] Further, in the step S5, since the industrial fabric often has a wide range of repeating textures and relatively stable normal state, a learnable background feature vector is introduced in the feature space for modeling the common texture under the "no defect" case. This vector can be updated during the training process to make it as close as possible to the most common normal background distribution in industrial production.

[0037] Specifically, the calculation method of the mesh segment enhanced feature vector is:

[0038]

[0039] wherein, is the feature vector of the enhanced defect information at time T0+kΔt.

[0040] Specifically, if the input mesh segment does not contain defects, then should be close to z template , and at this time each element in it is close to 0; otherwise, if there are defects, the difference between them will increase, and at this time there will be significant non-zero components.

[0041] Specifically, the mesh segment enhanced feature vectors at T time are collected to obtain a mesh segment enhanced feature vector sequence:

[0042]

[0043] Further, in the step S6, the mesh segment enhanced feature vector sequence is filtered in the time dimension to strengthen the defect signal appearing across frames and suppress noise spikes appearing only in a single frame or a very short period of time by aggregating convolution responses of different time scales, thereby obtaining a more discriminative defect feature representation.

[0044] Specifically, for the mesh segment enhanced feature vector , B parallel convolution calculation branches Conv1D (b) with different kernel sizes A (b) (b∈{1,2,...,B}) are set to perform 1D convolution calculation in the time dimension respectively:

[0045]

[0046] wherein, C' represents the number of channels of each convolution kernel.

[0047] Specifically, the calculation results of the parallel convolution calculation branches are aggregated to obtain a network segment filter feature vector:

[0048]

[0049] wherein, α b represents a trainable convolution calculation branch weight, and σ(·) represents an activation function.

[0050] Specifically, the mesh segment filter feature vectors at T time points are collected to obtain a mesh segment filter feature vector sequence:

[0051]

[0052] Further, in step S7, the mesh segment filter feature vector sequence is input into a full connection classifier to obtain a classification result.

[0053] Specifically, the mesh segment filter feature vector sequence is flattened into a one-dimensional vector Supposing there are Q-1 types of defects, plus the defect-free category, a total of Q categories. A full connection layer is used to process , and a softmax activation is used to obtain the probability of classification into each category:

[0054]

[0055] wherein, and represent the weight and bias of the full connection layer, respectively; represents a prediction result vector, represents a mesh segment image sequence classified into category q. The category with the maximum probability is taken as the model classification result.

[0056] Further, in step S8, the model training uses a classification loss and a consistency loss.

[0057] Specifically, for any time T0 corresponding to a fabric image sequence its classification loss can be expressed as:

[0058]

[0059] wherein, is a one-hot vector, if the value of 1 indicates that the category label of the mesh segment image sequence is q, represents the time index of all samples in the training set, denotes the size of the training set.

[0060] Specifically, a consistency loss is introduced to constrain the similarity between the defect-free grid segment image sequence constraining z template with

[0061]

[0062] wherein, denotes a data set formed by the defect-free grid segment image sequence, denotes the size of the training set.

[0063] Specifically, the classification loss and the consistency loss are combined as a total loss by weighting:

[0064] L = L cls + λL cons

[0065] wherein, λ denotes a loss balance coefficient.

[0066] Compared with the prior art, the core innovation of the present application lies in introducing time sequence analysis and multi-frame information fusion of video stream data. By performing feature extraction and hidden state modeling on continuous frames of the same spatial region, the present application additionally fuses the dynamic evolution features in the time dimension under the premise of unchanged spatial resolution, so that the detection model can learn the time sequence law of defect features from the cross-frame information. In addition, the multi-scale multi-frame consistency time convolution filtering module can highlight the defect features that continuously appear in the continuous frames, and smooth the random noise points that may be caused by environmental factors. Compared with the prior art, the present application has stronger robustness and generalization ability for defect detection under high-speed fabric movement. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0068] Figure 1 is a flowchart according to the present application;

[0069] Figure 2 is a structure diagram of the spatio-temporal feature fusion defect detection model designed according to the present application. DETAILED DESCRIPTION

[0070] ​Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is to be understood that the present application can be embodied in various forms without being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. It should be noted that the embodiments of the present application and features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0071] The application discloses a fabric defect detection method based on spatio-temporal feature fusion enhancement, comprising the following contents:

[0072] Obtaining at least one fabric moving image sequence data on a cloth inspection machine;

[0073] Inputting the fabric moving image sequence data into a spatio-temporal feature fusion defect detection model to obtain a defect detection result of the fabric moving image sequence data output by the spatio-temporal feature fusion defect detection model, wherein the defect detection result comprises a defect type and fabric physical position information corresponding to the defect.

[0074] The spatio-temporal feature fusion defect detection model comprises a fabric image space alignment module, a fabric image grid division module, a pre-trained image encoder module, a defect feature enhancement module, a multi-scale multi-frame consistency time convolution filtering module, and a classification head. The fabric image space alignment module performs spatial alignment on fabric moving image sequences along the fabric moving direction, and each frame of image obtains a region segment corresponding to the same fabric physical position. The fabric image grid division module divides the region segment in each frame of image in the spatially aligned image sequence into a plurality of grid segments in the same way along the direction perpendicular to the fabric movement, takes out the grid segments corresponding to the same fabric physical position in each frame of image, and generates a grid segment image sequence of each fabric physical position at different time points. The pre-trained image encoder module is used to extract features of the grid segment image sequence and generate a grid segment feature vector sequence. The defect feature enhancement module performs feature enhancement on the grid segment feature vector sequence by introducing a learnable background feature vector in the feature space to obtain a grid segment enhanced feature vector sequence. The multi-scale multi-frame consistency time convolution filtering module processes the grid segment enhanced feature vector sequence by using a multi-scale convolution strategy, and performs parallel convolution in different time windows, so as to better model defects of different durations. The classification head is processed by feature flattening, a fully connected layer, and a softmax activation function in sequence to obtain a classification result. For the grid segment classified as a defect, the corresponding fabric physical position information is obtained according to a meter counter.

[0075] The overall reasoning process of the spatiotemporal feature fusion defect detection model is as follows: Fabric image sequence data is input into the fabric image spatial alignment module to obtain a sequence of region fragment images; the region fragment image sequence is input into the fabric image grid division module to obtain multiple sets of grid fragment image sequences; each set of grid fragment image sequences is input into the pre-trained image encoder module to obtain a grid fragment feature vector sequence; each set of grid fragment feature vector sequences is input into the defect feature enhancement module to obtain a grid fragment enhanced feature vector sequence; each set of grid fragment enhanced feature vector sequences is input into the multi-scale, multi-frame consistent temporal convolutional filtering module to obtain a grid fragment filtered feature vector sequence; and the grid fragment filtered feature vector sequence is input into the classification head to obtain the grid fragment classification result.

[0076] The training process of the spatiotemporal feature fusion defect detection model is as follows: Figure 1 , Figure 2 As shown, the specific steps are as follows:

[0077] S1: Construct a fabric image sequence defect detection dataset, preprocess the image sequence data and divide it into training set, validation set and test set.

[0078] Specifically, a camera fixedly mounted on the fabric inspection machine first acquires a sequence of images of the fabric surface in the form of consecutive frames. Where Δt represents the sampling time interval between two adjacent image frames. This represents the fabric image captured by the camera at time T0+kΔt, where H, W, and C represent the image height, width, and number of channels, respectively; then, the image sequence... The first frame of the fabric image The process yields a sequence of grid fragment images of the leftmost region. The x-axis is defined as the direction from left to right and the y-axis as the direction from top to bottom in the acquired fabric image. Each grid fragment is located within the image... The coordinates of the top-left and bottom-right corners of the corresponding regions are (0, (m-1)h) and (w-1, (m-1)h+h-1) respectively (m=1,2,…,M). Here, w represents the width of each grid segment, h represents the height of each grid segment, and M=H / h represents the number of grid segments. Then, each grid segment is manually labeled, with the labeling result indicating either a defect category or no defects. Finally, the dataset is divided, and a total of R sets of image sequences are collected. The grid segments obtained from the earliest acquired certain proportion (e.g., 60%) of the image sequence are used as the training set, the grid segments obtained from the middle period acquired certain proportion (e.g., 20%) of the image sequence are used as the validation set, and the grid segments obtained from the latest acquired certain proportion (e.g., 20%) of the image sequence are used as the test set.

[0079] S2: input the fabric image sequence data into the fabric image space alignment module to obtain a region segment image sequence, wherein each region segment image corresponds to the sampling of the same fabric physical position at different time.

[0080] Specifically, for the region segment image sequence composed of T region segments The rectangular region with the upper left corner coordinate ((k-1)v, 0) and the lower right corner coordinate ((k-1)v+w-1, H-1) in the fabric image obtained at T0+kΔt (k∈{1,2,...,T}) forms the kth region segment, and a region segment image sequence composed of T region segments is formed

[0081] S3: input the region segment image sequence into the fabric image grid division module to obtain a plurality of grid segment image sequences, wherein each grid segment image sequence corresponds to the sampling of the same fabric physical position at different time.

[0082] Specifically, for the region segment image sequence composed of T region segments Grid division is performed along the y-axis direction, and a total of M grid segments are divided, and the height of each grid segment is h=H / M. In the fabric image obtained at T0+kΔt (k∈{1,2,...,T}), the mth grid segment m∈{1,2,...,M} The upper left corner coordinate is ((k-1)v, (m-1)h), and the lower right corner coordinate is ((k-1)v+w-1, (m-1)h+h-1). The region segment image sequence Each grid segment of each region segment forms a grid segment image sequence composed of T grid segments

[0083] S4: input each grid segment image sequence into the pre-trained image encoder module to obtain a grid segment feature vector sequence.

[0084] Specifically, the pre-trained image encoder module can be represented as:

[0085]

[0086] Wherein θ represents the parameters of the pre-trained network, and d is the output feature dimension. The pre-trained image encoder module is used to convert each grid segment image into a d-dimensional feature vector:

[0087]

[0088] The feature vectors at T time points are collected to obtain a grid segment feature vector sequence:

[0089]

[0090] S5: Input the feature vector sequence of each group of grid segments into the defect feature enhancement module to obtain the enhanced feature vector sequence of grid segments.

[0091] Specifically, the calculation method for the enhanced feature vector of the mesh segment is as follows:

[0092]

[0093] in, This is the feature vector that enhances defect information at time T0+kΔt.

[0094] If input grid fragment If there are no defects, then Should be with z template Approaching, at this time Each element in the matrix is ​​close to 0; otherwise, if defects exist, the difference between the two increases, at which point... Significant non-zero components will appear in it.

[0095] By combining the enhanced feature vectors of the grid segments at time T, we obtain the sequence of enhanced feature vectors of the grid segments:

[0096]

[0097] S6: Input the enhanced feature vector sequence of each group of grid segments into the multi-scale, multi-frame consistent temporal convolutional filtering module to obtain the grid segment filtered feature vector sequence.

[0098] Specifically, for enhancing feature vectors of grid fragments Set up B parallel convolutional vectors A with different kernel sizes. (b) Conv1D, the branch of convolution computation (b) (b∈{1,2,...,B}), perform 1D convolution calculations in the time dimension respectively:

[0099]

[0100] Where C′ represents the number of channels in each convolution kernel.

[0101] The results of the parallel convolution computation branches are aggregated to obtain the filtered feature vector of the network segment:

[0102]

[0103] Where, α b σ(·) represents the weights of the trainable convolutional computation branches, and σ(·) represents the activation function.

[0104] The mesh segment filter feature vectors of T time instants are collected to obtain a mesh segment filter feature vector sequence:

[0105]

[0106] S7: The mesh segment filter feature vector sequence is input into a classification head to obtain a mesh segment classification result.

[0107] Specifically, the mesh segment filter feature vector sequence is flattened into a one-dimensional vector Suppose there are Q-1 types of defects, plus the defect-free category, for a total of Q categories. A fully connected layer is used to process and a softmax activation is used to obtain the probability of classification into each category:

[0108]

[0109] wherein, and represent the weights and biases of the fully connected layer, respectively; represents a prediction result vector, represents a mesh segment image sequence classified into category q. The category with the maximum probability is taken as the model classification result.

[0110] S8: According to the model output result and the artificial labeling result, the loss is calculated, the gradient is backpropagated, and the model parameters are updated.

[0111] Specifically, the model training uses a classification loss and a consistency loss. For any time instant T0 corresponding to a fabric image sequence its classification loss can be expressed as:

[0112]

[0113] wherein, is a one-hot vector, if the value of 1 indicates that the category label of the mesh segment image sequence is q, represents the time instant index of all samples in the training set, represents the size of the training set.

[0114] The consistency loss can be expressed as:

[0115]

[0116] wherein, represents a data set formed by the defect-free mesh segment image sequence, represents​ the size of the object.

[0117] The classification loss and the consistency loss are combined as a total loss by weighting:

[0118] L = L cls + λL cons

[0119] wherein λ represents a loss balance coefficient.

[0120] S9: repeating steps S2 to S8, iteratively training until the model converges, and ending the training phase.

[0121] S10: selecting the optimal hyperparameters for the model using the validation set.

[0122] Specifically, different hyperparameters are set for the model to obtain different model instances; for each model instance, training is performed on the training set according to steps S2 to S9, and inference is performed on the validation set to calculate the performance indicators of each model instance on the validation set; the performance indicators of the model instances on the validation set are compared, and the model instance with the optimal performance indicator is selected as the final model to be used.

[0123] S11: evaluating the generalization performance of the model using the test set.

[0124] Specifically, for the final model to be used selected by the validation set, model inference is performed on the test set to calculate the performance indicators of the model on the test set.

[0125] The application has been described in detail through the embodiments above, but the content described is only exemplary embodiments of the application and cannot be considered to limit the implementation scope of the application. The protection scope of the application is defined by the claims. Any similar technical solutions that utilize the technical solutions described in the application or are inspired by the technical solutions of the application within the essence and protection scope of the application, and achieve the above technical effects, or equivalent changes and improvements to the application scope, should still belong to the patent coverage and protection scope of the application.

Claims

1. A fabric defect detection method based on spatiotemporal feature fusion enhancement, characterized in that: Includes the following steps: Acquire at least one sequence of fabric movement image data from a fabric inspection machine; The fabric moving image sequence data is input into the spatiotemporal feature fusion defect detection model to obtain the defect detection result of the fabric moving image sequence data output by the spatiotemporal feature fusion defect detection model. The defect detection result includes the defect type and the physical location information of the fabric corresponding to the defect. The spatiotemporal feature fusion defect detection model includes a fabric image spatial alignment module, a fabric image grid division module, a pre-trained image encoder module, a defect feature enhancement module, a multi-scale multi-frame consistent temporal convolutional filtering module, and a classification head. The fabric image spatial alignment module spatially aligns the fabric moving image sequence along the fabric moving direction, and each frame image obtains a region segment corresponding to the same physical location of the fabric. The fabric image grid division module divides the region segment in each frame of the spatially aligned image sequence into several grid segments in the same way and along the direction perpendicular to the fabric movement. It extracts grid segments corresponding to the same physical location of the fabric in each frame to generate a grid segment image sequence for each physical location of the fabric at different times. The pre-trained image encoder module extracts features from the grid segment image sequence to generate a grid segment feature vector sequence. The defect feature enhancement module enhances the grid segment feature vector sequence by introducing a learnable background feature vector into the feature space, resulting in an enhanced grid segment feature vector sequence. The multi-scale, multi-frame consistent temporal convolutional filtering module processes the enhanced grid segment feature vector sequence using a multi-scale convolution strategy, performing parallel convolutions in time windows of different durations to better model defects of different durations. The classification head processes the data sequentially through feature flattening, a fully connected layer, and a softmax activation function to obtain the classification result. For grid segments classified as defects, the corresponding physical location information of the fabric is obtained using a meter counter.

2. The fabric defect detection method based on spatiotemporal feature fusion enhancement according to claim 1, characterized in that: The spatiotemporal feature fusion defect detection model is obtained through the following steps: S1: Construct a fabric image sequence defect detection dataset, preprocess the image sequence data and divide it into training set, validation set and test set; S2: Input the fabric image sequence data into the fabric image spatial alignment module to obtain the region segment image sequence, where each region segment image corresponds to the sampling of the same physical location of the fabric at different times; S3: Input the region segment image sequence into the fabric image grid division module to obtain multiple sets of grid segment image sequences, where each set of grid segment image sequences corresponds to the sampling of the same physical location of the fabric at different times; S4: Input each set of grid segment image sequences into the pre-trained image encoder module to obtain the grid segment feature vector sequence; S5: Input the feature vector sequence of each group of grid segments into the defect feature enhancement module to obtain the enhanced feature vector sequence of grid segments; S6: Input the enhanced feature vector sequence of each grid segment into the multi-scale multi-frame consistent temporal convolutional filtering module to obtain the grid segment filtered feature vector sequence; S7: Input the filtered feature vector sequence of the grid fragments into the classification head to obtain the grid fragment classification result; S8: Calculate the loss, backpropagate the gradient, and update the model parameters based on the model output and manual annotation results; S9: Repeat steps S2 to S8, iteratively training until the model converges, and the training phase ends. S10: Use the validation set to select the optimal hyperparameters for the model; S11: Evaluate the generalization performance of the model using the test set.

3. The fabric defect detection method based on spatiotemporal feature fusion enhancement according to claim 2, characterized in that: In step S1, the system acquires a sequence of images of the fabric surface in the form of consecutive frames using a camera fixedly mounted on the fabric inspection machine; H represents the fabric image captured by the camera at time t, and W, C represent the height, width and number of channels of the image, respectively. The direction from left to right of the acquired fabric image is set as the x-axis, and the direction from top to bottom is set as the y-axis. The coordinates of the top left corner of the image are (0,0) and the coordinates of the bottom right corner are (W-1,H-1). The images captured by the camera are RGB images, so the number of channels C = 3; the fabric moves along the x-axis at a known constant speed v and at a pixel / frame rate, and the sampling time interval between two adjacent frames is Δt seconds; Fabric image sequence at any time T0 It contains T frames of continuously acquired images. When manually annotating, only the first frame of each fabric image sequence needs to be annotated.

4. The fabric defect detection method based on spatiotemporal feature fusion enhancement according to claim 3, characterized in that: In step S2, for the fabric image sequence At time T0+kΔt and k∈{1,2,...,T}, a rectangular region with the top-left corner coordinates ((k-1)v,0) and the bottom-right corner coordinates ((k-1)v+w-1,H-1) is cropped from the fabric image to form the k-th region segment, where w is the width of each grid segment, forming a region segment image sequence consisting of T region segments. In step S3, the image sequence of region segments composed of T region segments is processed. The fabric image is divided into M grid segments along the y-axis, with each segment having a height of h = H / M. The m-th grid segment in the fabric image acquired at time T0 + kΔt, where k ∈ {1, 2, ..., T}, is... Given the top-left corner coordinates as ((k-1)v, (m-1)h) and the bottom-right corner coordinates as ((k-1)v+w-1, (m-1)h+h-1), obtain the image sequence of the region segment. The m-th grid segment of each region segment is used to form a grid segment image sequence consisting of T grid segments. In step S4, the pre-trained image encoder module can be represented as: Where θ represents the parameters of the pre-trained network, and d is the output feature dimension; the pre-trained image encoder module is used to process each grid segment image Convert to a d-dimensional feature vector: By combining the feature vectors from T time points, we obtain the feature vector sequence of the grid segment: In step S5, since industrial fabrics often have a wide range of repeating textures and relatively stable normal states, a learnable background feature vector is introduced into the feature space. Used to model common textures under "defect-free" conditions, this vector can be updated during training to make it as close as possible to the most common normal background distribution in industrial production.

5. The fabric defect detection method based on spatiotemporal feature fusion enhancement according to claim 4, characterized in that: The method for calculating the enhanced feature vector of the mesh fragment is as follows: in, That is, the feature vector that enhances defect information at time T0+kΔt; If input grid fragment If there are no defects, then Should be with z template Approaching, at this time Each element in the model is close to 0; otherwise, if defects exist, the difference between the two increases, at which point... Significant non-zero components will appear in the data; by pooling the enhanced feature vectors of the grid segments at T time points, a sequence of enhanced feature vectors for the grid segments is obtained:

6. The fabric defect detection method based on spatiotemporal feature fusion enhancement according to claim 5, characterized in that: In step S6, the feature vector sequence of the grid fragment is enhanced. Multi-frame consistency filtering is performed in the time dimension to enhance defect signals that occur across frames by aggregating convolutional responses at different time scales, and suppress noise spikes that only occur in a single frame or a very short period of time, thereby obtaining more discriminative defect feature representations. Enhanced feature vectors for grid fragments Set up B parallel convolutional vectors A with different kernel sizes. (b) Conv1D, the branch of convolution computation (b) (b∈{1,2,...,B}), perform 1D convolution calculations in the time dimension respectively: Where C′ represents the number of channels in each convolution kernel; The results of the parallel convolution computation branches are aggregated to obtain the filtered feature vector of the network segment: Where, α b This represents the weights of the trainable convolutional computation branch, and σ(·) represents the activation function; By combining the filtered feature vectors of the grid segments at T time points, we obtain the filtered feature vector sequence of the grid segments:

7. The fabric defect detection method based on spatiotemporal feature fusion enhancement according to claim 6, characterized in that: In step S7, the grid fragment filter feature vector sequence is... Input the data into a fully connected classifier to obtain the classification result; Filter the feature vector sequence of grid fragments Flattened into a one-dimensional vector There are Q-1 defect categories, plus a no-defect category, for a total of Q categories; a fully connected layer is used to... The process is performed, and softmax activation is used to obtain the probabilities of being classified into each category: in, and These represent the weights and biases of the fully connected layer, respectively. Represents the predicted result vector, Represents a grid fragment image sequence The probability of classifying a class as category q is taken as the category with the highest probability. As the model classification result.

8. The fabric defect detection method based on spatiotemporal feature fusion enhancement according to claim 7, characterized in that: In step S8, the model is trained using classification loss and consistency loss; For any time T0, the fabric image sequence Its classification loss can be expressed as: in, It is a one-hot vector, if A value of 1 indicates a grid fragment image sequence. The category label is q. This represents the time index of all samples in the training set. Indicates the size of the training set; Introducing consistency loss to improve the performance of image sequences of flawless mesh segments. Upper constraint Z template and Similarity: in, This represents a dataset formed by a sequence of flawless mesh fragment images. express Dimensions; The classification loss and consistency loss are weighted and combined to obtain the total loss: L=L cls +λL cons Where λ represents the loss balance coefficient.

Images