An intelligent manufacturing-based textile fabric production online monitoring system
By constructing an energy-structure coupled multidimensional feature matrix and a long short-term memory recurrent neural network, the problem of early identification of minute defects in the online monitoring system of textile fabrics was solved, achieving high-precision real-time early warning and dynamic tracking, and improving the level of intelligence in textile production.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HAINING CHINA TEXTILE TECH CO LTD
- Filing Date
- 2026-02-27
- Publication Date
- 2026-06-26
Smart Images
Figure CN121740880B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent textile inspection technology, specifically to an online monitoring system for textile fabric production based on intelligent manufacturing. Background Technology
[0002] With the widespread application of intelligent manufacturing and flexible production in the textile industry, online production monitoring of fabrics has become a crucial link in ensuring product quality and improving production efficiency. Currently, mainstream online monitoring technologies mostly rely on image recognition, tension monitoring, or temperature and humidity sensors. These methods are effective in detecting a wide range of fabric defects (such as broken threads, weft skew, and pilling), but the following technical bottlenecks and challenges still exist in actual production:
[0003] First, during high-speed weaving, fabrics have high structural density and complex textures, making it difficult for conventional vision systems to accurately capture and label micron-level defects (such as yarn micro-breaks, latent weft shrinkage, and color difference fluctuations) in real time. Second, due to factors such as equipment status, environmental vibration, and yarn tension, the acquired signals exhibit high nonlinearity and temporal inconsistency, resulting in sparse distribution of monitoring data features and making it difficult to construct an effective temporal continuity model. In addition, current systems generally lack multi-source sensing and deep fusion capabilities, making it difficult to achieve linkage identification from energy disturbances to structural instability, resulting in the inability to provide timely warnings of potential quality hazards.
[0004] Therefore, there is an urgent need for an online monitoring method that can integrate multi-dimensional energy information, microscopic defect evolution characteristics and multi-scale tension response to achieve early perception, path tracking and intelligent early warning of extremely small defects in textile fabric production, and improve the timeliness and proactivity of fabric quality monitoring under intelligent manufacturing conditions. Summary of the Invention
[0005] The purpose of this invention is to provide an online monitoring system for textile fabric production based on intelligent manufacturing, in order to address the shortcomings of the prior art.
[0006] To achieve the above objectives, the present invention provides the following technical solution: an online monitoring system for textile fabric production based on intelligent manufacturing, comprising:
[0007] Data acquisition module: acquires microstructure image sequences, tension perturbation signals and infrared thermal imaging data of textile fabrics during the production process, and generates energy-structure coupled multidimensional feature matrix of fabric unit region through cross-scale convolution feature fusion calculation;
[0008] Feature modeling module: Based on the energy-structure coupled multidimensional feature matrix, construct a fabric pixel-level quality coordinate system and map it onto the fabric structure map to form a defect candidate region map;
[0009] Path extraction module: Based on the defect candidate region map, extract the gradient direction vector of the fabric microstructure change in adjacent time segments to generate a set of perturbation evolution paths;
[0010] Tension response modeling module: Input the set of micro-perturbation evolution paths into the dynamic tension response model, fit the tension perturbation energy surface, and output the spatial probability distribution of potential structural instability points;
[0011] Anomaly detection module: Based on the spatial probability distribution of the potential structural instability points, and combined with the current state parameters of the loom, a temporal anomaly detection network is constructed to calculate the real-time anomaly score and dynamically compare it with the quality and safety threshold.
[0012] Defect Enhancement and Recognition Module: If the real-time anomaly score exceeds the quality and safety threshold, a high-frequency sampling command is triggered to perform image enhancement resampling on the specified area, extract micro-defect detail features, and perform classification.
[0013] Defect visualization output module: Finally, a defect visualization heat map is generated based on the defect type, evolution trend and spatial location, so as to realize real-time early warning and dynamic tracking of minor defects in the online production process of textile fabrics.
[0014] Preferably, the step of generating the energy-structure coupled multidimensional feature matrix of a unit region of the fabric through cross-scale convolutional feature fusion includes the following steps:
[0015] The collected microstructure image sequences of fabric surfaces, tension disturbance signals, and infrared thermal imaging data were standardized and preprocessed respectively.
[0016] A feature extraction network containing multi-scale convolutional kernels is constructed to extract primary feature maps of edge texture, tension response, and heat distribution from various types of preprocessed data.
[0017] The primary feature maps are spliced together according to spatial alignment rules;
[0018] The fusion convolution module is used to perform feature compression and fusion operations on the stitched feature map, and outputs the energy-structure coupled multidimensional feature matrix of the fabric unit region.
[0019] Preferably, constructing a fabric pixel-level quality coordinate system based on the energy-structure coupled multidimensional feature matrix and mapping it to the fabric structure map to form a defect candidate region map includes the following steps:
[0020] Based on the response intensity of each pixel in the energy-structure coupled multidimensional feature matrix, a density-based spatial clustering algorithm is used to divide the fabric unit region into multiple quality-sensing sub-regions.
[0021] For each sub-region, a local quality factor is calculated, which is obtained by weighting the mean of energy perturbation, the variance of texture gradient, and thermal response stability.
[0022] A two-dimensional pixel-level quality coordinate system is constructed based on the quality factor values of each sub-region, where the coordinate values represent the quality risk level under spatial distribution.
[0023] The two-dimensional pixel-level quality coordinate system is matched with the preset fabric structure map. Based on the structural correspondence, the quality coordinates are mapped to the actual textile structure unit, and the response abnormal area is extracted as a defect candidate area map.
[0024] Preferably, the step of mapping the mass coordinates to the actual textile fabric unit based on the structural correspondence and extracting the response anomaly region as a defect candidate region map includes:
[0025] Based on the warp and weft yarn interlacing period parameters in the fabric structure diagram, the fabric pixel-level quality coordinate system is periodically divided into blocks to form a quality distribution area of the fabric unit that corresponds one-to-one with the actual fabric structure unit.
[0026] For each tissue unit's quality distribution area, calculate the regional quality deviation, which is determined by the difference between the average pixel quality coordinates within the unit and the average quality of adjacent units;
[0027] An adaptive anomaly detection threshold is set based on the quality deviation, and tissue unit regions with deviations exceeding the threshold are selected as quality anomaly units.
[0028] By merging continuously distributed quality anomaly units based on regional connectivity, a spatially continuous response anomaly region is generated and output as a defect candidate region map.
[0029] Preferably, generating the perturbation evolution path set includes the following steps:
[0030] Microstructure image sequences of corresponding defect candidate regions are extracted from continuous time segments, and the pixel change amplitude map of the same spatial location between adjacent time segments is calculated using the differential frame method.
[0031] For each variation amplitude map, the gradient operator is used to calculate the texture change gradient in the horizontal and vertical directions, and then the principal gradient direction vector of each pixel is extracted by combining the principal direction projection method.
[0032] The principal gradient direction vectors are matched between frames in chronological order, and the change path is tracked by angular similarity and spatial connectivity rules to construct a perturbation gradient sequence for each candidate region.
[0033] Path smoothing and morphological connection operations are performed on the perturbation gradient sequence to form a set of perturbation evolution paths with a continuous evolution trend.
[0034] Preferably, the spatial probability distribution of the output potential structural instability points includes the following steps:
[0035] Based on the time-series nodes of each perturbation evolution path, the tension perturbation signal sequence at the corresponding time point is extracted to construct a path-tension correlation sample set;
[0036] After normalizing the path-tension correlation sample set, a bidirectional gated recurrent unit network is used to model the tension response state and output the tension prediction value of each path node.
[0037] The predicted tension values of the path nodes are mapped back to the two-dimensional spatial coordinate system. A tension disturbance energy distribution map is constructed based on the tension amplitude. A surface is fitted using a Gaussian function to form a continuous tension disturbance energy surface.
[0038] By combining local extremum detection with probability density analysis, regions with tension concentration tendencies in the energy surface are identified, their corresponding structural instability probabilities are calculated, and the spatial probability distribution map of structural instability points is output.
[0039] Preferably, the calculation of the real-time anomaly score includes the following steps:
[0040] Extract the loom state parameters corresponding to the spatial location of the structural instability point, including spindle speed, warp tension, current fluctuation and weft insertion frequency, and construct a time-series state vector;
[0041] The temporal state vector and the spatial probability value of the structural instability point are synchronized in time and matched in space, and then fused to form a joint input feature sequence.
[0042] An anomaly perception network is constructed based on a long short-term memory recurrent neural network. The input joint feature sequence is used for time-series learning, and the output is the anomaly score value at the corresponding time point.
[0043] Set quality and safety threshold ranges, and use a dynamic update strategy to calculate the threshold range based on a sliding window of historical production data. When the abnormal score value exceeds the upper limit threshold, a defect warning is triggered.
[0044] Preferably, constructing an anomaly perception network based on a long short-term memory recurrent neural network and outputting anomaly scores includes the following steps:
[0045] The spatial probability distribution map of structural instability points is synchronized with the loom state parameters in time to construct a joint input feature sequence that integrates spatial tension probability and equipment operation characteristics;
[0046] The joint input feature sequence is normalized and feature embedded, and principal component analysis is used to reduce redundant dimensions while retaining the main temporal variation features.
[0047] Construct a recurrent neural network structure with two layers of long short-term memory units, each layer containing 64 neuron nodes;
[0048] The processed feature sequence is input into the long short-term memory recurrent neural network for temporal pattern learning, and the anomaly score value at each time point is calculated through the softmax activation function of the output layer.
[0049] Preferably, the step of performing image enhancement resampling on the designated area, extracting micro-defect detail features, and performing classification includes the following steps:
[0050] Based on the spatial location corresponding to the abnormal score value, the sampling frequency of the regional image is increased from 10 frames per second to 50 frames per second to obtain a high spatiotemporal resolution image sequence;
[0051] Multi-scale image enhancement processing is performed on the acquired image sequences;
[0052] Based on an improved residual attention convolutional neural network, microscale defect features in images are extracted, including the spatial distribution patterns of different types of defects such as broken yarn, loose yarn, weft shrinkage, and knots.
[0053] The extracted defect features are input into a trained convolutional classifier to perform defect type discrimination and output defect type labels and confidence values.
[0054] The technical effects and advantages provided by the present invention in the above technical solution are as follows:
[0055] 1. This invention significantly improves the early identification capability of minute defects in textile fabric production by constructing a temporal anomaly perception network guided by the spatial probability distribution of structural instability and combined with the multi-dimensional operating state of the loom. Compared with traditional monitoring methods that rely on setting fixed thresholds for single physical parameters, this invention integrates multi-source features such as image texture perturbation, tension response, and thermal distribution, and introduces a long short-term memory recurrent neural network to achieve deep temporal learning, effectively capturing potential instability evolution trends and achieving more accurate anomaly scoring calculation and dynamic early warning.
[0056] 2. This invention boasts significant advantages such as high anomaly detection accuracy, sufficient early warning lead time, low false alarm rate, and strong scalability. It can operate stably under actual high-speed weaving conditions and is adaptable to different fabric types and loom types. The system can dynamically adjust the image sampling density based on real-time scoring results, driving the image enhancement and defect recognition modules to accurately locate microscale defects. It also enables spatial visualization and evolution tracking of defects in the form of heat maps, significantly improving the intelligence level and quality control efficiency of the textile production process. Attached Figure Description
[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0058] Figure 1 This is a flowchart of the system modules of the present invention. Detailed Implementation
[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0060] For examples, please refer to Figure 1 As shown in this embodiment, an online monitoring system for textile fabric production based on intelligent manufacturing includes:
[0061] Data acquisition module: acquires microstructure image sequences, tension perturbation signals and infrared thermal imaging data of textile fabrics during the production process, and generates energy-structure coupled multidimensional feature matrix of fabric unit region through cross-scale convolution feature fusion calculation.
[0062] First, three types of raw data are acquired from the production line: a sequence of microstructure images of the fabric surface, tension disturbance signals, and infrared thermal imaging frames. To achieve unified analysis and processing, each of the three types of data undergoes standardized preprocessing.
[0063] For the microstructure image sequence, a bilateral filtering algorithm is used to perform edge-preserving denoising on each frame, preserving yarn details and texture features. Simultaneously, the images are scaled to a fixed resolution of 640×480 pixels to ensure consistent input size. Each frame is timestamped for subsequent timing synchronization.
[0064] For the tension disturbance signal, the sampling frequency was set to 100Hz, and a Butterworth bandpass filter with a cutoff frequency range of 0.1Hz to 20Hz was used to filter out low-frequency background fluctuations and high-frequency mechanical vibrations. The signal was reconstructed into a two-dimensional tension fluctuation spectrum using a sliding window method with a window length of 1 second and a step size of 0.5 seconds.
[0065] Each pixel in the infrared thermal imaging frame represents the temperature value of the current area. Missing pixels are filled using bilinear interpolation, and the original temperature values are normalized to the range [0,1] for subsequent network processing. Each thermal image is acquired at a production pace (10 frames per second) and includes a timestamp synchronized with the image.
[0066] To achieve the fusion processing of the three data sources in the time dimension, the tension signal and infrared data were time-aligned based on the image frame timestamp as the main axis and a linear interpolation method was used to construct a unified time series data set, laying the foundation for subsequent multimodal feature extraction.
[0067] After preprocessing, the three types of data are input into a multi-scale feature extraction network. This network structure is a three-branch parallel design, with each branch responsible for extracting features of one type of data and sharing a unified output format for subsequent fusion.
[0068] The first branch processes fabric image data, employing a convolutional neural network based on an improved VGG structure. It includes convolutional kernels of three scales: 3×3, 5×5, and 7×7, used to capture fine yarn edges, local texture, and overall arrangement patterns, respectively. Each scale convolution is followed by a ReLU activation function, and a max-pooling layer is added after every two convolutional layers to preserve the main spatial structure information.
[0069] The second branch targets the tension fluctuation spectrum. It first extracts frequency domain features through two-dimensional Fourier transform, and then inputs them into a one-dimensional convolutional network with a kernel length of 9 and a stride of 1. It mainly extracts frequency fluctuation and amplitude abnormal change patterns to identify periodic tension disturbances and occasional jitter signals.
[0070] The third branch is used for infrared thermographic analysis. The network input is a normalized temperature image, and an improved lightweight U-Net structure is used to perform contextual enhancement on hotspot concentration areas. An attention-guided mechanism is introduced into the network to highlight micro-thermal anomalies by comparing the gradient responses of regions with abrupt temperature changes with the surrounding stable regions. This helps to identify thermal energy focusing characteristics caused by friction and tension concentration.
[0071] Each branch ultimately outputs a feature map of size 128×128×1, and at this stage, the channels remain independent, preparing structurally consistent inputs for subsequent fusion and splicing operations.
[0072] To fuse the three types of feature maps, their spatial alignment must be ensured. First, all image data are uniformly cropped to a 128×128 pixel area for each fabric unit, corresponding to a single weaving pattern cycle in production. The microstructure image feature map is set as the primary reference coordinate system, and the tension feature map and thermal image feature map are registered using an affine transformation matrix.
[0073] The registration operation extracts feature points from three types of images (such as yarn crossing points, tension change points, and temperature peak points), and uses the minimum mean square error criterion to optimize the affine matrix parameters to ensure that the mapping error of the corresponding region is less than 2 pixels.
[0074] After spatial alignment, the three sets of feature maps are stacked in a channel-based manner to form a three-dimensional tensor with a size of 128×128×3. The first channel is the texture image feature, the second channel is the tension response map, and the third channel is the heat distribution map. To eliminate the influence of differences in the numerical ranges of the three types of features, normalization is performed, stretching all pixel values in each channel to the [0,1] range.
[0075] The spliced three-channel tensor serves as the fusion input, providing a data foundation for subsequent coupled analysis. Each pixel simultaneously represents information in three dimensions: structure, mechanics, and thermal energy.
[0076] The stitched 3D tensor is input into the fusion convolutional network to perform feature compression and cross-fusion, ultimately generating an energy-structure coupled multidimensional feature matrix for each fabric unit region.
[0077] First, channel compression is performed using a 1×1 convolutional kernel, fusing the three input channels into a single feature channel. This operation preserves the weighted response of each pixel across the three channels, with the weights initialized to equal weights. During training, the optimal combination is automatically learned, and the output is a 128×128×1 compressed feature map.
[0078] Then, a 3×3 convolution kernel is used to extract local neighborhood features with a stride of 1 and the padding method set to "same" to maintain the spatial dimensions. This convolution operation aims to identify the correlation between energy changes and structural perturbations within micro-local areas, paying particular attention to the coupling relationship between edge tension concentration and regions corresponding to sudden temperature increases.
[0079] Based on this, a residual connection path is introduced to add the initial fused feature map before compression to the feature map after convolution, thereby achieving joint expression of shallow and deep semantics and preventing network degradation.
[0080] The final output 128×128 two-dimensional matrix is the energy-structure coupled multidimensional feature matrix of a unit region of the fabric. The value of each pixel in this matrix represents the multimodal stress-structure response intensity exhibited by that region during the textile process, and is used as the input basis for the subsequent construction of the pixel-level mass coordinate system.
[0081] Feature modeling module: Based on the energy-structure coupled multidimensional feature matrix, construct a fabric pixel-level quality coordinate system and map it onto the fabric structure map to form a defect candidate region map.
[0082] Based on the generated energy-structure coupled multidimensional feature matrix, the feature response value of each pixel is extracted to form a set of feature points in two-dimensional space. A density-based spatial clustering algorithm is used for sub-region division, with DBSCAN (Density-Based Spatial Clustering of Applications with Noise) being the preferred clustering method.
[0083] During clustering, the neighborhood radius parameter ε is set to 3 pixels, and the minimum number of points (MinPts) is set to 10. For each pixel as the core, the number of points with similar response intensities within its neighborhood is calculated. When the number of points in the neighborhood is not less than MinPts, the pixel is considered a core point, and this process is gradually expanded to form multiple non-overlapping quality-aware sub-regions. This process effectively aggregates regions with similar response features, eliminates edge noise points, and improves the accuracy of local quality analysis.
[0084] For each of the above-mentioned quality-perceived sub-regions, its local quality factor is calculated. The local quality factor reflects the overall quality fluctuation within that region and is calculated by weighting the following three indicators:
[0085] Mean energy perturbation: Calculates the average response value of all pixels in the region, reflecting the overall energy anomaly trend;
[0086] Texture gradient variance: The Sobel operator is used to extract the horizontal and vertical gradients of the image texture channel, and the variance of the gradient values in this region is calculated to characterize the stability of the fabric texture.
[0087] Thermal response stability: The standard deviation of pixel temperature change in this region of the infrared thermal imaging channel is statistically analyzed to reflect the range of temperature fluctuation.
[0088] The local quality factor is expressed by the weighted formula: Quality factor = 0.4 × mean energy perturbation + 0.4 × texture gradient variance + 0.2 × standard deviation of thermal response.
[0089] The local quality factor calculated within each quality-perceived sub-region is mapped to its corresponding pixel location, generating a two-dimensional pixel-level quality coordinate system. The value of each pixel coordinate point in this coordinate system represents the quality risk level of the fabric unit area at that location; a higher value indicates a greater potential risk of quality anomalies.
[0090] The resulting pixel-level quality coordinate system has the same dimensions as the original image (e.g., 128×128), preserving spatial correspondences and providing a positional basis for subsequent structural mapping. This coordinate system can be viewed as a high-precision quality risk distribution map, used to identify potential defect areas within the fabric.
[0091] A pre-defined fabric weave structure map is introduced, which is constructed based on the interlacing pattern of warp and weft yarns and defines the weave cycle and structural pattern of the fabric within a unit area. According to the warp and weft yarn crossing cycle defined in the map (e.g., one interlacing cycle every 8 pixels), a periodic block operation is performed on the pixel-level quality coordinate system.
[0092] Specifically, the entire mass coordinate system is divided into blocks using a sliding window with a step size equal to the fabric's weave period, resulting in multiple weave unit mass distribution regions. The size of each sub-region corresponds one-to-one with the actual weave structure unit of the fabric, ensuring consistent spatial mapping between physical structure and mass information.
[0093] For each organizational unit's quality distribution area, its regional quality deviation is calculated to assess the degree of quality anomaly between that unit and surrounding units. The specific calculation method is as follows: The pixel mean is the average response value of all pixels within the tissue unit in the quality coordinate system. Adjacent units include eight adjacent units in the horizontal, vertical, and diagonal directions. A larger value indicates a more significant difference between this region and its surrounding areas, potentially suggesting a defect.
[0094] To improve the adaptability of anomaly identification, a judgment threshold is adaptively set based on the statistical characteristics of the quality distribution of the entire fabric pattern. The preferred formula is: Anomaly Judgment Threshold = Mean Deviation of All Fabric Units + 1.5 × Standard Deviation of Deviation; when the quality deviation of a fabric unit exceeds the above threshold, it is judged as a quality anomaly unit. This method avoids the insufficient adaptability problem caused by using a fixed threshold and can dynamically adapt to the quality distribution characteristics of different fabric batches.
[0095] For all tissue units identified as having quality anomalies, a region merging operation based on pixel connectivity is performed. Eight-neighborhood connectivity analysis is used to aggregate adjacent anomalous units, identifying spatially continuous anomalous regions. This step effectively filters out isolated anomalous points while extracting clustered anomalous regions with actual defect evolution potential, improving the localization accuracy of subsequent analyses.
[0096] Finally, the aforementioned spatially continuous anomalous regions are represented by a binary map, where anomalous regions are marked as 1 and other regions as 0, generating a defect candidate region map. This map serves as a key input for subsequent perturbation evolution path tracking and real-time early warning, supporting the early perception and source analysis of minute defects.
[0097] Path extraction module: Based on the defect candidate region map, extract the gradient direction vector of the fabric microstructure change in adjacent time segments to generate a set of perturbation evolution paths.
[0098] First, a sequence of microstructure images within a continuous time frame is extracted from the locations marked on the defect candidate region map. The image sampling frequency is set to 10 frames per second, and the time span is generally 2 to 3 seconds. The image sequence maintains spatial alignment to ensure that corresponding regions are in the same coordinate system.
[0099] For each pair of adjacent image frames in the sequence, the frame difference method is used to calculate pixel-level changes. The processing formula is: Change Amplitude Map Where I(t) represents the grayscale image at time t. The resulting variation amplitude map M(t) is an image of the same size as the original image, and its pixel values represent the brightness or texture structure variation amplitude at corresponding positions in adjacent frames. To enhance the response to small perturbations, a low-pass filter threshold is set, and regions with pixel variation values less than 10 are considered noise interference and set to zero, while regions with significant structural responses are retained for subsequent gradient analysis.
[0100] Texture gradient analysis is performed on the change amplitude map M(t) of each frame using a gradient operator. The Sobel gradient operator is preferably used, performing convolutions in the horizontal (x-axis) and vertical (y-axis) directions respectively to obtain two gradient maps Gx(t) and Gy(t). The calculation formula is as follows:
[0101] ;
[0102] ;
[0103] At each pixel, the principal direction angle θ(t) of the texture is calculated using the two directional gradient values mentioned above, and is implemented using the principal direction projection method. The specific calculation method is as follows: The angle θ(t) represents the main direction of the pixel's change trend in the current frame, and its value ranges from -180 degrees to 180 degrees. Using this method, the set of main gradient direction vectors of all pixels within the defect candidate region can be obtained in each frame, serving as a directional feature reflecting the change trend of the perturbation structure.
[0104] To track the evolution trend of microstructure perturbation over time, the principal gradient direction vectors in consecutive frames are matched in chronological order to establish inter-frame change paths.
[0105] Matching algorithms include the following two types of constraints:
[0106] Angle similarity constraint: Set the angle threshold Δθ to 20 degrees. Only when the difference in the main direction angle of corresponding pixels in consecutive frames or within their 3×3 neighborhood is less than Δθ, the point is considered to belong to the same change path.
[0107] Spatial connectivity constraint: The matching points are required to maintain connectivity in spatial location, that is, the offset distance of the matching points between time t and t+1 is allowed to not exceed 2 pixels, ensuring that the path has a continuous physical displacement trend.
[0108] Through the above dual constraints, the principal gradient direction vector of each frame is matched step by step from the first frame to the next, forming a set of time-increasing directional paths, and constructing a perturbation gradient direction sequence in each candidate region.
[0109] To eliminate path oscillations caused by minor inter-frame perturbations, a sliding window averaging method is used to smooth the path for each perturbation gradient direction sequence. The window length is set to 5 frames, and a weighted average is performed on the spatial location and direction vector of each path point to obtain a smooth evolution path with consistent direction.
[0110] Subsequently, a morphological connection algorithm is used to connect parallel paths in space that are collinear and less than 5 pixels apart, forming complete perturbation evolution paths. The connection principle is based on directional consistency (angle difference less than 15 degrees) and endpoint proximity (Euclidean distance less than a threshold) to determine whether they belong to the same evolution path.
[0111] The final output set of perturbation evolution paths is a set of space-time joint curves. Each path represents the perturbation propagation trajectory of the fabric microstructure in a continuous time segment, which is used as the fitting input for the subsequent dynamic tension response model.
[0112] Tension response modeling module: Input the set of micro-perturbation evolution paths into the dynamic tension response model, fit the tension perturbation energy surface, and output the spatial probability distribution of potential structural instability points.
[0113] First, based on the time-series node information of each perturbation evolution path, the tension perturbation signal at the corresponding time point is extracted. This tension perturbation signal originates from the tension sensor data collected in real time in the weaving equipment, with a sampling frequency of 100 Hz.
[0114] The path P is defined as consisting of several consecutive time points T1 to Tn. At each time point Ti, tension fluctuation segments within 0.5 seconds before and after are extracted to form a tension vector Si of length 100. The spatial coordinates (xi, yi), tension perturbation vector Si, and corresponding principal gradient direction θi triplet of each time point Ti are used as a set of path-tension samples.
[0115] By traversing the time nodes of all perturbation evolution paths, a path-tension correlation sample set is constructed. Each sample is associated with spatial location, tension temporal signal and path direction, which is used for subsequent tension dynamic modeling.
[0116] Each tension perturbation vector Si in the path-tension correlation sample set is normalized, and its magnitude is standardized to the range of [0,1] using the min-max scaling method to enhance the stability of the neural network input.
[0117] A set of bidirectional gated recurrent unit (Bi-GRU) neural networks was established to capture the bidirectional dependence of tension perturbation signals in the time dimension. This network contains two gated unit channels, used for modeling forward and reverse time signals respectively, and outputs a contextual state representation of tension evolution.
[0118] The network parameters are set as follows: 64 hidden layer units, 100 time steps, and hyperbolic tangent activation function. During training, mean squared error loss is used, with the objective of predicting the current node's tension value based on historical tension samples.
[0119] After training, each tension perturbation vector is input into the Bi-GRU network to obtain the tension prediction value Yi at the corresponding time node, and it is bound to the original spatial coordinates (xi, yi) as the basic data for tension space mapping.
[0120] The predicted tension values Yi and their spatial coordinates (xi, yi) of all path nodes are mapped to a two-dimensional coordinate plane to construct a tension disturbance energy distribution map. This map is then interpolated and filled based on the fabric image resolution to generate a dense heat map of the spatial distribution of tension amplitude.
[0121] To obtain a continuous and differentiable energy expression, a two-dimensional Gaussian function is used to fit the tension space distribution. The Gaussian fitting function is defined as: Where (μx,μy) are the coordinates of the center point, σx and σy are the spatial expansion scales, and A is the amplitude parameter. By fitting the above function using the least squares method, a continuous tension perturbation energy surface Z(x,y) is obtained, where each point reflects the degree of tension energy accumulation in the corresponding region.
[0122] Local extremum detection is performed on the tension perturbation energy surface Z(x,y) to identify all local peak regions. A sliding window local maximum algorithm is used during the detection process, with a window size of 9×9 pixels, retaining only points whose center value is the maximum value within their neighborhood as candidate extremum points.
[0123] To assess the structural instability probability at each extreme point, a probability density function is used for risk assessment, combining the mean μ and standard deviation σ of the overall tension energy. The instability probability P(x,y) is calculated as follows:
[0124] The closer the P(x,y) value is to 1, the more significantly the tension energy in that region deviates from the overall mean, and the higher the risk of structural instability. The final output is a spatial probability distribution map of the structural instability points, where the pixel values are the instability probability values at each location, used for subsequent anomaly detection and early warning judgment.
[0125] Anomaly detection module: Based on the spatial probability distribution of the potential structural instability points, and combined with the current state parameters of the loom, a temporal anomaly detection network is constructed to calculate the real-time anomaly score and dynamically compare it with the quality and safety threshold.
[0126] Key operating parameters of the loom are collected in real time during the production process, including spindle speed (revolutions per minute), warp tension (Newtons), main motor current fluctuation (amperes), and weft insertion frequency (times per minute), with a recording frequency of 10 times per second. These parameters are then combined into a four-dimensional loom state vector to form a continuous time series.
[0127] Based on the spatial probability distribution map of structural instability points generated in the previous stage, the time nodes associated with the corresponding spatial locations are extracted and synchronized with the aforementioned temporal state vector. The matching principle is to take a 1-second window before and after the time when the abnormal peak of tension probability occurs, forming a total of 20 frames (10 frames per second) of corresponding state subsequence.
[0128] The spatial probability value of each structural instability point is used as an additional feature and fused with the time-synchronized state vector in the time dimension to form a joint input feature sequence, which represents the evolution trajectory of the loom operating parameters under specific spatial anomaly risks.
[0129] Normalization was performed on the above joint input feature sequence. The spindle speed, warp tension, current fluctuation, weft insertion frequency and tension probability value were standardized by Z-score standardization method, with the mean set to 0 and the standard deviation set to 1 to eliminate the influence of different dimensions.
[0130] To avoid feature redundancy and excessive dimensionality, principal component analysis (PCA) is introduced to reduce the dimensionality of the 5-dimensional normalized features. By calculating the covariance matrix and its eigenvectors, the first three principal components are selected to retain the main change information with a cumulative contribution rate exceeding 95%. The dimensionality-reduced feature sequence retains the main abnormal change trends while reducing computational complexity.
[0131] The resulting processed feature sequence is used as the model input. Each time node corresponds to an embedded feature vector of length 3. There are a total of 20 time steps, forming a 20×3 time series input matrix.
[0132] Based on deep temporal modeling capabilities, a Long Short-Term Memory (LSTM) recurrent neural network is constructed to identify anomalous patterns in joint feature sequences.
[0133] The network comprises a two-layer stacked LSTM unit structure, each layer containing 64 neuron nodes. The output of the first LSTM unit is used as the input of the second layer. A Dropout layer is inserted between the two layers with a dropout rate of 0.3 to prevent overfitting.
[0134] The input to the LSTM network is a 20×3 time series matrix, and the output is the hidden state vector at each time point. Finally, it is mapped to the Softmax activation function through a fully connected layer to output 20 anomaly scores, ranging from 0 to 1. The larger the value, the higher the anomaly risk.
[0135] The model training uses the cross-entropy loss function. The training set consists of historical normal and abnormal loom operation data. The Adam optimizer is used for parameter updates, and the learning rate is set to 0.001.
[0136] To achieve highly adaptable anomaly detection, a quality and safety scoring threshold range is set, and a dynamic update strategy is adopted. This strategy is based on historical normal production data, extracting scoring values from the past N frames (preferably N=200), and calculating their mean μ and standard deviation σ.
[0137] The dynamic threshold is defined as: This reflects the normal upper limit fluctuation range of the current production process.
[0138] When the anomaly score at any time point exceeds the dynamic threshold, it is considered a potential weaving anomaly. The system automatically marks the spatial region corresponding to the current time point as a warning region, triggering the image enhancement and defect refinement classification process for further analysis of anomaly types and evolution trends.
[0139] To verify the actual effect of the anomaly sensing network described in this application in the textile fabric production process, a comparative experiment was conducted based on actual operating data collected during the continuous production of a certain type of high-density weft cotton fabric using a certain type of air-jet loom (loom model JAT810). The test platform configuration is as follows:
[0140] Image acquisition equipment: Industrial camera, resolution 1280×960, adjustable sampling frequency;
[0141] Data collection duration: Continuous collection of 8 hours of production process data, including approximately 28,800 time segments (1 frame per second).
[0142] Number of defects marked: A total of 372 abnormal weaving events were marked as real defects by human experts, including yarn breakage, weft shrinkage, loose yarn and knots;
[0143] The experimental setup includes:
[0144] The comparison methods include the following two:
[0145] Method A (Comparison Method): Traditional anomaly detection methods based on single-parameter threshold monitoring rely solely on the single signal of warp tension and set a fixed threshold.
[0146] Method B (the method of this application): an anomaly perception network based on spatial probability distribution of structural instability + LSTM deep temporal learning + multi-parameter fusion;
[0147] The two methods were tested on the same dataset, and their accuracy, recall, false alarm rate, and average warning lead time (in seconds) were recorded respectively.
[0148] Table 1 Comparison of Test Results
[0149]
[0150] The data analysis and results are explained below:
[0151] As shown in Table 1, the method of this application significantly improves the stability and sensitivity of anomaly detection by introducing the fusion of the spatial distribution of structural instability probability and the temporal characteristics of loom operating parameters.
[0152] Using LSTM deep time series models can effectively identify "weak signal anomalies" and "trend-type instability", improving the average warning time to 2.5 seconds, which is much better than traditional methods;
[0153] The 3D feature embedding vector structure after dimensionality reduction retains the main dynamic features, avoids the problem of excessive computational complexity caused by too many parameters, and balances accuracy and real-time performance.
[0154] The dynamic threshold mechanism can adaptively adjust the judgment criteria based on historical production data, significantly reducing the false alarm rate and improving the system's ability to adapt to complex production conditions.
[0155] Experiments show that during 8 hours of continuous production, this method can accurately identify the vast majority of real anomalies, while reducing false triggering caused by tension fluctuations in traditional methods.
[0156] The experimental results above demonstrate that the anomaly detection method proposed in this application has good robustness and adaptability in actual industrial environments. It can achieve early warning while maintaining high recognition accuracy, providing strong support for subsequent defect identification and automated control, and has significant practical value.
[0157] Defect Enhancement and Recognition Module: If the real-time anomaly score exceeds the quality and safety threshold, a high-frequency sampling command is triggered to perform image enhancement resampling on the specified area, extract micro-defect detail features, and perform classification.
[0158] When the anomaly score output by the previous stage anomaly perception network exceeds the dynamic quality safety threshold, the fabric space region corresponding to the score is automatically identified, triggering a high-frequency image acquisition operation. The acquisition device control command increases the sampling frame rate of this region from the original 10 frames per second to 50 frames per second, ensuring that transient image changes of minute defects are captured during rapid weaving.
[0159] The sampled image resolution was set to 1280×960 pixels to ensure effective visibility of individual yarns and their gaps. Image frames were synchronized with timestamps to form a high-temporal-precision image sequence, which forms the basis dataset for subsequent detail enhancement and defect extraction.
[0160] To further enhance the perceptibility of microscale fabric defects in images, multi-scale image enhancement processing was performed on the acquired high-resolution image sequences. The enhancement operation included the following two main stages:
[0161] Laplacian pyramid enhancement: Each frame of the image is decomposed into image subbands of multiple scales. By weighting and amplifying the high-frequency subbands (texture and edge parts) and reconstructing the image, structural enhancement is achieved in areas of abrupt edge changes. The preferred pyramid layer count is 3, with the high-frequency layer enhancement coefficient set to 1.5.
[0162] Local contrast enhancement: A local histogram equalization method is used to perform brightness distribution broadening on each image window (64×64 pixels in size), which enhances the texture visibility in low contrast areas. It is particularly suitable for low texture contrast defects such as loose yarn and knots.
[0163] The image sequence processed in the above way has higher visual sensitivity, which is beneficial for deep neural networks to extract more accurate micro-defect features.
[0164] The enhanced image sequence is input into a pre-constructed improved residual attention convolutional neural network, which is used to extract multi-layer semantic features of microscale defects. The network structure includes the following key components:
[0165] Initial feature extraction layer: Multiple 3×3 convolutional kernels are stacked to extract shallow texture features, and the output range is normalized by batch normalization and activation function (ReLU).
[0166] Residual connection structure: Standard residual units are introduced, each unit consists of two sets of convolutional layers, and gradient flow is kept stable by shorting, which improves the deep expression ability of the network.
[0167] Channel attention module: The SE channel attention mechanism is embedded in each residual unit. It uses global average pooling, two fully connected layers and the Sigmoid function to output channel weights and weight important channel features.
[0168] Spatial Attention Module: A pixel-level attention mechanism is introduced into the higher-order feature layer. Two-dimensional attention maps are generated through max pooling and average pooling to enhance the local response of defective regions.
[0169] After processing by this network, the output feature map has high resolution and high selectivity, and can accurately capture the spatial distribution patterns and morphological characteristics of microstructural defects such as broken yarn, loose yarn, weft shrinkage and knots.
[0170] The extracted deep texture feature maps are then input into a trained convolutional neural network classifier for defect identification. This classifier structure consists of two convolutional layers and two fully connected layers, with the output layer using the Softmax activation function.
[0171] The classifier is pre-defined to have five defect types: broken yarn, loose yarn, weft shrinkage, knots, and normal. The network is trained using the cross-entropy loss function, and the training data consists of a labeled sample set with at least 3000 samples.
[0172] During the model inference phase, a 5-dimensional classification probability vector is output for each input image. The label corresponding to the highest probability is selected as the defect type of the current frame image, and the confidence value of the label is output as the basis for subsequent warning map heat map intensity and maintenance decision.
[0173] Defect visualization output module: Finally, a defect visualization heat map is generated based on the defect type, evolution trend and spatial location, so as to realize real-time early warning and dynamic tracking of minor defects in the online production process of textile fabrics.
[0174] After completing the defect type identification, the system binds the defect classification label, corresponding confidence value, and image acquisition timestamp of each frame image to form a time-series defect record. The record format includes defect category (such as broken yarn, loose yarn, weft shrinkage, knot), spatial location coordinates (in pixels), identification confidence (range from 0 to 1), and time index T.
[0175] This structured data serves as the foundational input for heatmap construction and trend modeling, supporting subsequent defect evolution modeling in both spatial and temporal dimensions.
[0176] Based on the aforementioned defect time-series records, spatial location information of defects of the same type in consecutive image frames is extracted for trajectory tracking. A joint approach of optical flow estimation and location matching is employed.
[0177] Location matching criterion: If the spatial position of a certain type of defect shifts by no more than 5 pixels in two consecutive frames of images and the types are consistent, it is judged to be the evolution and continuation of the same defect.
[0178] Optical flow-assisted tracking: For inter-frame defect points that cannot be directly matched, the local optical flow field is calculated (using the Pyramidal Lucas-Kanade method) to track their motion trend in the image and correct the defect trajectory.
[0179] Finally, the temporal evolution trajectory sequence of each defect object is obtained, including the initial appearance location, movement path and evolution duration, which represents the propagation trend and dynamic change pattern of the defect on the fabric surface.
[0180] The defect location and its evolution trajectory are mapped to the current fabric area spatial coordinate system. This coordinate system is established based on the physical location and sampling resolution of the image acquisition device, ensuring that each pixel corresponds to a specific fabric area in the real physical space.
[0181] To construct a defect visualization heatmap, the defect intensity at each spatial location needs to be weighted and calculated. The heatmap intensity value H(x,y) is calculated as follows: H(x,y)=∑(wi×si×di); where wi is the defect type weight (e.g., broken yarn is set to 1.0, weft shrinkage to 0.8, knot to 0.6, and loose yarn to 0.5), si is the identification confidence level, and di is the number of frames the defect remains at that location (representing the persistence of evolution). All values are weighted and summed to form the final heatmap pixel value.
[0182] The heatmap uses a color gradient coding method, which changes continuously from green (normal) to red (high risk), intuitively reflecting the severity and dynamic trend of the defect area.
[0183] The heatmap is refreshed in real time according to the inter-frame update cycle and overlaid on the current fabric video image for dynamic visualization. The following information layers are retained in each output frame:
[0184] Original or enhanced textile image background;
[0185] A semi-transparent color heatmap overlay layer highlights areas of defect intensity.
[0186] The defect type label and confidence level value are displayed at the center of the defect.
[0187] Defect trajectory segments are used to indicate the direction and path of defect evolution.
[0188] When the intensity value H(x,y) of a pixel in the heatmap exceeds the system's preset acceptable upper limit threshold Th (e.g., Th=0.75), a bright flashing red border will be generated in that area, triggering an early warning signal to remind operators to take intervention measures or automatically adjust production parameters.
[0189] The heatmap output supports real-time saving in image frame sequence or video format, which facilitates subsequent quality traceability and process reproduction.
[0190] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. An online monitoring system for textile fabric production based on intelligent manufacturing, characterized in that: include: Data acquisition module: acquires microstructure image sequences, tension perturbation signals and infrared thermal imaging data of textile fabrics during the production process, and generates energy-structure coupled multidimensional feature matrix of fabric unit region through cross-scale convolution feature fusion calculation; Feature modeling module: Based on the energy-structure coupled multidimensional feature matrix, a fabric pixel-level quality coordinate system is constructed, and it is mapped onto the fabric structure map to form a defect candidate region map, including: Based on the response intensity of each pixel in the energy-structure coupled multidimensional feature matrix, a density-based spatial clustering algorithm is used to divide the fabric unit region into multiple quality-sensing sub-regions. For each sub-region, a local quality factor is calculated, which is obtained by weighting the mean of energy perturbation, the variance of texture gradient, and thermal response stability. A two-dimensional pixel-level quality coordinate system is constructed based on the quality factor values of each sub-region, where the coordinate values represent the quality risk level under spatial distribution. The two-dimensional pixel-level mass coordinate system is matched with the preset fabric structure map. Based on the structural correspondence, the mass coordinates are mapped to the actual textile structure unit, and the response abnormal area is extracted as the defect candidate area map. Path extraction module: Based on the defect candidate region map, extract the gradient direction vector of the fabric microstructure change in adjacent time segments to generate a set of perturbation evolution paths; Tension response modeling module: Input the set of micro-perturbation evolution paths into the dynamic tension response model, fit the tension perturbation energy surface, and output the spatial probability distribution of potential structural instability points; Anomaly detection module: Based on the spatial probability distribution of the potential structural instability points, and combined with the current state parameters of the loom, a temporal anomaly detection network is constructed to calculate the real-time anomaly score and dynamically compare it with the quality and safety threshold, including: Extract the loom state parameters corresponding to the spatial location of the structural instability point, including spindle speed, warp tension, current fluctuation and weft insertion frequency, and construct a time-series state vector; The temporal state vector and the spatial probability value of the structural instability point are synchronized in time and matched in space, and then fused to form a joint input feature sequence. An anomaly perception network is constructed based on a long short-term memory recurrent neural network. The input joint feature sequence is used for time-series learning, and the output is the anomaly score value at the corresponding time point. Set quality and safety threshold ranges, and use a dynamic update strategy to calculate the threshold range based on a sliding window of historical production data. When the abnormal score value exceeds the upper limit threshold, a defect warning is triggered. Defect Enhancement and Recognition Module: If the real-time anomaly score exceeds the quality and safety threshold, a high-frequency sampling command is triggered to perform image enhancement resampling on the specified area, extract micro-defect detail features, and perform classification. Defect visualization output module: Finally, a defect visualization heat map is generated based on the defect type, evolution trend and spatial location, so as to realize real-time early warning and dynamic tracking of minor defects in the online production process of textile fabrics.
2. The online monitoring system for textile fabric production based on intelligent manufacturing according to claim 1, characterized in that: The step of generating an energy-structure coupled multidimensional feature matrix of a unit region of fabric through cross-scale convolutional feature fusion includes the following steps: The collected microstructure image sequences of fabric surfaces, tension disturbance signals, and infrared thermal imaging data were standardized and preprocessed respectively. A feature extraction network containing multi-scale convolutional kernels is constructed to extract primary feature maps of edge texture, tension response, and heat distribution from various types of preprocessed data. The primary feature maps are spliced together according to spatial alignment rules; The fusion convolution module is used to perform feature compression and fusion operations on the stitched feature map, and outputs the energy-structure coupled multidimensional feature matrix of the fabric unit region.
3. The online monitoring system for textile fabric production based on intelligent manufacturing according to claim 1, characterized in that: The process of mapping mass coordinates to actual textile fabric units based on structural correspondence and extracting abnormal response regions as candidate defect regions includes: Based on the warp and weft yarn interlacing period parameters in the fabric structure diagram, the fabric pixel-level quality coordinate system is periodically divided into blocks to form a quality distribution area of the fabric unit that corresponds one-to-one with the actual fabric structure unit. For each tissue unit's quality distribution area, calculate the regional quality deviation, which is determined by the difference between the average pixel quality coordinates within the unit and the average quality of adjacent units; An adaptive anomaly detection threshold is set based on the quality deviation, and tissue unit regions with deviations exceeding the threshold are selected as quality anomaly units. By merging continuously distributed quality anomaly units based on regional connectivity, a spatially continuous response anomaly region is generated and output as a defect candidate region map.
4. The online monitoring system for textile fabric production based on intelligent manufacturing according to claim 1, characterized in that: The generation of the perturbation evolution path set includes the following steps: Microstructure image sequences of corresponding defect candidate regions are extracted from continuous time segments, and the pixel change amplitude map of the same spatial location between adjacent time segments is calculated using the differential frame method. For each variation amplitude map, the gradient operator is used to calculate the texture change gradient in the horizontal and vertical directions, and then the principal gradient direction vector of each pixel is extracted by combining the principal direction projection method. The principal gradient direction vectors are matched between frames in chronological order, and the change path is tracked by angular similarity and spatial connectivity rules to construct a perturbation gradient sequence for each candidate region. Path smoothing and morphological connection operations are performed on the perturbation gradient sequence to form a set of perturbation evolution paths with a continuous evolution trend.
5. The online monitoring system for textile fabric production based on intelligent manufacturing according to claim 1, characterized in that: The spatial probability distribution of the output potential structural instability points includes the following steps: Based on the time-series nodes of each perturbation evolution path, the tension perturbation signal sequence at the corresponding time point is extracted to construct a path-tension correlation sample set; After normalizing the path-tension correlation sample set, a bidirectional gated recurrent unit network is used to model the tension response state and output the tension prediction value of each path node. The predicted tension values of the path nodes are mapped back to the two-dimensional spatial coordinate system. A tension disturbance energy distribution map is constructed based on the tension amplitude. A surface is fitted using a Gaussian function to form a continuous tension disturbance energy surface. By combining local extremum detection with probability density analysis, regions with tension concentration tendencies in the energy surface are identified, their corresponding structural instability probabilities are calculated, and the spatial probability distribution map of structural instability points is output.
6. The online monitoring system for textile fabric production based on intelligent manufacturing according to claim 1, characterized in that: The steps involved in constructing an anomaly detection network based on a long short-term memory recurrent neural network and outputting anomaly scores are as follows: The spatial probability distribution map of structural instability points is synchronized with the loom state parameters in time to construct a joint input feature sequence that integrates spatial tension probability and equipment operation characteristics; The joint input feature sequence is normalized and feature embedded, and principal component analysis is used to reduce redundant dimensions while retaining the main temporal variation features. Construct a recurrent neural network structure with two layers of long short-term memory units, each layer containing 64 neuron nodes; The processed feature sequence is input into the long short-term memory recurrent neural network for temporal pattern learning, and the anomaly score value at each time point is calculated through the softmax activation function of the output layer.
7. The online monitoring system for textile fabric production based on intelligent manufacturing according to claim 1, characterized in that: The process of enhancing and resampling the image of the specified region, extracting micro-defect detail features, and performing classification includes the following steps: Based on the spatial location corresponding to the abnormal score value, the sampling frequency of the regional image is increased from 10 frames per second to 50 frames per second to obtain a high spatiotemporal resolution image sequence; Multi-scale image enhancement processing is performed on the acquired image sequences; Based on an improved residual attention convolutional neural network, microscale defect features in images are extracted, including the spatial distribution patterns of different types of defects such as broken yarn, loose yarn, weft shrinkage, and knots. The extracted defect features are input into a trained convolutional classifier to perform defect type discrimination and output defect type labels and confidence values.