A fabric multi-modal layered asynchronous calibration fusion estimation method and system

Abstract

The application discloses a kind of fabric multi-modal layered asynchronous calibration fusion estimation method and system, belong to intelligent manufacturing and computer vision technical field, the layered asynchronous calibration fusion framework is proposed in the application: the data collected by line array vision camera, laser profile sensor, flexible pressure sensing array and near infrared spectrometer are carried out distortion correction and noise modeling, execute space-time alignment;Geometric, mechanical, optical multi-physical field joint feature representation is constructed;The weight of each mode is dynamically calculated through attention mechanism, and the fabric state vector and uncertainty are output combined with adaptive extended Kalman filter;Synchronous maintenance fabric defect semantic map, realize defect coordinate consistency across process.The application uses the above method and system, the end-to-end processing delay is less than 8 milliseconds at 200 meters per minute speed, can identify 52 kinds of defect categories, and the state estimation uncertainty is reduced by 67% compared with single-mode scheme.

Description

Technical Field

[0001] This invention relates to the fields of intelligent manufacturing, computer vision, and online textile inspection technology, and in particular to a method and system for multimodal layered asynchronous calibration fusion estimation of fabrics. Background Technology

[0002] Rapid response manufacturing in the garment industry requires production systems capable of configuring and executing processes for multiple varieties and small batches of orders within extremely short cycles. Unlike traditional manufacturing materials such as metals and plastics, fabrics are materials with significant intrinsic uncertainty: at the microscale, they are composed of interwoven yarn networks, exhibiting nonlinear elasticity, plasticity, and path dependence; at the macroscale, their morphology is constantly changing due to the combined effects of gravity, contact constraints, temperature and humidity, and processing history. This "intrinsic uncertainty" renders traditional manufacturing information technologies, based on the assumption of "known and stable workpiece geometry," completely ineffective in fabric processing scenarios.

[0003] Existing technologies for online textile inspection suffer from the following main shortcomings: 1. Limitations of single-modal detection: Current industrial fabric inspection machines mainly rely on single-line array cameras to acquire fabric surface images, which can only perceive two-dimensional surface optical information and cannot obtain key state quantities such as fabric thickness distribution, internal stress state, and fiber composition, resulting in a high rate of missed defects and incomplete state estimation dimensions; 2. Information isolation: Existing inspection systems output defect coordinates and inspection conclusions only in the form of reports for manual review, and cannot form data linkage with downstream layout, cutting, and sewing equipment, causing a large amount of quality information to be lost during process handover, and failing to support collaborative decision-making throughout the entire process; 3. Static calibration system: Existing systems use offline calibration methods, which cannot adaptively adjust when fabric types change or environmental temperature and humidity fluctuate, resulting in a significant decrease in detection accuracy after long-term operation; 4. Insufficient real-time performance: With the improvement of production efficiency, the transmission speed of modern fabric inspection machines has reached 80~120m / min, and in some scenarios it exceeds 200m / min. Existing multi-sensor fusion algorithms have high computational complexity and processing delays generally exceed 30ms, which cannot meet the requirements of high-speed online processing.

[0004] In summary, existing technologies suffer from key deficiencies such as insufficient sensing information dimensions, information gaps across processes, poor dynamic adaptability, and insufficient real-time performance. There is an urgent need for a method that can achieve real-time perception of multi-dimensional fabric states, dynamic modeling, and full-process information integration under high-speed operating conditions. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for multimodal hierarchical asynchronous calibration fusion estimation of fabrics. Through an innovative hierarchical asynchronous calibration (HAC) fusion framework, multi-source heterogeneous sensor data is transformed into a digital twin of fabric state with spatiotemporal consistency and physical interpretability. Quality semantics are integrated into the entire garment manufacturing process in a standardized manner (FDSM), thereby fundamentally solving the problems of "difficulty in sensing, transmission, adaptation, and real-time processing" in flexible material processing, and ultimately realizing the intelligent upgrade of fast-response garment manufacturing.

[0006] To achieve the above objectives, this invention provides a method for asynchronous calibration and fusion estimation of multimodal layered fabrics, comprising the following steps: S100. Initialization: Perform internal and external parameter calibration on the linear array vision camera, laser profile sensor, flexible pressure sensor array and near-infrared spectrometer respectively, and determine the measurement model parameters of each sensor and the installation pose parameters of each sensor relative to the fabric reference coordinate system. S110, Fabric Detection and Tracking Startup: The fabric entry status is identified through the edge detection algorithm. After the fabric enters the detection area, the raw data collected by each sensor is timestamped based on a unified clock source, and a digital identity (DID) is established for the current fabric. S120, Multi-sensor concurrent acquisition: Based on hardware synchronization pulses, each sensor is triggered to acquire data, and a timestamp is added to each frame of acquired data. The sampling frequency of each sensor is set according to the fabric transmission speed and the spatial resolution of the corresponding sensor. S130 and HAC-L1 processing: Distortion correction and noise modeling are performed on the data output from each sensor, and spatiotemporal alignment is performed. Spatiotemporal alignment includes time synchronization and spatial projection operations. The time synchronization operation is as follows: when the time difference between adjacent sensor frames exceeds the synchronization threshold, interpolation compensation or time alignment strategies are used to align asynchronous data to the master clock time. The spatial projection operation is as follows: according to the predetermined installation pose parameters, the data of each sensor is projected onto a unified world coordinate system, and an aligned data packet is output. S140, HAC-L2 processing: Based on the aligned data package output in step S130, extract multimodal features representing the geometric, mechanical, and optical states of the fabric, and construct a multi-physics joint feature representation; S150 and HAC-L3 processing: The fusion weights of each modal feature are determined according to the fabric type and operating conditions. The multi-physics joint feature representation constructed in step S140 is weighted and fused to obtain the observation vector. Adaptive extended Kalman filtering is performed based on the fabric state transition model to obtain the fabric state estimation result at the current moment and the uncertainty of the fabric state estimation result. The fabric state transition model is composed of the fabric dynamics physical model. The fabric dynamics physical model is based on the fabric continuum theory. The fabric state vector is defined as a joint state vector containing deformation field displacement, strain tensor, yarn tension distribution and defect location set. The state equation satisfies the nonlinear continuum mechanics equations. S160, Anomaly Detection and Triggering: Calculate the uncertainty. When the uncertainty exceeds the preset threshold, trigger the sensor recalibration process and re-estimate the current frame. S170, Output: Output the fabric state vector and uncertainty, and update the fabric defect semantic map (FDSM) with the current frame defect detection results. The fabric defect semantic map is used to continuously maintain the defect location mapping relationship through coordinate tracking in subsequent processes.

[0007] Preferably, in step S120, the line frequency of the linear array vision camera is set according to the fabric transmission speed, satisfying the relational expression... , The transmission speed is indicated; the laser profile sensor consists of at least three measurement modules; a flexible pressure sensing array is deployed on the fabric edge guide device, with a sensor element spacing of 3 mm to 10 mm; the wavelength range of the near-infrared spectrometer is 900 nm to 2500 nm.

[0008] Preferably, in step S130, the time synchronization operation uses the acquisition time of the linear array vision camera as the master clock time; the interpolation compensation is linear interpolation or spline interpolation.

[0009] Preferably, in step S140, the multimodal features include geometric feature vectors, mechanical feature vectors, and optical feature vectors; the geometric feature vectors are extracted from the laser contour data after recovering the three-dimensional shape of the fabric, including the average curvature, Gaussian curvature, and normal vector of the grid cells; the mechanical feature vectors are obtained by calculating the transverse stress distribution gradient of the fabric from the pressure sensor array data, including the edge tension non-uniformity index; the optical feature vectors are obtained by extracting the local binary pattern (LBP) texture descriptor from the visual image data and the defect probability distribution map from the near-infrared spectroscopy data.

[0010] Preferably, in step S150, the fabric type is... With transmission speed As contextual information, dynamic weight vectors for linear array visual features, laser contour features, pressure-sensing features, and near-infrared spectral features are calculated using a scaled dot product attention mechanism. The fused observation vector is obtained by weighted aggregation of the joint feature vector composed of linear array visual features, laser profile features, pressure sensing features, and near-infrared spectral features. ; Using a fabric dynamics physical model as a priori, and fusing observation vectors For the observed values, the state is updated using an adaptive extended Kalman filter (AEKF), outputting a fabric state vector. and the covariance matrix of the fabric state vector ; The scaled dot product attention mechanism includes calculating the attention matrix. : ; According to the attention matrix Obtain the dynamic weight vector ,in, Indicates context information The query matrix obtained through linear mapping, This represents the key matrix obtained by linear mapping from the joint eigenvectors. express transpose, The dimension of the key vector; Dynamic weight vector Each component satisfies and , Indicates modal index, The weighted components correspond to linear array visual features, laser contour features, pressure sensing features, and near-infrared spectral features, respectively. Adaptive extended Kalman filtering achieves adaptive parameter adjustment for different fabric types and operating speeds by estimating the process noise covariance and observation noise covariance online.

[0011] Preferably, in step S170, the fabric defect semantic map uses the fabric's own coordinate system as a reference and adopts a quadtree sparse storage structure. Each defect node contains a defect category label, center coordinates, bounding box, area, quality grade, and available bit field markers. Subsequent processes use the digital identity identifier (DID) to query the quality semantics of any coordinate position in real time (quality semantics refers to the structured information related to fabric quality, such as defect category labels, center coordinates, bounding box, area, quality grade, and available bit field markers, stored in the fabric defect semantic map). The fabric defect semantic map continuously updates its coordinate mapping as the fabric's position changes during the laying and cutting processes, using a joint tracking algorithm of optical flow field and tension field.

[0012] Preferably, the method of the present invention adopts a hierarchical processing strategy: steps S130 and S140 are processed first, and step S150 is processed later; when the fabric transmission speed exceeds the preset speed threshold, predictive state extrapolation is enabled.

[0013] The present invention also provides a fabric multimodal layered asynchronous calibration fusion estimation system, comprising: The sensing module includes a linear array vision camera, a laser profile sensor, a flexible pressure sensor array, and a near-infrared spectrometer. The sensing module is mounted on a sensor bracket above the fabric transport line and is connected to the master clock via a hardware synchronization bus. The HAC fusion processing unit includes an independent modeling and spatiotemporal alignment submodule, a multiphysics joint representation submodule, and an attention-weighted Bayesian estimation submodule. The HAC fusion processing unit receives the output of the perception module in sequence and outputs the fabric state vector and the uncertainty of the fabric state vector. The fabric digital twin management module includes a fabric digital identity registration unit, a fabric defect semantic map storage and query unit, and a coordinate tracking unit. The fabric digital twin management module receives the fabric state vector and the uncertainty of the fabric state vector from the HAC fusion processing unit and maintains the digital twin of the entire process. The downstream interface module is used to push the fabric state vector, the uncertainty of the fabric state vector, and the semantic map of fabric defects to the typesetting engine, the fabric laying control system, the cutting equipment, and the sewing production line management platform.

[0014] Preferably, the HAC fusion processing unit adopts a heterogeneous computing architecture: the independent modeling and spatiotemporal alignment submodule runs on a programmable logic device or edge computing unit, the multiphysics joint representation submodule runs on an edge graphics processor or artificial intelligence accelerator, and the attention-weighted Bayesian estimation submodule runs on a central processing unit or server.

[0015] Preferably, the sensing module also includes an adjustable sensor bracket, which is used to adjust the mounting height and lateral position of each sensor relative to the fabric surface. The linear array vision camera includes a linear array vision acquisition unit set on the front or back of the fabric, and is used in conjunction with a synchronous strobe light source for illumination; The fabric digital twin management module also includes a constitutive parameter rapid identification unit. The constitutive parameter rapid identification unit uses a physical augmentation neural network to identify the constitutive parameters of the new fabric varieties, and updates the fabric dynamics physical model parameters of the adaptive extended Kalman filter after the identification is completed. The system also includes a calibration management and data interaction module, which is used to trigger online calibration or parameter updates when there are changes in fabric type, shift switching, or multimodal consistency anomalies. It also outputs the fabric state vector, the uncertainty of the fabric state vector, and the semantic map of fabric defects to the layout engine, the fabric laying control system, the cutting equipment, and the sewing production line management platform.

[0016] Therefore, the above-mentioned fabric multimodal layered asynchronous calibration fusion estimation method and system has the following beneficial effects: (1) Significantly improved multi-dimensional state perception capability. Through the collaboration of four types of sensors, the fabric state perception dimension is expanded from a single optical image to four physical domains: deformation, stress, optics, and composition. The number of detectable defects is expanded from 20-30 to 52, and the state estimation uncertainty is reduced by 67% compared with the single-modal scheme. (2) Breakthrough in high-speed real-time performance. Using a layered heterogeneous computing architecture, the three layers of HAC processing are deployed on FPGA, edge GPU, and CPU respectively. At a speed of 200m / min, the end-to-end processing latency is less than 8ms, which is more than 4 times higher than the existing multi-sensor fusion scheme (30-50ms), meeting the requirements of high-speed online processing. (3) Cross-process information integration. Fabric Defect Semantic Map (FDSM) is bound to fabric digital identity (DID). Through fabric coordinate tracking algorithm, it maintains spatial consistency in subsequent processes such as laying and cutting. The coordinate tracking accuracy is ±0.8mm, realizing the first case of lossless transmission of defect quality information across processes, providing a data foundation for intelligent decision-making in downstream processes. (4) Strong adaptability. AEKF automatically adjusts the state model parameters when the fabric type changes by estimating the noise covariance online; the attention mechanism dynamically adjusts the modal weights according to the running speed and fabric type, so that the system can run stably in a wide range of fabric types with a width of 90~220cm and a thickness of 0.1~6mm without manual parameter adjustment. (5) System integrability. Through the downstream interface module, a standardized API is provided, which can be seamlessly connected with the existing MES / ERP system and various cutting and sewing equipment to achieve intelligent fabric inspection function upgrade with low modification cost.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a diagram of the architecture of a fabric multimodal hierarchical asynchronous calibration fusion estimation system according to the present invention; Figure 2 This is a detailed schematic diagram of the three-layer HAC framework structure of the fabric multimodal layered asynchronous calibration fusion estimation method and system of the present invention; Figure 3 This is a sensor array installation layout diagram of the fabric multimodal layered asynchronous calibration fusion estimation method and system of the present invention; Figure 4 This is a flowchart of the spatiotemporal alignment algorithm of a fabric multimodal layered asynchronous calibration fusion estimation method and system according to the present invention; Figure 5 This is a structural diagram of the attention-weighted Bayesian estimation submodule of the fabric multimodal layered asynchronous calibration fusion estimation method and system of the present invention; Figure 6 This is a flowchart of a fabric multimodal layered asynchronous calibration fusion estimation method according to the present invention; Figure 7 This is a schematic diagram of the device structure of an embodiment of the fabric multimodal layered asynchronous calibration fusion estimation method and system of the present invention; Figure 8 Here are performance comparison charts of an embodiment of the fabric multimodal layered asynchronous calibration fusion estimation method and system of the present invention: (a) Comparison chart of state estimation uncertainty under different sensor combinations; (b) Comparison chart of processing delay of different fusion methods.

[0019] Figure Labels 100. Perception Module; 101. Linear Array Vision Camera; 102. Laser Contour Sensor; 103. Flexible Pressure Sensing Array; 104. Near-Infrared Spectrometer; 200. HAC Fusion Processing Unit; 201. Independent Modeling and Spatiotemporal Alignment Submodule; 202. Multiphysics Joint Representation Submodule; 203. Attention-Weighted Bayes Estimation Submodule; 300. Fabric Digital Twin Management Module; 301. Fabric Digital Identity Registration Unit; 302. Fabric Defect Semantic Map Storage and Query Unit; 303. Coordinate Tracking Unit; 400. Downstream Interface Module. Detailed Implementation

[0020] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0022] Example 1 This embodiment uses a high-speed fabric inspection scenario with a fabric transport speed of 120 meters per minute as an example to illustrate the specific implementation of the fabric multimodal state fusion estimation method and system described in this invention. The system used in this embodiment includes: Sensing module 100: such as Figure 3 As shown, the system includes a linear array vision camera 101, a laser profile sensor 102, a flexible pressure sensor array 103, and a near-infrared spectrometer 104, which are mounted on a sensor bracket above the fabric transport line and connected to the master clock via a hardware synchronization bus.

[0023] HAC fusion processing unit 200 includes independent modeling and spatiotemporal alignment submodule 201, multiphysics joint representation submodule 202 and attention-weighted Bayesian estimation submodule 203, which sequentially receive the output of perception module 100 and output the fabric state vector and the uncertainty of the fabric state vector.

[0024] Fabric digital twin management module 300 includes fabric digital identity registration unit 301, fabric defect semantic map storage and query unit 302 and coordinate tracking unit 303. It receives fabric state vector and uncertainty output by HAC fusion processing unit 200 and maintains the digital twin of the whole process.

[0025] Downstream interface module 400: pushes the fabric state vector, uncertainty and fabric defect semantic map (FDSM) to the layout engine, fabric laying control system, cutting equipment and sewing production line management platform.

[0026] The sampling frequency of the linear array vision camera 101 is preferably set to 12000Hz, the sampling frequency of the laser contour sensor 102 is preferably set to 3000Hz, the sampling frequency of the flexible pressure sensor array 103 is preferably set to 100Hz, and the sampling frequency of the near-infrared spectrometer 104 is preferably set to 50Hz.

[0027] like Figure 7 As shown, the hardware device corresponding to this embodiment includes: a Dalsa Linea series 20K pixel color linear array vision camera 101, a laser profile sensor 102 composed of three measurement modules, a flexible pressure sensor array 103 arranged along both sides of the fabric guide plate, and a near-infrared spectrometer 104 using an InGaAs detector. The edge computing nodes include Xilinx Kintex-7 FPGAs and NVIDIA Jetson AGX Xavier GPUs, and the central server includes an Intel Core i9-13900K processor and 64GB of memory.

[0028] 1. Definition of coordinate system, symbols, and system parameters: In this embodiment, a reference coordinate system established with the center of the feed guide roller is used as the unified world coordinate system. This coordinate system is a fixed coordinate system used to unify the spatial reference of each sensor. The fabric reference coordinate system uses the fabric itself as a reference and moves with the fabric to record defect locations and construct a semantic map of fabric defects. Through the calibration in step S100, the unified world coordinate system and the fabric reference coordinate system are made to coincide at the initial moment, and the mapping relationship between the two is subsequently maintained by the coordinate tracking unit.

[0029] Let the reference coordinate system be denoted as ,in, Fabric transport direction, Along the fabric width direction, Perpendicular to the fabric plane. The local coordinate system of the sensor is denoted as , These correspond to the linear array vision camera 101, the laser profile sensor 102, the flexible pressure sensor array 103, and the near-infrared spectrometer 104, respectively.

[0030] The transformation relationships between each sensor coordinate system and the unified world coordinate system are as follows: ; in, Indicates the first The coordinates of a point or feature measured by a sensor in its local coordinate system. and These represent the rotation matrix and translation vector obtained through step S100, respectively. This represents the coordinates after transformation to the unified world coordinate system. For the linear scan vision camera 101, and The results are given from the visual calibration; for the laser profile sensor 102, and The results are given by splicing calibration and baseline correction; for the flexible pressure sensor array 103 and the near-infrared spectrometer 104, the results are obtained based on their installation positions and the correspondence with the reference target, respectively.

[0031] In this embodiment, the fabric conveying speed is denoted as... The line frequency of the linear array visual camera 101 is recorded as To ensure that the sampling density along the transmission direction meets the minimum defect size detection requirement, the camera's line frequency is set as follows: ; In the formula, The speed is measured in m / s; when the speed output by the field control system is in m / min, it is preferable to convert it to m / s first. In the high-speed fabric inspection scenario of this embodiment, the fabric transmission speed is 120 m / min, corresponding to 2 m / s, and the camera line frequency is preferably 12000 Hz to meet the longitudinal sampling requirement of 0.167 mm / pixel.

[0032] 2. Step S100: Initialization and calibration of internal and external parameters: After the system is powered on, step S100 is executed first. The intrinsic and extrinsic parameters of the linear array vision camera 101, laser profile sensor 102, flexible pressure sensor array 103, and near-infrared spectrometer 104 are calibrated to determine the measurement model parameters of each sensor and the installation pose parameters relative to the fabric reference coordinate system.

[0033] During the calibration process of the linear array visual camera 101, a combination of planar calibration board and multi-pose shooting was used to obtain the camera intrinsic parameter matrix. Distortion parameters and external references The calibration of the laser profile sensor 102 includes fitting the laser plane equation, correcting module splicing errors, and solving for extrinsic parameters between the laser and the linear array vision camera 101. The calibration of the flexible pressure sensing array 103 includes zero-point drift correction, sensitivity calibration, and crosstalk matrix calculation; the calibration of the near-infrared spectrometer 104 includes dark current sampling, white board reference calibration, and wavelength axis linear calibration.

[0034] The crosstalk cancellation formula is: ; In the formula, This represents the original electrical response vector acquired by the flexible pressure sensing array 103. Represents the zero-point bias vector. Represents the crosstalk matrix. This represents the corrected pressure response vector. Crosstalk cancellation can reduce coupling interference between adjacent pressure sensing units.

[0035] 3. Steps S110 and S120: Fabric detection, Digital Identification ID (DID) establishment, and concurrent data acquisition by multiple sensors: like Figure 6 As shown, steps S110 and S120 are executed after calibration. First, the fabric feeding status is identified using an edge detection algorithm. When the leading edge of the fabric enters the detection area, the main control unit generates a digital identity (DID) for the current fabric and establishes an association with the fabric roll, batch, and current process task. Subsequently, each sensor is triggered to acquire data using a 10 kHz hardware synchronization pulse, and a precise timestamp is written into each frame of raw data. .

[0036] The original multi-sensor data packet is represented as follows: ; In the formula, This indicates that the linear visual camera 101 is at time... The collected image row data, This indicates that the laser profile sensor 102 is at time [time missing]. The collected contour data, This indicates that the flexible pressure sensing array 103 is at time 103. The collected pressure data, This indicates that the near-infrared spectrometer 104 is at time 104. The collected spectral data. The DID (Digital ID) is used to identify the specific fabric object to which this data set belongs, thereby ensuring consistent tracking of the same roll of fabric across different processes. Indicates time The original multi-sensor data packet, This indicates the master clock time corresponding to this group of raw data packets.

[0037] 4. Step S130: Independent modeling and spatiotemporal alignment of HAC-L1: like Figure 4 As shown, step S130 first performs distortion correction, noise modeling, and spatiotemporal alignment on the four types of sensor data. For the linear array vision camera 101, the Tsai model is used for distortion correction; for the laser contour sensor 102, baseline drift compensation and multi-module stitching are performed; for the flexible pressure sensor array 103, crosstalk cancellation formula is used for crosstalk cancellation; and for the near-infrared spectrometer 104, background subtraction, whiteboard correction, and spectral smoothing are performed.

[0038] When the time difference between adjacent sensor frames Exceeding the synchronization threshold At that time, using the acquisition time of the linear scan vision camera 101 as the master clock time, the remaining sensor data that were not sampled at the master clock time are resampled. For any signal to be resampled... If its two adjacent sampling times are and ,and Then linear interpolation can be performed as follows, where, Indicates the target time to be aligned during resampling: ; ; In implementations requiring improved interpolation smoothness, cubic spline interpolation can be used instead of the linear interpolation shown in the two formulas above. After time alignment is completed, the sensor data is mapped to the unified world coordinate system according to the transformation relationship between each sensor coordinate system and the unified world coordinate system to obtain the aligned data package. : ; In the formula, This represents a comprehensive spatiotemporal alignment operator that includes temporal resampling, spatial mapping, and data encapsulation.

[0039] 5. Step S140: HAC-L2 multiphysics joint representation: like Figure 2 and Figure 5 As shown, step S140 extracts geometric feature vectors based on the aligned data packets. Mechanical eigenvectors and optical eigenvectors And construct a joint feature representation of multiphysics fields.

[0040] The contour point set output by the laser contour sensor 102 is reconstructed into a 3D point cloud to generate the height field of the fabric surface. Using a 10 mm × 10 mm grid as the basic unit, the average curvature, Gaussian curvature, and normal vector are extracted from each grid to form a geometric feature vector. : ; In the formula, Indicates the number of grid cells. Indicates the mean curvature. Indicates Gaussian curvature, These represent the components of the normal vector along the three coordinate axes.

[0041] For the flexible pressure sensing array 103, based on the corrected pressure vector Calculate the edge tension non-uniformity and transverse stress gradient to construct the mechanical characteristic vector. For the linear array vision camera 101 and the near-infrared spectrometer 104, the local binary mode LBP texture descriptor, defect probability map, and spectral band response features are extracted respectively to form an optical feature vector. .

[0042] The joint characteristics of multiphysics fields are represented as follows: ; In this embodiment, canonical correlation analysis is used to enhance the correlation of each modal feature, or contrastive learning is used to jointly encode the co-location information of different modalities, so as to improve the semantic consistency between geometric, mechanical and optical features.

[0043] 6. Step S150: Attention-weighted fusion and adaptive extended Kalman filter Bayesian state estimation: like Figure 5 As shown, step S150 first determines the fabric type. and transmission speed Constructing context information And map it to a query matrix Simultaneously, the linear array visual features are extracted from the fabric image data acquired by the linear array visual camera 101. The laser contour features are obtained by reconstructing the contour data from the laser contour sensor 102 using 3D point cloud data. Pressure sensing features are extracted from pressure data collected by the flexible pressure sensor array 103 after correction. Near-infrared spectral features were extracted from spectral data acquired by near-infrared spectrometer 104 after background subtraction, whiteboard correction, and spectral smoothing. Mapped to key matrices respectively Sum matrix Calculate the attention matrix. Attention matrix Calculated using the scaled dot product attention mechanism: ; In the formula, The scaling factor is used to calculate the attention matrix using this formula. Further transformed into a dynamic weight vector The sum of each component is 1 and each component is non-negative. Based on the dynamic weight vector, the joint feature vector composed of linear array visual features, laser contour features, pressure sensing features, and near-infrared spectral features is weighted and aggregated to obtain the fused observation vector. : ; In this embodiment, when the fabric is running at high speed, the system can appropriately increase the weights corresponding to linear array visual features and laser contour features while decreasing the weights corresponding to pressure sensing features. For a typical working condition of woven cotton fabric and a speed of 120 meters per minute, the example weights are: The resulting fused observation vector The state is updated using an adaptive extended Kalman filter. (Fabric state vector) Defined as displacement including deformation field strain tensor Yarn tension distribution and defect location set Joint state vector: ; The prediction and update of the adaptive extended Kalman filter are performed as follows: ; ; ; ; ; In the formula, Indicates time The prior fabric state vector, Indicates time Updated fabric state vector, This represents the corresponding prior error covariance matrix. This represents the updated error covariance matrix. State transition function For the first-order Jacobian matrix of the state, For observation function For the first-order Jacobian matrix of the state, The process noise covariance matrix is... To observe the noise covariance matrix, Indicates process noise. This indicates the input of external operating conditions.

[0044] The uncertainty of the current fabric condition estimation result is: ; in, This represents the operation of extracting the elements from the main diagonal of a matrix and constructing a vector.

[0045] 7. Step S160: Anomaly Detection and Triggering: The uncertainty of the obtained fabric state estimation results Compare with a preset threshold; when When the preset threshold is exceeded, the sensor recalibration process is triggered and the current frame is re-estimated.

[0046] 8. Step S170: Fabric defect semantic map update, coordinate tracking and output: Obtain the updated fabric state vector and uncertainty Then, the defect detection results of the current frame are updated to the fabric defect semantic map. The fabric defect semantic map uses the fabric's own coordinate system as a reference and employs a quadtree sparse storage structure. Each defect node... It can be represented as: ; In the formula, Indicates the defect category label, Indicates the coordinates of the defect center. Indicates bounding box, Indicates the area of ​​the defect. Indicates the quality level. This indicates that a bit field can be used for marking, and DID represents the digital identity of the fabric to which the defect node belongs.

[0047] When the fabric enters subsequent processes such as laying and cutting, such as Figure 1 and Figure 6 As shown, the coordinate tracking unit 303 continuously updates the coordinate mapping relationship in the fabric defect semantic map based on the joint tracking algorithm of optical flow field and tension field. For any defect node, its coordinate update at adjacent time points can be expressed as: ; In the formula, This represents the optical flow displacement calculated from the image sequence. This represents the tension field gradient derived from the flexible pressure sensing array 103 and the state estimation results. This represents the tension gradient correction coefficient. This process incorporates positional shifts caused by fabric wrinkles, slack, or changes in lateral tension into the defect mapping update process.

[0048] In this embodiment, after completing the current frame state estimation, the system outputs the fabric state vector through the downstream interface module 400. and uncertainty In addition, it provides semantic map query results for fabric defects. The output can be accessed by the typesetting engine, fabric laying control system, cutting equipment, and sewing production line management platform via REST API or event push.

[0049] 9. Implementation Results: In a typical test, the fabric type was woven cotton with a width of 160 cm and a transmission speed of 120 meters per minute. The system continuously processed the fabric data through steps S100 to S170. The processing delay of HAC-L1 was approximately 0.8 milliseconds, that of HAC-L2 was approximately 3.2 milliseconds, and that of HAC-L3 was approximately 1.8 milliseconds. The overall end-to-end delay was approximately 5.8 milliseconds, as shown in Table 1.

[0050] Table 1

[0051] according to Figure 8 (a) and Figure 8 The comparison results in (b) show that, compared with the single-modal vision scheme, the present invention can significantly reduce the uncertainty of state estimation and maintain a low processing latency under high-speed conditions.

[0052] Example 2 1. Application scenarios and technology adaptation: This embodiment describes a testing scenario (200 cm width) for four-way stretch knitted fabric under varying moisture content conditions. This type of fabric exhibits significant lateral shrinkage and in-plane elastic recovery during transmission, and changes in moisture content affect the surface optical response and the stability of laser profile measurement.

[0053] In this embodiment, in addition to adopting the basic process of steps S100 to S170 as described in Embodiment 1, the moisture content estimation result of the near-infrared spectrometer 104 is also introduced into HAC-L3 as additional contextual information.

[0054] Fabric moisture content estimate The calculation formula is: ; In the formula, This indicates the estimated moisture content of the fabric. and These represent the reflectance or absorption response of the near-infrared spectrum at 1920 nm and 2100 nm, respectively. and These are the coefficients obtained through calibration. (The remaining text appears to be incomplete and requires further context.) When fabric type and running speed are input into the attention module as contextual information, the weight of spectral modes can be appropriately increased when the moisture content increases.

[0055] In the context of elastic knitted fabrics, a lateral elastic contraction model can also be introduced into the state transition function. Let the edge tension be... The lateral shrinkage rate is The mapping between the fabric's natural coordinate system and the detection coordinate system can then be estimated using the following relationship: ; ; In the formula, and These are parameters identified from historical experimental data or PANN. This represents the horizontal coordinate in the detection coordinate system. This represents the lateral coordinates converted to the natural tension-free state. The above can reduce the impact of elastic contraction on the consistency of FDSM coordinates.

[0056] 2. Implementation Results: Under experimental conditions of a fabric width of 200cm, a transmission speed of 60m / min, and a moisture content varying between 8% and 18%, this embodiment, by introducing the compensation mechanism described above, maintains the FDSM coordinate tracking error within ±1.2mm; without moisture content compensation and elasticity correction, the coordinate error can increase to ±5.8mm. These results demonstrate that the present invention maintains good state estimation stability even with elastic fabrics and in moisture-sensitive scenarios.

[0057] Example 3 1. Cross-process coordinate mapping: This embodiment illustrates how the Fabric Defect Semantic Map (FDSM) is transferred between the fabric inspection and fabric laying processes. After the fabric inspection process, the fabric is rolled into a roll and then unfolded again during the fabric laying process. Since the physical coordinate systems of the two processes are different, the fabric's own coordinates in the FDSM need to be mapped to the current coordinates of the fabric laying machine.

[0058] The encoder count is recorded during the winding stage, and the fabric length integral is recorded during the fabric laying stage. Accumulated errors are corrected using optical edge feature points. The coordinate mapping relationship is as follows: ; In the formula, This represents the fabric's own coordinates in the semantic map of fabric defects. Indicates the first roll of cloth The starting coordinates for the expansion of the circle. This indicates the cumulative length of fabric laid during the fabric laying stage. This represents the correction amount calculated by feature point relocation or the joint tracking algorithm shown in Example 1. Downstream devices can query the defect semantics of the current fabric area in real time from the fabric defect semantic map based on this coordinate mapping relationship.

[0059] 2. Defect-driven typesetting: In this embodiment, the layout engine constrains the arrangement of fabric pieces based on the quality semantics in the fabric defect semantic map. In this invention, quality semantics refers to structured information related to fabric quality stored in the fabric defect semantic map, such as defect category labels, center coordinates, bounding boxes, areas, quality grades, and available bit field markers, which is used by downstream equipment for process decisions. For defect areas marked as critical areas that cannot be cut, the layout engine sets them as prohibited areas; for defect areas marked as non-critical areas that can be cut, the layout engine allows them to fall on non-critical areas such as collars and facings; for defect-free areas, they are preferentially assigned to quality-sensitive areas such as necklines and front panels. Through this method, the quality information obtained during the fabric inspection stage can be used for subsequent fabric laying and cutting decisions.

[0060] In a test involving 3200m of fabric with a defect rate of 3.2% and an interval of approximately 2 hours between two processes, the cumulative tracking error of the fabric defect semantic map after adopting the cross-process coordinate mapping of this embodiment was approximately ±1.6mm, meeting the process requirement that the layout accuracy should not exceed ±3mm. The overall fabric utilization rate increased from the baseline value of 85.6% to 89.8% (+4.2 percentage points), of which approximately 2.1 percentage points came from the reasonable replanning of defect areas (some defect areas were changed from "discarded" to be usable for secondary parts), and approximately 2.1 percentage points came from avoiding rework and recutting caused by defects.

[0061] Example 4 1. Adaptive handover process: This embodiment describes a scenario where the same fabric inspection machine processes 8 to 12 different types of fabrics (including pure cotton, polyester, linen, elastic knit, etc.) in one shift.

[0062] When the operator scans the barcode of a new fabric variety or the system recognizes the change in fabric variety, the calibration management and data interaction module triggers the online calibration or parameter update process. This process includes three steps: rapid identification of constitutive parameters, adjustment of sensor working parameters, and setting of attention priors. Specifically: (1) Rapid identification of constitutive parameters: retrieve the historical constitutive parameters of the fabric variety from the fabric variety database (if there are no historical records, trigger the PANN rapid identification program to complete the identification of parameters such as Young's modulus under 50 mechanical test samples, which takes about 3 minutes). The constitutive parameter set is represented as: ; In the formula, This represents the set of constitutive parameters for the new fabric variety, including parameters such as Young's modulus, anisotropy ratio, and bending stiffness. This indicates the mechanical test sample input to PANN. If a historical database record exists, it can be read directly. And update the state transition function; in the absence of historical records, PANN can be used for fast identification.

[0063] (2) Automatic adjustment of sensor parameters: Based on the thickness and color characteristics of the fabric, the camera exposure time (range 0.05~2ms), light source brightness (0~100%) and laser scanning frequency are automatically adjusted; for dark fabrics, the high contrast lighting scheme is automatically switched.

[0064] (3) Attention Prior Setting: Based on the product characteristics (woven / knitted, elastic / inelastic, transparent / opaque), a prior attention weight is preset so that HAC-L3 completes weight convergence within the first 20 frames of data, instead of starting from random initialization. The online update formulas for process noise covariance and observation noise covariance are: ; ; In the formula, and These represent the estimated values ​​of process noise and observation noise obtained from the statistical analysis of the current batch residuals, respectively. and The above formula allows for rapid updating of the noise statistics of the Adaptive Extended Kalman Filter (AEKF) during the initial stage of variety switching, thereby improving the convergence speed of state estimation in the first few frames.

[0065] 2. Implementation Results: Actual tests on 12 typical fabric varieties show that, after adopting the fast switching process of this embodiment, the median time from scanning barcodes to outputting a steady-state estimate is approximately 2.3 minutes; when PANN recognition is required, the total time is approximately 4.8 minutes. Compared with the random initialization method, the uncertainty of the state estimate in the first 20 frames after variety switching is reduced by approximately 43%, indicating that online updating of constitutive parameters and AEKF noise covariance can effectively improve the system's adaptability to new fabric varieties.

[0066] Table 2 summarizes the implementation results of the above four embodiments: Table 2

[0067] In this invention, the typical value ranges and meanings of each parameter are as follows: the applicable range for fabric transmission speed is 20 meters per minute to 200 meters per minute; the applicable range for sensor installation height is 80 millimeters to 150 millimeters; the applicable range for fabric width is 90 centimeters to 220 centimeters; the applicable range for fabric thickness is 0.1 millimeters to 6 millimeter; the number of identifiable defect types is no less than 52; the system's missed detection rate is no higher than 1%; the end-to-end processing delay is no higher than 8 milliseconds; the accuracy of fabric defect semantic map coordinate tracking is no higher than ±2 millimeters (2 hours of continuous tracking); and the adaptation time for variety switching is no more than 5 minutes. Any appropriate adjustments to the above parameter ranges made by those skilled in the art without departing from the basic principles of this invention should be considered as falling within the protection scope of this invention. The above four embodiments illustrate the implementation of this invention from the perspectives of high-speed fabric inspection, moisture content-sensitive scenarios for elastic knitwear, FDSM cross-process transfer, and rapid switching between multiple varieties. It should be noted that the sensor models, installation dimensions, sampling frequencies, computing platforms, and parameter values ​​in each embodiment are preferred examples given for illustrating this invention, and those skilled in the art can make adjustments according to different fabric types, production cycles, and on-site equipment conditions. Therefore, this invention employs the aforementioned method and system, leveraging the collaboration of four types of sensors to expand the fabric state perception dimension from a single optical image to four physical domains: deformation, stress, optics, and composition. The number of detectable defects has been expanded to 52 types, and the state estimation uncertainty has been reduced by 67% compared to single-modal solutions. A hierarchical heterogeneous computing architecture is adopted, deploying the three layers of HAC processing on FPGA, edge GPU, and CPU respectively. At a speed of 200 meters per minute, the end-to-end processing latency is less than 8 milliseconds, representing an improvement of more than four times compared to existing multi-sensor fusion solutions.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for asynchronous calibration and fusion estimation of multimodal hierarchical fabrics, characterized in that, Includes the following steps: S100. Initialization: Perform internal and external parameter calibration on the linear array vision camera, laser profile sensor, flexible pressure sensor array and near-infrared spectrometer respectively, and determine the measurement model parameters of each sensor and the installation pose parameters of each sensor relative to the fabric reference coordinate system. S110, Fabric Detection and Tracking Startup: The fabric entry status is identified through the edge detection algorithm. After the fabric enters the detection area, the raw data collected by each sensor is timestamped based on a unified clock source, and a digital identity (DID) is established for the current fabric. S120, Multi-sensor concurrent acquisition: Based on hardware synchronization pulses, each sensor is triggered to acquire data, and a timestamp is added to each frame of acquired data. The sampling frequency of each sensor is set according to the fabric transmission speed and the spatial resolution of the corresponding sensor. S130 and HAC-L1 processing: Distortion correction and noise modeling are performed on the data output from each sensor, and spatiotemporal alignment is performed. Spatiotemporal alignment includes time synchronization and spatial projection operations. The time synchronization operation is as follows: when the time difference between adjacent sensor frames exceeds the synchronization threshold, interpolation compensation or time alignment strategies are used to align asynchronous data to the master clock time. The spatial projection operation is as follows: according to the predetermined installation pose parameters, the data of each sensor is projected onto a unified world coordinate system, and an aligned data packet is output. S140, HAC-L2 processing: Based on the aligned data package output in step S130, extract multimodal features representing the geometric, mechanical, and optical states of the fabric, and construct a multi-physics joint feature representation; S150 and HAC-L3 processing: The fusion weights of each modal feature are determined according to the fabric type and operating conditions. The multi-physics joint feature representation constructed in step S140 is weighted and fused to obtain the observation vector. Adaptive extended Kalman filtering is performed based on the fabric state transition model to obtain the fabric state estimation result at the current moment and the uncertainty of the fabric state estimation result. The fabric state transition model is composed of the fabric dynamics physical model. The fabric dynamics physical model is based on the fabric continuum theory. The fabric state vector is defined as a joint state vector containing deformation field displacement, strain tensor, yarn tension distribution and defect location set. The state equation satisfies the nonlinear continuum mechanics equations. S160, Anomaly Detection and Triggering: Calculate the uncertainty. When the uncertainty exceeds the preset threshold, trigger the sensor recalibration process and re-estimate the current frame. S170, Output: Output the fabric state vector and uncertainty, and update the fabric defect semantic map to the current frame defect detection result. The fabric defect semantic map is used to continuously maintain the defect location mapping relationship through coordinate tracking in subsequent processes.

2. The fabric multimodal layered asynchronous calibration fusion estimation method according to claim 1, characterized in that, In step S120, the line frequency of the linear scan vision camera is set according to the fabric transmission speed, satisfying the relational expression. , The transmission speed is indicated; the laser profile sensor consists of at least three measurement modules; a flexible pressure sensing array is deployed on the fabric edge guide device, with a sensor element spacing of 3 mm to 10 mm; the wavelength range of the near-infrared spectrometer is 900 nm to 2500 nm.

3. The fabric multimodal layered asynchronous calibration fusion estimation method according to claim 1, characterized in that, In step S130, the time synchronization operation uses the acquisition time of the linear array vision camera as the master clock time; the interpolation compensation is linear interpolation or spline interpolation.

4. The fabric multimodal layered asynchronous calibration fusion estimation method according to claim 1, characterized in that, In step S140, the multimodal features include geometric feature vectors, mechanical feature vectors, and optical feature vectors; The geometric feature vectors are extracted from the laser contour data after reconstructing the three-dimensional shape of the fabric, including the average curvature, Gaussian curvature and normal vector of the grid cells; the mechanical feature vectors are obtained from the pressure sensor array data to calculate the transverse stress distribution gradient of the fabric, including the edge tension non-uniformity index; the optical feature vectors are obtained from the local binary pattern texture descriptor extracted from the visual image data and the defect probability distribution map extracted from the near-infrared spectroscopy data.

5. The fabric multimodal layered asynchronous calibration fusion estimation method according to claim 1, characterized in that, In step S150, the fabric type is... With transmission speed As contextual information, dynamic weight vectors for linear array visual features, laser contour features, pressure-sensing features, and near-infrared spectral features are calculated using a scaled dot product attention mechanism. The fused observation vector is obtained by weighted aggregation of the joint feature vector composed of linear array visual features, laser profile features, pressure sensing features, and near-infrared spectral features. ; Using a fabric dynamics physical model as a priori, and fusing observation vectors The observed values ​​are used for state updates via adaptive extended Kalman filtering, outputting a fabric state vector. and the covariance matrix of the fabric state vector ; The scaled dot product attention mechanism includes calculating the attention matrix. : ； According to the attention matrix Obtain the dynamic weight vector ,in, Indicates context information The query matrix obtained through linear mapping, This represents the key matrix obtained by linear mapping from the joint eigenvectors. express transpose, The dimension of the key vector; Dynamic weight vector Each component satisfies and , Indicates modal index, The weighted components correspond to linear array visual features, laser contour features, pressure sensing features, and near-infrared spectral features, respectively. Adaptive extended Kalman filtering achieves adaptive parameter adjustment for different fabric types and operating speeds by estimating the process noise covariance and observation noise covariance online.

6. The fabric multimodal layered asynchronous calibration fusion estimation method according to claim 1, characterized in that, In step S170, the fabric defect semantic map uses the fabric's own coordinate system as a reference and adopts a quadtree sparse storage structure. Each defect node contains a defect category label, center coordinates, bounding box, area, quality grade, and available bit field markers. Subsequent processes use the digital identity identifier (DID) to query the quality semantics of any coordinate position in real time. The semantic map of fabric defects changes with the position of the fabric during the laying and cutting processes, and the coordinate mapping is continuously updated by a joint tracking algorithm of optical flow field and tension field.

7. A fabric multimodal layered asynchronous calibration fusion estimation method according to any one of claims 1-6, characterized in that, The method employs a hierarchical processing strategy: steps S130 and S140 are processed first, while step S150 is processed later; when the fabric transmission speed exceeds a preset speed threshold, predictive state extrapolation is enabled.

8. A fabric multimodal layered asynchronous calibration fusion estimation system, used to execute the fabric multimodal layered asynchronous calibration fusion estimation method according to claim 7, characterized in that, include: The sensing module includes a linear array vision camera, a laser profile sensor, a flexible pressure sensor array, and a near-infrared spectrometer. The sensing module is mounted on a sensor bracket above the fabric transport line and is connected to the master clock via a hardware synchronization bus. The HAC fusion processing unit includes an independent modeling and spatiotemporal alignment submodule, a multiphysics joint representation submodule, and an attention-weighted Bayesian estimation submodule. The HAC fusion processing unit receives the output of the perception module in sequence and outputs the fabric state vector and the uncertainty of the fabric state vector. The fabric digital twin management module includes a fabric digital identity registration unit, a fabric defect semantic map storage and query unit, and a coordinate tracking unit. The fabric digital twin management module receives the fabric state vector and the uncertainty of the fabric state vector from the HAC fusion processing unit and maintains the digital twin of the entire process. The downstream interface module is used to push the fabric state vector, the uncertainty of the fabric state vector, and the semantic map of fabric defects to the typesetting engine, the fabric laying control system, the cutting equipment, and the sewing production line management platform.

9. A fabric multimodal layered asynchronous calibration fusion estimation system according to claim 8, characterized in that, The HAC fusion processing unit adopts a heterogeneous computing architecture: the independent modeling and spatiotemporal alignment submodule runs on programmable logic devices or edge computing units, the multiphysics joint representation submodule runs on edge graphics processors or artificial intelligence accelerators, and the attention-weighted Bayesian estimation submodule runs on central processing units or servers.

10. A fabric multimodal layered asynchronous calibration fusion estimation system according to claim 8, characterized in that, The sensing module also includes an adjustable sensor bracket, which is used to adjust the mounting height and lateral position of each sensor relative to the fabric surface. The linear array vision camera includes a linear array vision acquisition unit set on the front or back of the fabric, and is used in conjunction with a synchronous strobe light source for illumination; The fabric digital twin management module also includes a constitutive parameter rapid identification unit. The constitutive parameter rapid identification unit uses a physical augmentation neural network to identify the constitutive parameters of the new fabric variety, and updates the fabric state transition model parameters of the adaptive extended Kalman filter after the identification is completed. The system also includes a calibration management and data interaction module, which is used to trigger online calibration or parameter updates when there are changes in fabric type, shift switching or multimodal consistency anomalies, and outputs the fabric state vector, the uncertainty of the fabric state vector and the fabric defect semantic map to the layout engine, the fabric laying control system, the cutting equipment and the sewing production line management platform.

Images