Warp knitting machine fault early warning method and device, electronic equipment and storage medium

Abstract

The application provides a warp knitting machine fault early warning method and device, electronic equipment and storage medium, relates to the field of textile intelligent manufacturing and equipment predictive maintenance, and through a main shaft phase trigger mechanism, heterogeneous data in multi-modal observation data is forcibly aligned to the same physical coordinate system, and deep semantic alignment of the heterogeneous data is realized. The discrete fault primitive sequence is obtained through the pre-trained feature encoding quantization model, and the non-stationary noise inherent in the efficient operation of the target warp knitting machine is filtered out from the source. In the process of joint training, the mask fault modeling is performed on the Transformer backbone network, so that the pre-trained Transformer backbone network can master the long-range structure prior in the production process, and then the Transformer fault prediction model can realize millisecond-level early warning only by relying on slight semantic deviation without waiting for obvious damage on the fabric surface.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and predictive maintenance technology for textile equipment, and in particular to a method, device, electronic equipment and storage medium for early warning of warp knitting machine faults. Background Technology

[0002] In the field of textile machinery, warp knitting machines, as typical high-speed precision weaving equipment, directly affect fabric quality and production efficiency through their operational stability. Modern industrial production places extremely high demands on warp knitting machines, with spindle speeds typically reaching 2000-2500 rpm. At such extremely high speeds, the physical reciprocating motion of loop-forming components such as groove needles, sinkers, and guide bars is extremely frequent. Any minor mechanical abnormality, yarn fluctuation, or environmental disturbance can rapidly escalate into large-area fabric defects within milliseconds, causing irreversible production losses. Therefore, achieving early warning and preventative maintenance of warp knitting machine malfunctions has become a core issue urgently needing to be addressed in the field of intelligent textile manufacturing.

[0003] Currently, fault detection for warp knitting machines and similar textile equipment mainly relies on manual inspection, real-time detection technology based on machine vision, and time-series detection technology based on high-frequency sensor signals.

[0004] Manual inspection is cost-effective and has a good logic for judging known, obvious macroscopic defects. However, this results-oriented response mode has serious shortcomings in timeliness. Because warp knitting machines operate at extremely high speeds, by the time a person discovers obvious defects such as holes or oil stains on the fabric surface and triggers an alarm, production losses have already occurred, making early warning and prevention impossible. In addition, manual monitoring is susceptible to fatigue and subjectivity, making it difficult to control the false alarm and missed alarm rates.

[0005] Real-time inspection technology based on machine vision mainly utilizes image sensors to acquire images of the fabric surface and uses convolutional neural networks to extract texture features from these images to identify defects such as holes and needle marks. However, image sensors can only capture the results of faults already exposed on the fabric surface and cannot perceive the underlying physical causes of these defects in real time. For example, when physical disturbances such as spindle overheating or abnormal tension fluctuations occur, it often takes several loop cycles for visible abnormalities to form on the fabric, by which time the optimal intervention window has been missed. In addition, image sensors are highly susceptible to interference from changes in workshop lighting and environmental noise such as lint, leading to an increased false alarm rate. Summary of the Invention

[0006] This invention provides a method, device, electronic device, and storage medium for early warning of warp knitting machine faults, in order to overcome the deficiencies existing in related technologies.

[0007] This invention provides a method for early warning of faults in a warp knitting machine, comprising:

[0008] Acquire multimodal observation data during the operation of the target warp knitting machine, the multimodal observation data including multivariable sensor data sequences and fabric surface image sequences;

[0009] Based on the principal axis phase triggering mechanism, the multivariable sensor data sequence and the fabric surface image sequence are aligned to obtain a multimodal aligned data pair.

[0010] Based on a pre-trained feature encoding quantization model, feature extraction and fusion are performed on the multimodal aligned data pairs to obtain a target fused feature sequence in a continuous feature space. Combined with a vector quantization mechanism, the target fused feature sequence is mapped to a discrete fault primitive sequence.

[0011] The fault primitive sequence is input into a pre-trained Transformer fault prediction model to perform forward inference, and the fault prediction result output by the Transformer fault prediction model is obtained.

[0012] The Transformer fault prediction model includes a Transformer backbone network and a classification head connected in sequence. The feature encoding quantization model and the Transformer backbone network are jointly trained based on multimodal observation data samples of warp knitting machine samples and fault labels. During the joint training process, the Transformer backbone network is masked for fault modeling.

[0013] According to the fault early warning method for warp knitting machines provided by the present invention, the training steps of the feature encoding quantization model and the Transformer backbone network include:

[0014] Based on the feature encoding quantization model, feature extraction and fusion are performed on the multimodal aligned data samples corresponding to the multimodal observation data samples to obtain a joint feature vector sequence in the continuous feature space. Then, combined with the vector quantization mechanism, the joint feature vector sequence is mapped into a discrete fault primitive sample sequence.

[0015] A random masking operation is performed on the fault primitive sample sequence to construct a mask sequence, and the mask sequence is input into the Transformer backbone network to perform forward inference to obtain the hidden feature representation output by the Transformer backbone network.

[0016] The hidden feature representations are input to the classification head and the reconstruction head respectively. The classification head outputs the initial fault prediction result, and the reconstruction head outputs the reconstruction result of the time step corresponding to the masking operation in the masking sequence.

[0017] Based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label, the feature encoding quantization model and the Transformer backbone network are jointly trained.

[0018] According to the fault early warning method for a warp knitting machine provided by the present invention, the method further includes jointly training the feature encoding quantization model and the Transformer backbone network based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label.

[0019] The inverse denoiser based on the diffusion model uses the hidden feature representation of the Transformer backbone network as a constraint to perform denoising in the continuous feature space and synthesize virtual fault features.

[0020] Based on the virtual fault characteristics, the joint feature vector sequence is updated;

[0021] The inverse denoiser is trained based on the noisy feature representation obtained by injecting noise into the joint feature vectors in the joint feature vector sequence.

[0022] According to the fault early warning method for warp knitting machines provided by the present invention, the step of jointly training the feature encoding quantization model and the Transformer backbone network based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label specifically includes:

[0023] Based on the gradient stopping operator, the fault primitive sample sequence and the joint feature vector sequence are processed respectively to obtain a first processing result and a second processing result. The word segmenter loss is calculated based on the difference between the first processing result and the joint feature vector sequence and the difference between the second processing result and the fault primitive sample sequence.

[0024] Based on the reconstruction results and the fault primitive samples, the reconstruction loss is calculated;

[0025] Based on the initial fault prediction results and the fault labels, calculate the classification loss;

[0026] The feature encoding quantization model and the Transformer backbone network are jointly trained based on the word segmentation loss, the reconstruction loss, and the classification loss.

[0027] According to the present invention, a fault early warning method for a warp knitting machine is provided, wherein the reconstruction loss is calculated based on the reconstruction result and the fault primitive sample, and then the method includes:

[0028] Solve the gradient flow of the reconstruction loss with respect to the fault primitive samples at each historical time step in the fault primitive sample sequence, and quantify the long-range structural prior of the Transformer backbone network based on the gradient flow.

[0029] Based on the spindle speed and sampling frequency of the warp knitting machine sample, each historical time step in the fault primitive sample sequence is mapped to the phase space to obtain the phase interval in the phase space. Based on the attention weight of the Transformer backbone network at each historical time step, an attention distribution map of the phase interval is generated to quantify the explanatory power of the initial fault prediction result.

[0030] According to the present invention, a fault early warning method for a warp knitting machine includes a feature encoding and quantization model comprising a one-dimensional convolutional neural network, a two-dimensional residual neural network, a fusion layer, and a context-aware word segmenter. The pre-trained feature encoding and quantization model extracts and fuses features from the multimodal aligned data pairs to obtain a target fused feature sequence in a continuous feature space. Combined with a vector quantization mechanism, the target fused feature sequence is mapped to a discrete fault primitive sequence, including:

[0031] Based on the one-dimensional convolutional neural network, the temporal dynamic features of multivariable sensor data in the multimodal aligned data pair are extracted;

[0032] Based on the two-dimensional residual neural network, the spatial texture features of the fabric surface image in the multimodal aligned data pair are extracted;

[0033] Based on the fusion layer, the temporal dynamic features and the spatial texture features are concatenated and linearly projected onto the continuous feature space to generate the target fused feature sequence.

[0034] Based on the context-aware word segmenter, nearest neighbor search is used to map the target fusion feature sequence to the nearest entry in the fault code book, thereby obtaining the fault primitive sequence.

[0035] According to the present invention, a fault early warning method for a warp knitting machine includes aligning the multivariable sensor data sequence with the fabric surface image sequence based on a spindle phase triggering mechanism to obtain a multimodal aligned data pair, comprising:

[0036] Calculate the physical phase of the spindle of the target warp knitting machine;

[0037] The fabric surface image in the fabric surface image sequence when the physical phase of the main axis reaches a preset value is associated with the multivariable sensor data in the multivariable sensor data sequence to obtain the multimodal aligned data pair.

[0038] The present invention also provides a warp knitting machine fault early warning device, comprising:

[0039] The data acquisition module is used to acquire multimodal observation data during the operation of the target warp knitting machine. The multimodal observation data includes multivariable sensor data sequences and fabric surface image sequences.

[0040] The data alignment module is used to align the multivariable sensor data sequence with the fabric surface image sequence based on the main axis phase triggering mechanism to obtain multimodal aligned data pairs.

[0041] The feature processing module is used to extract and fuse features from the multimodal aligned data pairs based on a pre-trained feature encoding and quantization model to obtain a target fused feature sequence in a continuous feature space, and to map the target fused feature sequence into a discrete fault primitive sequence by combining a vector quantization mechanism.

[0042] The forward inference module is used to input the fault primitive sequence into the pre-trained Transformer fault prediction model to perform forward inference and obtain the fault prediction result output by the Transformer fault prediction model.

[0043] The Transformer fault prediction model includes a Transformer backbone network and a classification head connected in sequence. The feature encoding quantization model and the Transformer backbone network are jointly trained based on multimodal observation data samples of warp knitting machine samples and fault labels. During the joint training process, the Transformer backbone network is masked for fault modeling.

[0044] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the warp knitting machine fault early warning method as described above.

[0045] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the warp knitting machine fault early warning method as described above.

[0046] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the warp knitting machine fault early warning method as described above.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The warp knitting machine fault early warning method, device, electronic equipment, and storage medium provided by this invention, through a spindle phase triggering mechanism, forcibly aligns heterogeneous data in multimodal observation data to the same physical coordinate system, achieving deep semantic alignment of heterogeneous data. Discrete fault primitive sequences are obtained through a pre-trained feature encoding quantization model, filtering out inherent non-stationary noise during the efficient operation of the target warp knitting machine from the source. Masked fault modeling of the Transformer backbone network during joint training allows the pre-trained Transformer backbone network to grasp long-range structural priors in the production process, enabling the Transformer fault prediction model to achieve millisecond-level early warning based on only minor semantic deviations, without waiting for obvious damage to the fabric surface. This method not only improves the accuracy of fault prediction results but also enhances early warning precision and real-time response capabilities. Moreover, by combining multivariate sensor data sequences and fabric surface image sequences, this method is insensitive to light fluctuations during long-cycle operation and can effectively cover blind spots of faults without visual representation, such as spindle bearing overheating. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the fault early warning method for warp knitting machines provided by the present invention.

[0051] Figure 2 This is a schematic diagram of the structure of the warp knitting machine fault early warning device provided by the present invention.

[0052] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] In recent years, warp knitting equipment has been widely equipped with sensors such as tension sensors, speed encoders, and vibration accelerometers. Anomaly detection of one-dimensional sensor signals using LSTM or Transformer architectures can capture dynamic fluctuations in physical parameters with strong timeliness. However, due to severe non-stationary noise interference in industrial environments, single-mode sensor signals often struggle to accurately distinguish between normal transient disturbances and early fault signs, easily leading to false alarms. Although native Transformer and its variants (such as Informer and PatchTST) have made progress in time modeling, they often lack a deep understanding of the physical causal logic when dealing with warp knitting machines, which have strict physical syntax and periodic structures (0°-360° phase).

[0055] Although existing technologies have achieved certain results in specific scenarios, their direct application to fault prediction tasks of high-speed warp knitting machines still faces the following four core obstacles:

[0056] 1) The spatiotemporal scale gap and alignment difficulties of cross-modal data. The data generated during the warp knitting machine production process is heterogeneous. One-dimensional sensor signals are high-frequency streaming data at the millisecond level, while fabric surface images are two-dimensional spatial snapshots. Existing methods often only perform coarse-grained timestamp alignment, making it difficult to bridge this scale gap and achieve accurate semantic alignment. If the physical perturbations captured by the sensor and the corresponding image features cannot be accurately correlated at the physical scale, the model will be unable to learn the complete causal chain from physical cause to visual representation, thereby reducing the sensitivity of discrimination.

[0057] 2) Imbalanced samples and difficulty in obtaining rare faults in industrial scenarios. In actual production, warp knitting machines are in normal operation most of the time, resulting in a severe shortage of fault samples, especially early-stage potential faults. Existing discriminative models heavily rely on balanced training sets. Due to a lack of learning of extreme operating conditions and rare fault modes, these models often fail when faced with novel faults, making it difficult to construct effective decision boundaries. Traditional image enhancement techniques such as rotation and scaling can only alter the surface visual appearance and cannot generate deep fault features that conform to physical dynamics.

[0058] 3) Poor robustness to noise in complex industrial environments. The workshop environment contains a large amount of non-stationary electromagnetic noise and high-frequency vibration, which can mask the true early fault semantics. Existing continuous signal processing methods often mix noise with useful semantics, causing the model to easily suffer from gradient vanishing or misidentification when processing long-period data such as 10,000 hours, and failing to focus on the underlying physical manifold changes.

[0059] 4) Lack of modeling of the physical grammar of equipment operation. Warp knitting machines operate according to strict physical laws, such as the periodic reciprocating motion of the looping components and the viscoelastic deformation of the yarn. Most existing deep learning models are treated as black boxes, performing only simple feature matching and lacking prior understanding of the equipment's structure. When the machine experiences unprecedented failures, if the model lacks the ability to identify which states violate physical logic, it cannot effectively capture new risks.

[0060] Based on this, in order to overcome at least one of the above obstacles, this embodiment of the invention provides a method for early warning of warp knitting machine faults.

[0061] Figure 1 This is a flowchart illustrating a fault early warning method for a warp knitting machine provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0062] S1, acquire multimodal observation data during the operation of the target warp knitting machine, the multimodal observation data including multivariable sensor data sequences and fabric surface image sequences;

[0063] S2, based on the main axis phase triggering mechanism, the multivariable sensor data sequence and the fabric surface image sequence are aligned to obtain a multimodal aligned data pair;

[0064] S3, based on the pre-trained feature encoding quantization model, perform feature extraction and fusion on the multimodal aligned data pair to obtain the target fused feature sequence in the continuous feature space, and combine the vector quantization mechanism to map the target fused feature sequence into a discrete fault primitive sequence;

[0065] S4, input the fault primitive sequence into the pre-trained Transformer fault prediction model to perform forward inference, and obtain the fault prediction result output by the Transformer fault prediction model;

[0066] The Transformer fault prediction model includes a Transformer backbone network and a classification head connected in sequence. The feature encoding quantization model and the Transformer backbone network are jointly trained based on multimodal observation data samples of warp knitting machine samples and fault labels. During the joint training process, the Transformer backbone network is masked for fault modeling.

[0067] Specifically, the warp knitting machine fault early warning method provided in this embodiment of the invention is executed by a warp knitting machine fault early warning device, which can be configured in a computer. The computer can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0068] First, step S1 is executed to acquire multimodal observation data during the operation of the target warp knitting machine. The target warp knitting machine can be a high-speed warp knitting machine such as the Karl Mayer HKS 3-M. The multimodal observation data includes multivariable sensor data sequences. and fabric surface image sequences A multivariable sensor data sequence comprises multivariable sensor data from multiple time steps. Each time step is a multivariable time series with a time step length of L, where the time step length is the length of the time window for each time step. The multivariable sensor data involved in each time step may include... Core physical characteristics, such as spindle speed Yarn tension Equipment vibration Motor current , amount of delivery and oil temperature wait. Let W be the height of each fabric surface image in the fabric surface image sequence, W be the width of each fabric surface image in the fabric surface image sequence, and C be the number of channels of each fabric surface image in the fabric surface image sequence. Each fabric surface image in the fabric surface image sequence is used to visually reflect the fabric texture and forming quality.

[0069] Multimodal observation data can be represented as: ;

[0070] Where T represents the number of time steps in S and I, that is, the number of discrete time steps in S and I. For multivariable sensor data at time step t in S, This is an image of the fabric surface at time step t in I.

[0071] Following step S2, due to the extremely high speed of the warp knitting machine, typically 2000-2500 rpm, even minor anomalies can evolve into defects within milliseconds. Therefore, this embodiment of the invention abandons traditional coarse-grained timestamp alignment and introduces a spindle physical phase triggering mechanism to align the multivariable sensor data sequence with the fabric surface image sequence. This involves associating the multivariable sensor data and the fabric surface image when the spindle physical phase reaches a specific value, thereby obtaining a multimodal aligned data pair. The spindle physical phase... This refers to the spindle of the target warp knitting machine in The mechanical angular position during the rotation cycle.

[0072] Next, step S3 is executed, using a pre-trained feature encoding quantization model to extract and fuse features from the multimodal aligned data pairs, obtaining a target fusion feature sequence in a continuous feature space. This target fusion feature sequence can include target fusion features from different time steps. The underlying physical state space of the target warp knitting machine is defined as a compact manifold M, and the obtained target fusion feature sequence satisfies local Lipschitz continuity.

[0073] ;

[0074] in, Let be the Lipschitz constant, representing the target fused feature sequence. The upper limit of smoothness, Target feature sequence fusion The i-th target fusion feature in the data. Target feature sequence fusion The j-th target fusion feature in the data. for The corresponding physical state of the target warp knitting machine for The corresponding physical state of the target warp knitting machine for In compact manifolds The distance on. It is an L2 norm.

[0075] During the normal production phase of the target warp knitting machine, the feature trajectory of the target fusion feature sequence is constrained by physical laws and presents as a highly periodic limit cycle in the latent space.

[0076] When a mechanical malfunction occurs in the target warp knitting machine, the physical disturbance generates a velocity component perpendicular to the current manifold tangent plane, forcing the characteristic trajectory to deviate from the original manifold direction. By monitoring this mathematically represented shift, potential faults can be detected early, enabling preventative maintenance.

[0077] Subsequently, by combining a vector quantization mechanism, the target fused feature sequence is mapped into a discrete fault primitive sequence, achieving cross-modal semantic alignment. The fault primitive sequence can include fault primitives at different time steps, and each fault primitive is a semantic basic unit that deeply integrates the mechanical dynamic state and the visual features of the fabric.

[0078] Finally, step S4 is executed, where the fault primitive sequence is input into the pre-trained Transformer fault prediction model for forward inference, yielding the fault prediction result output by the Transformer fault prediction model. The Transformer fault prediction model comprises a Transformer backbone network and a classification head connected sequentially. The Transformer backbone network obtains and outputs hidden feature representations at different time steps, while the classification head determines the fault prediction result based on these hidden feature representations. This fault prediction result is a fault probability; a fault warning is immediately triggered when the fault probability exceeds a preset threshold.

[0079] After that, it can generate early warning reports and trigger preventive maintenance commands such as shutdown, speed adjustment, and parameter correction.

[0080] The feature encoding quantization model and the Transformer backbone network can be jointly trained using multimodal observation data samples and fault labels from warp knitting machine samples. The warp knitting machine samples can be the same as the target warp knitting machine, or different warp knitting machines of the same model as the target warp knitting machine; no specific limitation is made here. The multimodal observation data samples can be multimodal observation data from each of the past 7 days (24 hours per day). The data type of the multimodal observation data samples is the same as the data type of the multimodal observation data during the operation of the target warp knitting machine. The multimodal observation data samples can include multivariate sensor data sequence samples and fabric surface image sequence samples.

[0081] Fault labels can include information indicating normal, unknown, and different types of faults, which can be represented by different numbers.

[0082] During joint training, masked feature modeling (MFM) can be performed on the Transformer backbone network, forcing the Transformer backbone network to infer and reconstruct the masked mechanical state based on the global context, thereby learning and capturing the long-range structural priors and physical coupling relationships that span time and space within the operating cycle of the warp knitting machine sample.

[0083] Furthermore, in the application process, the Transformer fault prediction model has mastered long-range structural priors. The Transformer fault prediction model does not rely on specific fault labels, but rather on the degree of violation of physical priors as the criterion for judgment, so as to achieve very early warning of potential faults.

[0084] The warp knitting machine fault early warning method provided in this embodiment of the invention uses a spindle phase triggering mechanism to force heterogeneous data in multimodal observation data to be aligned to the same physical coordinate system, achieving deep semantic alignment of heterogeneous data. Discrete fault primitive sequences are obtained through a pre-trained feature encoding quantization model, filtering out inherent non-stationary noise during the efficient operation of the target warp knitting machine from the source. Masked fault modeling is performed on the Transformer backbone network during joint training, enabling the pre-trained Transformer backbone network to grasp long-range structural priors in the production process. This allows the Transformer fault prediction model to achieve millisecond-level early warning based on only minor semantic deviations, without waiting for obvious damage to the fabric surface. This method not only improves the accuracy of fault prediction results but also enhances early warning precision and real-time response capabilities. Moreover, by combining multivariate sensor data sequences and fabric surface image sequences, this method is insensitive to light fluctuations during long-term operation and can effectively cover blind spots of faults without visual representation, such as spindle bearing overheating.

[0085] Based on the above embodiments, the training steps of the feature encoding quantization model and the Transformer backbone network include:

[0086] Based on the feature encoding quantization model, feature extraction and fusion are performed on the multimodal aligned data samples corresponding to the multimodal observation data samples to obtain a joint feature vector sequence in the continuous feature space. Then, combined with the vector quantization mechanism, the joint feature vector sequence is mapped into a discrete fault primitive sample sequence.

[0087] The fault primitive sample sequence is randomly masked to construct a mask sequence, and the mask sequence is input into the Transformer backbone network to perform forward inference to obtain the hidden feature representation output by the Transformer backbone network.

[0088] The hidden feature representations are input to the classification head and the reconstruction head respectively. The classification head outputs the initial fault prediction result, and the reconstruction head outputs the reconstruction result of the time step corresponding to the masking operation in the masking sequence.

[0089] Based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label, the feature encoding quantization model and the Transformer backbone network are jointly trained.

[0090] Specifically, in the joint training process of the feature encoding and quantization model and the Transformer backbone network, the feature encoding and quantization model is first used to extract and fuse features from the multimodal aligned data pairs corresponding to the multimodal observation data samples, obtaining a joint feature vector sequence in a continuous feature space. Then, combined with a vector quantization mechanism, the joint feature vector sequence is mapped to a discrete fault primitive sample sequence. Specifically, using a principal axis phase triggering mechanism, the multivariate sensor data sequence samples and the fabric surface image sequence samples are aligned to obtain the multimodal aligned data pairs corresponding to the multimodal observation data samples. The joint feature vector sequence can include joint feature vectors from different time steps, and the fault primitive sample sequence can include fault primitive samples from different time steps.

[0091] Subsequently, a masking matrix can be used to randomly mask the sequence of fault primitive samples, constructing a masked sequence:

[0092] ;

[0093] in, For mask sequence Elements at time step t, For fault primitive sample sequences Fault primitive samples at time step t, Let be the mask matrix at time step t. This represents a special mask placeholder vector. In the mask fault modeling phase, Replace some information in the fault primitive sample sequence to construct a mask sequence with missing information.

[0094] Subsequently, the mask sequence is input into the Transformer backbone network to perform forward inference, and the hidden feature representation output by the Transformer backbone network is obtained by utilizing the surrounding global context information.

[0095] The hidden feature representations are input into the classification head and the reconstruction head, respectively. The classification head outputs the initial fault prediction result, and the reconstruction head outputs the reconstruction result of the time step corresponding to the masking operation in the masking sequence.

[0096] By using the fault primitive sample sequence, joint feature vector sequence, reconstruction results, fault primitive samples at the time step corresponding to the masking operation, initial fault prediction results, and fault labels, the training loss can be calculated. The training loss is then used to jointly train the feature encoding quantization model and the Transformer backbone network until the training loss converges or the preset number of iterations is reached. The joint training ends, resulting in an applicable feature encoding quantization model and Transformer backbone network.

[0097] In this embodiment of the invention, a mask sequence is constructed by randomly masking the fault primitive sample sequence, and a reconstruction head is introduced to achieve mask fault modeling.

[0098] Based on the above embodiments, the joint training of the feature encoding quantization model and the Transformer backbone network based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label further includes:

[0099] The inverse denoiser based on the diffusion model uses the hidden feature representation of the Transformer backbone network as a constraint to perform denoising in the continuous feature space and synthesize virtual fault features.

[0100] Based on the virtual fault characteristics, the joint feature vector sequence is updated;

[0101] The inverse denoiser is trained based on the noisy feature representation obtained by injecting noise into the joint feature vectors in the joint feature vector sequence.

[0102] Specifically, during the joint training of the feature encoding quantization model and the Transformer backbone network, the inverse denoiser of the diffusion model can be utilized. Using the hidden feature representation of the Transformer backbone network as a constraint, this ensures that the generated features conform to the dynamic logic of the warp knitting process. Denoising is performed within a continuous feature space to synthesize logically consistent virtual fault features. The synthesized virtual fault features possess physical coherence, rather than being merely statistically correlated terms.

[0103] The hidden features of the Transformer backbone network can be represented as follows:

[0104] ;

[0105] Where H is the hidden feature representation sequence of the Transformer backbone network. The hidden feature representation for time step t.

[0106] Subsequently, the synthesized virtual fault features can be directly used as supplementary samples to update the joint feature vector sequence, so that the virtual fault features can participate in subsequent joint training and be evaluated as an important indicator in the final performance verification.

[0107] The inverse denoiser is trained based on the noisy feature representation obtained by injecting noise into the joint feature vector in the joint feature vector sequence.

[0108] The noise-adding feature is represented as:

[0109] ;

[0110] In the forward propagation process of the diffusion model, the joint feature vector sequence of the (τ-1)th noise addition step is given. At time, the joint feature vector sequence of the τth noise-adding step The conditional probability distribution, The Gaussian mean determines the joint feature vector sequence for the τ-th noise-adding step. Retain the original information from the previous time step. It is the covariance matrix of the noise injected in the τth noise-adding step. Let be the identity matrix, and let the injected noise be isotropic in the multidimensional space. It is the step-size related variance coefficient, which controls the noise intensity or variance magnitude injected in the τth noise-adding step. It follows a normal distribution.

[0111] Loss function of inverse denoiser Used for training inverse denoising Synthesize virtual fault features:

[0112] ;

[0113] in, It is the expectation, representing the joint feature vector sequence at each given noisy addition step. Injected noise and noise addition steps Find the average value over the distribution.

[0114] In this embodiment of the invention, to address the scarcity of rare fault samples in industrial data, virtual fault features conforming to physical logic are synthesized at the continuous feature space level for data augmentation. This enables the Transformer fault prediction model to effectively identify abnormal deviations in manifold trajectories even when faced with unprecedented faults, such as sinker wear or jacquard machine misalignment. Simultaneously, the inverse denoising unit uses the hidden feature representation of the Transformer backbone network as a constraint condition, ensuring that the synthesized virtual fault features adhere to physical laws such as momentum conservation and yarn viscoelastic deformation, thus meeting the requirements for rationality.

[0115] Based on the above embodiments, the joint training of the feature encoding quantization model and the Transformer backbone network based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label specifically includes:

[0116] Based on the gradient stopping operator, the fault primitive sample sequence and the joint feature vector sequence are processed respectively to obtain a first processing result and a second processing result. The word segmenter loss is calculated based on the difference between the first processing result and the joint feature vector sequence and the difference between the second processing result and the fault primitive sample sequence.

[0117] Based on the reconstruction results and the fault primitive samples, the reconstruction loss is calculated;

[0118] Based on the initial fault prediction results and the fault labels, calculate the classification loss;

[0119] The feature encoding quantization model and the Transformer backbone network are jointly trained based on the word segmentation loss, the reconstruction loss, and the classification loss.

[0120] Specifically, during the joint training of the feature encoding quantization model and the Transformer backbone network, the word segmenter loss can be calculated using the following formula:

[0121] ;

[0122] in, For the loss of the word segmenter, The codeword loss is used to stop the joint feature vector sequence. Gradient backpropagation forces the entries in the fault code book to move to the fault primitive sample sequence. Get closer To commit to loss, the commitment to loss is achieved by stopping the fault primitive sample sequence. Gradient backpropagation, constraining fault primitive sample sequences Don't stray too far from the fault code list. For gradient stopping operators, This is a balancing coefficient used to control the constraint strength between codebook loss and commitment loss.

[0123] By using the word segmentation loss, the fault code book can be constrained and updated, ensuring that the fault code book learned by the feature encoding quantization model can accurately reflect the structured features and underlying logic of the warp knitting process.

[0124] Using the reconstruction results and fault primitive samples, the reconstruction loss is calculated. This reconstruction loss can be either mean squared error (MSE) or cross-entropy loss. For example, the reconstruction loss can be calculated using the following formula:

[0125] ;

[0126] in, The reconstruction loss is M, where M is the set of time steps corresponding to the masking operation. This is a fault primitive sample for time step m corresponding to the masking operation in the masking sequence. This represents the reconstruction result of time step m corresponding to the masking operation in the masking sequence.

[0127] The greater the reconstruction loss, the stronger the irrelevance between the fault primitive sample and the corresponding reconstruction result, that is, the more serious the violation of the structural prior by the reconstruction process.

[0128] Using the initial fault prediction results and fault labels, calculate the classification loss. This classification loss can be the binary cross-entropy loss, which can be calculated using the following formula:

[0129] ;

[0130] in, For classifying losses, For the first Fault label at each time step, The initial fault prediction result at the i-th time step is the probability of a fault.

[0131] It can be represented as:

[0132] ;

[0133] in, It is a non-linear activation function. This is the weight matrix of the classification heads. Let be the hidden feature representation at the i-th time step. This is the bias vector for the classification head.

[0134] The training loss can be calculated using the word segmentation loss, reconstruction loss, and classification loss, as follows:

[0135] ;

[0136] in, For training loss, These are the weight coefficients corresponding to the word segmentation loss, reconstruction loss, and classification loss, respectively.

[0137] The feature encoding quantization model and the Transformer backbone network are jointly trained using training loss. Specifically, the word segmentation loss is used to optimize the feature encoding quantization model, the reconstruction loss is used to drive the Transformer backbone network to master structural priors, and the classification loss is used to optimize the classification accuracy of the Transformer fault prediction model.

[0138] In this embodiment of the invention, multi-task collaborative optimization can be achieved through word segmentation loss, reconstruction loss and classification loss, thereby improving the performance of the feature encoding quantization model and the Transformer backbone network.

[0139] Based on the above embodiments, the step of calculating the reconstruction loss based on the reconstruction results and the fault primitive samples includes:

[0140] Solve the gradient flow of the reconstruction loss with respect to the fault primitive samples at each historical time step in the fault primitive sample sequence, and quantify the long-range structural prior of the Transformer backbone network based on the gradient flow.

[0141] Based on the spindle speed and sampling frequency of the warp knitting machine sample, each historical time step in the fault primitive sample sequence is mapped to the phase space, and based on the attention weights of the Transformer backbone network at each historical time step, an attention distribution map of a specific phase interval is generated to quantify the explanatory power of the initial fault prediction result.

[0142] Specifically, after calculating the reconstruction loss, the gradient flow of the reconstruction loss with respect to the fault primitive samples at each historical time step in the fault primitive sample sequence can be solved using the following formula:

[0143] ;

[0144] in, To reconstruct the loss, the fault primitive samples at historical time step j in the fault primitive sample sequence are used. gradient, Let be the hidden feature representation of time step m corresponding to the masking operation in the masking sequence. This represents the attention weight between historical time step j and the time step m corresponding to the masking operation in the Transformer backbone network.

[0145] when When it is large, it means that the fault primitive sample of historical time step j is large. Even the slightest change can cause The dramatic fluctuations indicate that the Transformer backbone network has learned the strong physical coupling between the historical time step j and the fault determination time step t. That is, the causal coupling relationship between physical states can be quantified through gradient flow, thereby quantifying the long-range structural prior of the Transformer backbone network, enabling the Transformer backbone network to master the physical laws of the target warp knitting machine operation.

[0146] In the visualization diagnosis of the Transformer fault prediction model, the gradient transformation of the feature trajectory of the target fusion feature sequence and the divergence direction of different feature trajectories can provide clear diagnostic clues. The Transformer fault prediction model can achieve very early capture of unknown risks based solely on physical and logical violations.

[0147] Furthermore, to transform the black-box reasoning process of the Transformer fault prediction model into interpretable evidence, an attention analysis method based on phase aggregation is introduced. Based on the spindle speed and sampling frequency of the warp knitting machine samples, each historical time step in the fault primitive sample sequence is mapped to a phase space of 0°-360°, thus obtaining a specific phase interval. For example, the number of sampling points per revolution can be calculated first based on the spindle speed and sampling frequency of the warp knitting machine sample: ;in, Number of sampling points per revolution f is the spindle speed, and f is the sampling frequency.

[0148] Subsequently, the specific phase obtained by mapping historical time step j It can be represented as:

[0149] .

[0150] By utilizing the attention weights of the Transformer backbone network at each historical time step, an attention distribution map for a specific phase interval is generated to quantify the explanatory power of the initial fault prediction results. The attention distribution map can be represented as:

[0151] ;

[0152] in, For a specific phase interval Attention distribution map Indicates a specific phase interval Attention weights corresponding to historical time step j.

[0153] Attention distribution maps quantify the explanatory power of historical data from specific process stages on initial fault prediction results, enabling early capture of model inference mode shifts.

[0154] Based on the above embodiments, the feature encoding quantization model includes a one-dimensional convolutional neural network, a two-dimensional residual neural network, a fusion layer, and a context-aware word segmenter; the pre-trained feature encoding quantization model extracts and fuses features from the multimodal aligned data pairs to obtain a target fused feature sequence in a continuous feature space, and combines a vector quantization mechanism to map the target fused feature sequence into a discrete fault primitive sequence, including:

[0155] Based on the one-dimensional convolutional neural network, the temporal dynamic features of multivariable sensor data in the multimodal aligned data pair are extracted;

[0156] Based on the two-dimensional residual neural network, the spatial texture features of the fabric surface image in the multimodal aligned data pair are extracted;

[0157] Based on the fusion layer, the temporal dynamic features and the spatial texture features are concatenated and linearly projected onto the continuous feature space to generate the target fused feature sequence.

[0158] Based on the context-aware word segmenter, nearest neighbor search is used to map the target fusion feature sequence to the nearest entry in the fault code book, thereby obtaining the fault primitive sequence.

[0159] Specifically, the feature encoding quantization model includes a one-dimensional convolutional neural network (CNN), a two-dimensional residual neural network (ResNet), a fusion layer, and a context-aware word segmenter.

[0160] One-dimensional convolutional neural networks can perform nonlinear mapping on multivariate sensor data in multimodal aligned data pairs to extract the temporal dynamic features of the multivariate sensor data. These temporal dynamic features are designed to capture anomalous triggers with strong temporal correlation, such as instantaneous tension drops or high-frequency vibration shocks.

[0161] Two-dimensional residual neural networks can perform pixel-by-pixel convolution processing on fabric surface images in multimodal aligned data pairs to extract spatial texture features of the fabric surface images.

[0162] The feature extraction process for multimodal aligned data pairs can be represented as:

[0163] ;

[0164] in, For time-dynamic features, Spatial texture features, It is a one-dimensional convolutional neural network. It is a two-dimensional residual neural network. For the multivariable sensor data in the i-th multimodal aligned data pair, This is the fabric surface image in the i-th multimodal aligned data pair.

[0165] The fusion layer can include a stitching layer and a linear projection layer. The stitching layer performs a concatenation operation on the temporal dynamic features of multivariate sensor data and the spatial texture features of fabric surface images in multimodal aligned data pairs to obtain the stitched result. This concatenation operation can effectively bridge the spatiotemporal scale gap between one-dimensional multivariate sensor data and two-dimensional fabric surface images, enabling the initial fusion of information from different modalities at the mathematical representation level.

[0166] The linear projection layer maps the stitching results to a unified continuous feature space, generating a target fusion feature sequence. By transforming temporal dynamic features and spatial texture features of different dimensions into a unified representation of feature dimensions in a continuous feature space, the dynamic temporal information and spatial texture information of the weaving process can be mined.

[0167] Within a unified, continuous feature space, shared parameter weights force the feature encoding quantization model to establish an intrinsic causal relationship between abnormal fluctuations in sensor physical parameters and sparse visual texture on the fabric surface. For example, when the sensor detects yarn tension... During abnormal drops, the feature encoding quantization model can synchronously associate with the warp breakage semantics appearing in the fabric surface image, thereby ensuring that the target fused feature sequence can completely describe the entire chain of fault logic from mechanical disturbance cause to visual result at a unified semantic level.

[0168] The target fusion feature sequence can be represented as:

[0169] ;

[0170] in, To fuse feature sequences for the target, For the target fusion feature sequence at time step t, For linear projection layers, For the splicing result, It represents the feature dimension of the continuous feature space, that is, the dimension of the target fusion feature at each time step in the target fusion feature sequence.

[0171] The context-aware word segmenter can be constructed using a vector quantization variational autoencoder (VQ-VAE). It employs nearest neighbor search to map the target fused feature sequence to the nearest entry in the codebook, resulting in a sequence of fault primitives. The codebook contains K entries, each a high-dimensional state cluster center automatically learned during the training of the feature encoding quantization model. Physically, each entry corresponds to a specific fault primitive or mechanical state snapshot during the operation of a warp knitting machine sample, such as "normal," "fault," or "unknown."

[0172] The fault code could be represented as , For fault codes, This is the k-th entry in the fault codebook. This fault codebook, serving as the core vocabulary for semantic abstraction, can provide a standardized input benchmark for subsequent long-range structural prior learning.

[0173] By using nearest neighbor search, Mapped to the entry with the closest Euclidean distance in the fault code book. Fault primitives for generating time step t :

[0174] ; in, Represents a mapping function. Indicates to make Take the minimum value of k.

[0175] Fault primitives at each time step can collectively form a fault primitive sequence. Here, the fault primitives at each time step are no longer simple numerical signals, but semantic snapshots that deeply integrate mechanical dynamics and visual features and are supported by mathematical logic. This achieves deep alignment between multivariate sensor data sequences and fabric surface image sequences at the semantic level, laying the foundation for subsequent long-range structural prior knowledge through mask modeling tasks. It laid the mathematical foundation for discretization.

[0176] To address the challenge of cross-modal data alignment in existing technologies, based on the above embodiments, the alignment process of the multivariable sensor data sequence and the fabric surface image sequence using a principal axis phase triggering mechanism to obtain multimodal aligned data pairs includes:

[0177] Calculate the physical phase of the spindle of the target warp knitting machine;

[0178] The fabric surface image in the fabric surface image sequence when the physical phase of the principal axis reaches a specific value is associated with the multivariable sensor data in the multivariable sensor data sequence to obtain the multimodal aligned data pair.

[0179] Specifically, in order to overcome the spatiotemporal scale gap between one-dimensional high-frequency multivariable sensor data and two-dimensional fabric surface images, a principal axis phase triggering mechanism is introduced:

[0180] ;

[0181] in, For multimodal aligned data pairs, These are the principal axis physical phases. Multivariable sensor data and fabric surface images at specific values, where T is the alignment model based on the principal axis phase triggering mechanism, and L is... The time step.

[0182] Different specific values ​​correspond to different specific stages of the loop-forming cycle. For example, a specific value of 90° means that the main shaft of the target warp knitting machine has just rotated to the press-in stage; a specific value of 270° means that the main shaft of the target warp knitting machine has reached the loop-breaking stage; and a specific value of 360° means that the main shaft of the target warp knitting machine has reached the loop-forming stage.

[0183] The physical processes of loop formation, needle pressing, and loop release in a target warp knitting machine are entirely controlled by the physical phase of the spindle. The mechanical state at the same clock moment is completely different at different rotational speeds, but the physical phase of the same spindle remains the same. Corresponding to the same mechanical state. By using the physical phase of the main shaft of the target warp knitting machine, it can be ensured that the fabric surface image always corresponds to a specific stage of the loop forming cycle, such as the press-in stage, so that each fabric surface image is precisely correlated with multivariable sensor data with a time step of L on a physical scale.

[0184] The following multimodal observation data samples were extracted from 10,000 hours of real production logs of a large textile enterprise production line to verify the warp knitting machine fault early warning method provided in this embodiment of the invention. Seven core indicators were selected: precision (P), recall (R), F1 score, false alarm rate (FAR), false negative rate (MAR), mean detection delay (MTTD), and area under the curve (AUC). Among them, mean detection delay refers to the time difference between the physical fault occurrence time and the alarm time.

[0185] The formulas for calculating P and R are:

[0186] ;

[0187] ;

[0188] Here, TP refers to the number of samples that are actually faulty but the Transformer fault prediction model correctly identifies them as faulty. FP refers to the number of samples that are actually operating normally but the Transformer fault prediction model incorrectly identifies them as faulty. FN refers to the number of samples that are actually faulty but the Transformer fault prediction model incorrectly identifies them as normal.

[0189] In comparisons with benchmark models such as YOLOv5-Fabric, Informer, TimesNet, and PatchTST, the feature-encoding quantization model-Transformer fault prediction model framework proposed in this embodiment of the invention demonstrates significant advantages. The F1 score reaches 0.961, an improvement of approximately 3% compared to the second-best models. The MTTD is only 124ms. The FAR is only 0.012, far lower than the 0.042 of pure vision models.

[0190] The MTTD relying solely on image input surged to 248ms, demonstrating the lag in visual information; multimodal coupling improved the F1 score by 7%-12%. At this point, the Transformer fault prediction model was forced to reconstruct the state through complex long-range dependencies, most effectively capturing physical correlations.

[0191] The effectiveness of the generated signal was verified through a dual evaluation of physical simulation and expert experience. The dynamic residual was only 0.042, far lower than GAN's 0.185; the expert score was as high as 4.72. The generated energy distribution bias was only 0.021.

[0192] Furthermore, faults not observed during the training phase were introduced, such as sinker wear and jacquard machine misalignment. When the baseline model's F1 score dropped sharply from 0.93 to 0.72, the warp knitting machine fault warning method provided in this embodiment of the invention still maintained a high F1 score of 0.912. Visualization using t-SNE dimensionality reduction demonstrated that although the novel faults were dispersed, they all maintained a clear physical vacuum zone relative to the normal manifold, thus triggering the warning.

[0193] Taking the warp breakage fault early warning in the production process of the Karl Mayer HKS 3-M high-speed warp knitting machine as an example. Real-time reception of yarn tension ( (and synchronized fabric surface images). The feature encoding quantization model transforms microsecond-level yarn tension oscillations into a series of discrete fault primitives, effectively filtering out high-frequency interference in the workshop. The Transformer backbone network uses the learned 0°-360° loop-forming periodic syntax to examine the current primitive sequence. When the yarn experiences an abnormal tension drop due to a potential hazard, and this fluctuation violates the periodic trend of the physical manifold, a shift in the feature trajectory towards feature component 1 is detected in the latent space. A shutdown alarm is issued 124ms in advance, before visible holes appear in the fabric.

[0194] Based on this, the warp knitting machine fault early warning method provided in this embodiment of the invention can significantly improve the accuracy and real-time performance of fault detection in high-speed warp knitting environments, providing an effective solution for preventive maintenance of intelligent textile equipment.

[0195] like Figure 2As shown, based on the above embodiments, this embodiment of the invention provides a warp knitting machine fault early warning device, comprising:

[0196] Data acquisition module 21 is used to acquire multimodal observation data during the operation of the target warp knitting machine. The multimodal observation data includes multivariable sensor data sequences and fabric surface image sequences.

[0197] Data alignment module 22 is used to align the multivariable sensor data sequence and the fabric surface image sequence based on the main axis phase triggering mechanism to obtain multimodal aligned data pairs;

[0198] The feature processing module 23 is used to extract and fuse features from the multimodal aligned data pairs based on a pre-trained feature encoding quantization model to obtain a target fused feature sequence in a continuous feature space, and to map the target fused feature sequence into a discrete fault primitive sequence by combining a vector quantization mechanism.

[0199] The forward inference module 24 is used to input the fault primitive sequence into the pre-trained Transformer fault prediction model to perform forward inference and obtain the fault prediction result output by the Transformer fault prediction model.

[0200] The Transformer fault prediction model includes a Transformer backbone network and a classification head connected in sequence. The feature encoding quantization model and the Transformer backbone network are jointly trained based on multimodal observation data samples of warp knitting machine samples and fault labels. During the joint training process, the Transformer backbone network is used for masked fault modeling.

[0201] Specifically, the functions of each module in the warp knitting machine fault early warning device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0202] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the warp knitting machine fault warning method provided in the above embodiments.

[0203] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0204] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the warp knitting machine fault warning method provided in the above embodiments.

[0205] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the warp knitting machine fault early warning method provided in the above embodiments. This computer-readable storage medium can be either a non-transitory computer-readable storage medium or a transient computer-readable storage medium, and is not specifically limited herein.

[0206] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0207] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for early warning of faults in a warp knitting machine, characterized in that, include: Acquire multimodal observation data during the operation of the target warp knitting machine, the multimodal observation data including multivariable sensor data sequences and fabric surface image sequences; Based on the principal axis phase triggering mechanism, the multivariable sensor data sequence and the fabric surface image sequence are aligned to obtain a multimodal aligned data pair. Based on a pre-trained feature encoding quantization model, feature extraction and fusion are performed on the multimodal aligned data pairs to obtain a target fused feature sequence in a continuous feature space. Combined with a vector quantization mechanism, the target fused feature sequence is mapped to a discrete fault primitive sequence. The fault primitive sequence is input into a pre-trained Transformer fault prediction model to perform forward inference, and the fault prediction result output by the Transformer fault prediction model is obtained. The Transformer fault prediction model includes a Transformer backbone network and a classification head connected in sequence. The feature encoding quantization model and the Transformer backbone network are jointly trained based on multimodal observation data samples of warp knitting machine samples and fault labels. During the joint training process, the Transformer backbone network is masked for fault modeling. The training steps for the feature encoding quantization model and the Transformer backbone network include: Based on the feature encoding quantization model, feature extraction and fusion are performed on the multimodal aligned data samples corresponding to the multimodal observation data samples to obtain a joint feature vector sequence in the continuous feature space. Then, combined with the vector quantization mechanism, the joint feature vector sequence is mapped into a discrete fault primitive sample sequence. A random masking operation is performed on the fault primitive sample sequence to construct a mask sequence, and the mask sequence is input into the Transformer backbone network to perform forward inference to obtain the hidden feature representation output by the Transformer backbone network. The hidden feature representations are input to the classification head and the reconstruction head respectively. The classification head outputs the initial fault prediction result, and the reconstruction head outputs the reconstruction result of the time step corresponding to the masking operation in the masking sequence. Based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label, the feature encoding quantization model and the Transformer backbone network are jointly trained. The update steps for the joint feature vector sequence include: The diffusion-based inverse denoiser uses the hidden feature representation of the Transformer backbone network as a constraint to perform denoising in the continuous feature space and synthesize virtual fault features. Based on the virtual fault characteristics, the joint feature vector sequence is updated; The inverse denoiser is trained based on a noisy feature representation obtained by injecting noise into the joint feature vectors in the joint feature vector sequence. The loss function of the inverse denoiser Used for training inverse denoising Synthesize virtual fault features: ； in, It is the expectation, representing the joint feature vector sequence at each given noisy addition step. Injected noise and the noise addition step Find the average value over the distribution. Let H be the joint feature vector sequence of the τth noise-adding step, and let H be the hidden feature representation sequence of the Transformer backbone network. The alignment process based on the principal axis phase triggering mechanism, which aligns the multivariable sensor data sequence with the fabric surface image sequence to obtain multimodal aligned data pairs, includes: Calculate the physical phase of the spindle of the target warp knitting machine; The fabric surface image in the fabric surface image sequence when the physical phase of the main axis reaches a preset value is associated with the multivariable sensor data in the multivariable sensor data sequence to obtain the multimodal aligned data pair.

2. The warp knitting machine fault early warning method according to claim 1, characterized in that, The joint training of the feature encoding quantization model and the Transformer backbone network based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label specifically includes: Based on the gradient stopping operator, the fault primitive sample sequence and the joint feature vector sequence are processed respectively to obtain a first processing result and a second processing result. The word segmenter loss is calculated based on the difference between the first processing result and the joint feature vector sequence and the difference between the second processing result and the fault primitive sample sequence. Based on the reconstruction results and the fault primitive samples, the reconstruction loss is calculated; Based on the initial fault prediction results and the fault labels, calculate the classification loss; The feature encoding quantization model and the Transformer backbone network are jointly trained based on the word segmentation loss, the reconstruction loss, and the classification loss.

3. The warp knitting machine fault early warning method according to claim 2, characterized in that, Based on the reconstruction results and the fault primitive samples, the reconstruction loss is calculated, followed by: Solve the gradient flow of the reconstruction loss with respect to the fault primitive samples at each historical time step in the fault primitive sample sequence, and quantify the long-range structural prior of the Transformer backbone network based on the gradient flow. Based on the spindle speed and sampling frequency of the warp knitting machine sample, each historical time step in the fault primitive sample sequence is mapped to the phase space to obtain the phase interval in the phase space. Based on the attention weight of the Transformer backbone network at each historical time step, an attention distribution map of the phase interval is generated to quantify the explanatory power of the initial fault prediction result.

4. The warp knitting machine fault early warning method according to any one of claims 1-3, characterized in that, The feature encoding and quantization model includes a one-dimensional convolutional neural network, a two-dimensional residual neural network, a fusion layer, and a context-aware word segmenter. Based on the pre-trained feature encoding and quantization model, features are extracted and fused from the multimodal aligned data pairs to obtain a target fused feature sequence in a continuous feature space. Then, combined with a vector quantization mechanism, the target fused feature sequence is mapped to a discrete sequence of fault primitives, including: Based on the one-dimensional convolutional neural network, the temporal dynamic features of multivariable sensor data in the multimodal aligned data pair are extracted; Based on the two-dimensional residual neural network, the spatial texture features of the fabric surface image in the multimodal aligned data pair are extracted; Based on the fusion layer, the temporal dynamic features and the spatial texture features are concatenated and linearly projected onto the continuous feature space to generate the target fused feature sequence. Based on the context-aware word segmenter, nearest neighbor search is used to map the target fusion feature sequence to the nearest entry in the fault code book, thereby obtaining the fault primitive sequence.

5. A fault early warning device for a warp knitting machine, characterized in that, include: The data acquisition module is used to acquire multimodal observation data during the operation of the target warp knitting machine. The multimodal observation data includes multivariable sensor data sequences and fabric surface image sequences. The data alignment module is used to align the multivariable sensor data sequence with the fabric surface image sequence based on the main axis phase triggering mechanism to obtain multimodal aligned data pairs. The feature processing module is used to extract and fuse features from the multimodal aligned data pairs based on a pre-trained feature encoding and quantization model to obtain a target fused feature sequence in a continuous feature space, and to map the target fused feature sequence into a discrete fault primitive sequence by combining a vector quantization mechanism. The forward inference module is used to input the fault primitive sequence into the pre-trained Transformer fault prediction model to perform forward inference and obtain the fault prediction result output by the Transformer fault prediction model. The Transformer fault prediction model includes a Transformer backbone network and a classification head connected in sequence. The feature encoding quantization model and the Transformer backbone network are jointly trained based on multimodal observation data samples of warp knitting machine samples and fault labels. During the joint training process, the Transformer backbone network is masked for fault modeling. The training steps for the feature encoding quantization model and the Transformer backbone network include: Based on the feature encoding quantization model, feature extraction and fusion are performed on the multimodal aligned data samples corresponding to the multimodal observation data samples to obtain a joint feature vector sequence in the continuous feature space. Then, combined with the vector quantization mechanism, the joint feature vector sequence is mapped into a discrete fault primitive sample sequence. A random masking operation is performed on the fault primitive sample sequence to construct a mask sequence, and the mask sequence is input into the Transformer backbone network to perform forward inference to obtain the hidden feature representation output by the Transformer backbone network. The hidden feature representations are input to the classification head and the reconstruction head respectively. The classification head outputs the initial fault prediction result, and the reconstruction head outputs the reconstruction result of the time step corresponding to the masking operation in the masking sequence. Based on the fault primitive sample sequence, the joint feature vector sequence, the reconstruction result, the fault primitive samples at the time step corresponding to the masking operation, the initial fault prediction result, and the fault label, the feature encoding quantization model and the Transformer backbone network are jointly trained. The update steps for the joint feature vector sequence include: The diffusion-based inverse denoiser uses the hidden feature representation of the Transformer backbone network as a constraint to perform denoising in the continuous feature space and synthesize virtual fault features. Based on the virtual fault characteristics, the joint feature vector sequence is updated; The inverse denoiser is trained based on a noisy feature representation obtained by injecting noise into the joint feature vectors in the joint feature vector sequence. The loss function of the inverse denoiser Used for training inverse denoising Synthesize virtual fault features: ； in, It is the expectation, representing the joint feature vector sequence at each given noisy addition step. Injected noise and the noise addition step Find the average value over the distribution. Let H be the joint feature vector sequence of the τth noise-adding step, and let H be the hidden feature representation sequence of the Transformer backbone network. The data alignment module is specifically used for: Calculate the physical phase of the spindle of the target warp knitting machine; The fabric surface image in the fabric surface image sequence when the physical phase of the main axis reaches a preset value is associated with the multivariable sensor data in the multivariable sensor data sequence to obtain the multimodal aligned data pair.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the warp knitting machine fault early warning method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the warp knitting machine fault early warning method as described in any one of claims 1-4.

Images