A multi-modal perception-based inorganic fiber drawing broken end online monitoring method and system

Abstract

This invention provides a multimodal sensing method and system for online monitoring of inorganic fiber breakage during drawing. The method includes: acquiring infrared thermal and visible light images of the drawing furnace stencil based on a time synchronization mechanism; preprocessing to extract temperature and visual features of the fiber forming area at each stencil; spatiotemporally registering the temperature and visual features of the fiber forming area at the same stencil to form a multimodal feature vector for that fiber; performing a preliminary judgment on the multimodal feature vector of each fiber forming area based on a threshold method to mark suspected broken fibers; sequentially inputting the multimodal features of the broken fibers into a trained fiber breakage detection model for further identification to obtain a breakage confidence score; if the breakage confidence score exceeds the threshold, the fiber is determined to be broken, triggering an alarm and location processing. This invention effectively solves the problem of unreliability of traditional single sensors in high-temperature, high-speed, and multi-filament environments, significantly improving detection accuracy and efficiency.

Description

Technical Field

[0001] This invention belongs to the field of inorganic fiber manufacturing process monitoring technology, and particularly relates to an online monitoring method and system for inorganic fiber drawing breakage based on multimodal sensing. Background Technology

[0002] Inorganic fibers are continuous fibers made by melting raw materials such as glass and minerals at high temperatures and then drawing them through a spinneret, such as glass fiber and basalt fiber. During the drawing process, single or multiple fibers may break due to process fluctuations, uneven raw material distribution, uneven spinneret temperature, or equipment vibration. This breakage not only leads to production interruptions, spinneret overflow, and raw material waste, but also affects the quality stability of subsequent products. Therefore, developing an online fiber breakage monitoring technology capable of achieving high accuracy and real-time monitoring under complex operating conditions is of significant practical importance for improving the automation level of inorganic fiber production, reducing production costs, and ensuring product quality.

[0003] Traditional monitoring methods mainly rely on manual inspection. Manual inspection is limited by labor costs, inspection frequency, and the experience and judgment of operators, making it difficult to achieve 24-hour uninterrupted real-time monitoring. Furthermore, in the high-temperature and high-dust environment of wire drawing, manual observation is prone to fatigue, resulting in a high rate of missed or false detections.

[0004] Chinese invention patent application CN202110024043.3 discloses an online inspection device for glass fiber yarn based on machine vision. It uses an industrial camera to acquire images of the glass fiber yarn and transmits the image data to an industrial control computer. Image processing algorithms are used to process the acquired yarn images to achieve defect monitoring and alarm prompts for the glass fiber yarn. Chinese invention patent application CN202510786252.X discloses a rapid detection method for fiber breakage rate. It uses a laser confocal microscope to magnify and continuously photograph the fiber cross-section. Computer image analysis software automatically identifies and integrates the obtained cross-sections of the entire fiber bundle, eliminating overlapping areas between images to obtain a complete and accurate fiber cross-section. Metallurgical software is used to identify and mark the fiber filaments, achieving rapid and accurate measurement of the fiber filaments. Chinese invention patent application CN201611143551.9 discloses a real-time non-contact yarn breakage detection method based on line laser. The method uses a line scan camera to acquire yarn images at the irradiation position, uses image processing algorithms to process and analyze the acquired yarn images, detects the number of yarns in the image, compares it with the pre-input number of yarns, determines whether a yarn breakage has occurred during the weaving process, and outputs the result to the controller.

[0005] While the aforementioned existing technologies have explored the field of fiber inspection, they still have certain limitations. For example, the machine vision-based online inspection equipment for glass fiber yarn (CN202110024043.3) mainly relies on a single visual modality, and its imaging quality and detection accuracy may be significantly affected when faced with high-temperature radiation interference during the inorganic fiber drawing process; the rapid detection method for fiber breakage rate (CN202510786252.X) focuses on offline cross-sectional analysis, which is difficult to meet the needs of online real-time monitoring; the real-time non-contact yarn breakage detection method based on line laser (CN201611143551.9), although real-time detection, mainly judges breakage by detecting changes in the number of yarns. For the subtle breakage characteristics of a single fiber, especially in complex scenarios with multiple filament bundles running in parallel, its recognition accuracy and anti-interference ability need to be improved. In addition, the aforementioned existing technologies mainly involve fiber breakage monitoring in low-temperature environments. In response to the complex working conditions of high temperature, high speed, multiple filament bundles, and strong interference during the inorganic fiber drawing process, there is an urgent need to develop an efficient and rapid online monitoring method and system for fiber breakage that can overcome the high temperature environment. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention is based on the changing characteristics of the fiber-forming melt at the nozzle during the fiber drawing process of inorganic fibers. For example, during normal fiber formation, the melt morphology is an inverted cone shape with regular temperature changes; while when fiber breakage occurs, the melt morphology in the fiber-forming zone is spherical, with large temperature fluctuations and discontinuous flow. The purpose of this invention is to provide a multimodal sensing method and system for online monitoring of fiber breakage during inorganic fiber drawing, aiming to solve the problems of unreliable and inefficient monitoring of fiber breakage rate during fiber drawing under high-temperature, high-speed, and multi-filament bundle environments.

[0007] In view of this, the present invention proposes an online monitoring method for inorganic fiber breakage based on multimodal sensing, comprising: Step 1: Based on the time synchronization mechanism, acquire infrared thermal images and visible light images of the wire drawing furnace stencil; Step 2: Perform preprocessing to extract the temperature and visual characteristics of the fiber forming area at each nozzle; Step 3: Spatiotemporally register the temperature characteristics and visual characteristics of the fiber forming area at the same nozzle to form a multimodal feature vector for the fiber; Step 4: Based on the threshold method, make a preliminary judgment on the multimodal feature vector of each fiber forming area and mark suspected broken fibers; Step 5: Input the multimodal features of the severed fiber into the trained severed fiber detection model for further identification to obtain the severed fiber confidence score; Step 6: If the confidence level of the broken fiber exceeds the set threshold, it is determined to be a broken fiber, and an alarm and location processing are triggered.

[0008] Preferably, the time synchronization mechanism in step 1 includes: The infrared thermal image and the visible light image are acquired simultaneously by using a synchronous trigger signal. Add a uniform timestamp to each frame of the acquired infrared thermal image and visible light image; An image frame index mapping table is established to associate and store infrared thermal images and visible light images corresponding to the same timestamp.

[0009] Preferably, the preprocessing in step 2 includes: Noise is removed by Gaussian filtering, contrast is enhanced, and complex background is subtracted by Gaussian mixture model to highlight the target fiber forming area at the nozzle. The fiber forming area morphology at each nozzle is separated using threshold-based segmentation and edge detection image segmentation algorithms, and a unique ID number is assigned.

[0010] Preferably, step 2 involves extracting the temperature and visual features of the fiber forming area at each nozzle, including: extracting the longitudinal temperature distribution curve for each ID number of the fiber forming area from the preprocessed infrared thermal image, and calculating the real-time temperature change rate, temperature standard deviation, and low-temperature anomaly area at suspected breakpoints; and extracting the forming area contour width and motion trajectory for each ID number of the fiber forming area from the preprocessed visible light image.

[0011] Preferably, the preliminary judgment in step 4 includes: marking suspected broken fibers when the slope of the longitudinal temperature distribution curve is greater than a first threshold or the width of the fiber forming area is greater than a second threshold.

[0012] Preferably, the method further includes: updating the first threshold and the second threshold using a statistical method every set number of frames.

[0013] Preferably, the decapitation detection model in step 5 is a convolutional neural network, with inputs including multimodal features such as temperature value, temperature gradient, contour width and trajectory coordinates, and outputs a decapitation confidence score between 0 and 1.

[0014] Preferably, the method further includes a training step for the decapitation detection model, using the cross-entropy loss function as the training objective, minimizing the loss function through the Adam optimizer until the loss function value tends to stabilize and no longer decreases, thereby obtaining a trained decapitation detection model.

[0015] Preferably, step 6 includes: recording the fiber's ID number and timestamp, triggering an alarm, and marking the location of the broken end and leak.

[0016] On the other hand, the present invention provides an online monitoring system for inorganic fiber breakage based on multimodal sensing, comprising: The infrared sensing module is installed below the spindle of the drawing furnace and aligned with the fiber forming area below the furnace outlet to collect infrared thermal images of the fiber bundles during operation. The visual perception module is used to acquire visible light images of the fiber forming area at the nozzle. The data processing and judgment module is used to extract the temperature and visual features of the fiber forming area at each nozzle; to perform spatiotemporal registration of the temperature and visual features of the fiber forming area at the same nozzle to form a multimodal feature vector for that fiber; to perform a preliminary judgment on the multimodal feature vector of each fiber forming area based on a threshold method, marking suspected broken fibers; and to input the multimodal features of the broken fibers sequentially into a trained broken fiber detection model for further identification, obtaining the broken fiber confidence score. The alarm and execution module is used to determine the confidence level. If the confidence level of the broken fiber exceeds the set threshold, it is determined to be a broken fiber, and an alarm and location processing are triggered.

[0017] Preferably, the infrared sensing module uses an infrared thermal imager, and the visual sensing module uses a visual sensor, both installed below the outlet of the stencil and near the fiber forming area, with a field of view covering all fibers to be monitored; the data processing and judgment module is deployed in a high-performance computer equipped with a multi-core processor; the alarm and execution module includes an audible and visual alarm and a monitoring screen.

[0018] Compared with the prior art, the advantages of the present invention are: 1. Multimodal Information Fusion: This invention innovatively combines infrared thermal imaging technology with high-speed visual sensing technology to simultaneously capture the temperature and morphological motion characteristics of fibers. Temperature characteristics can sensitively reflect localized low-temperature anomalies caused by heat loss during fiber breakage, while visual characteristics can intuitively capture abrupt changes in fiber morphology and abnormal motion trajectories. The complementary advantages of both effectively overcome the shortcomings of single sensors, such as susceptibility to interference and incomplete information in complex environments.

[0019] 2. High detection accuracy and reliability: Through a two-level detection mechanism of "threshold initial judgment + machine learning model fine judgment", most normal fibers are quickly screened out using a simple and efficient threshold method, reducing the amount of subsequent calculations. Then, for fibers suspected of being broken, a trained machine learning model (such as CNN) is used to perform in-depth analysis of multimodal fusion features, which significantly improves the accuracy of broken fiber identification and reduces the false detection rate and false negative rate. It is especially suitable for complex working conditions with high temperature, high speed and multiple filament bundles.

[0020] 3. Real-time performance and operability: The system employs high-speed sensors and efficient image processing algorithms to ensure real-time monitoring of rapidly changing fiber conditions. In the event of a fiber breakage, an alarm is quickly triggered and the specific broken fiber bundle is located (by its ID number), facilitating timely handling by operators or automated equipment and minimizing production losses and quality impacts caused by fiber breakage.

[0021] 4. Strong robustness: Through preprocessing of infrared and visible light images (denoising, enhancement, background subtraction), precise spatiotemporal registration, and independent ID tracking of each fiber, the system can effectively resist interference from harsh environmental factors such as high temperature radiation, dust, and vibration, and operate stably.

[0022] In summary, this invention provides an efficient and reliable solution for end-break monitoring during the inorganic fiber drawing process through the deep integration of a multimodal sensing system and algorithms. This solution helps improve the automation and intelligence levels of inorganic fiber production and has significant value for widespread application. Attached Figure Description

[0023] Figure 1 This is a block diagram of the system structure of the present invention; Figure 2 This is a comparison of the normal morphology and the broken morphology of the fiber forming area in this invention; Figure 3 This is a flowchart of the online monitoring method for inorganic fiber breakage during drawing according to the present invention.

[0024] Figure Labels 1. Squeegee 2. Extruder nozzle 3. Fiber forming area 4. Fiber 5. Infrared thermal imager; 6. Vision sensor; 7. Central processing unit; 8. Alarm execution unit. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0026] Example 1 Embodiment 1 of the present invention provides an online monitoring method for inorganic fiber breakage during drawing, comprising the following steps: Step 1: Acquire infrared thermal and visible light images of the stencil and nozzle; through a precise time synchronization mechanism, ensure that the image frames acquired by the infrared sensing module and the visual sensing module correspond one-to-one, laying the foundation for subsequent spatiotemporal registration and feature fusion. Specifically, this includes: a) A global clock generator is used to generate a synchronization trigger signal. The trigger frequency is consistent with the image acquisition frame rate. The synchronization signal is transmitted to the infrared thermal imager and the high-speed visible light camera respectively, controlling the two devices to start image acquisition at the same time. b) Add a uniform timestamp (t, unit: s) to each frame of acquired infrared thermal image and visible light image, with timestamp accuracy reaching the microsecond level (≤0.1s), to ensure that subsequent dual-modal image frames at the same time can be quickly matched through timestamp; c) Establish an image frame index mapping table to associate and store infrared image frames (denoted as I_IR(t)) and visible light image frames (denoted as I_VIS(t)) corresponding to the same timestamp. The mapping relationship is shown in formula (1): (1) Map(t) represents the image frame mapping relationship corresponding to timestamp t. This mapping table allows for the rapid retrieval of dual-modal images at the same time, laying the foundation for subsequent spatiotemporal registration and feature fusion.

[0027] Step 2: Perform preprocessing to extract the temperature and visual characteristics of the fiber forming area at all nozzles, specifically including: a) The acquired infrared thermal image and visible light image are preprocessed respectively. Gaussian filtering is used to remove noise, contrast is enhanced, and a Gaussian mixture model is used to subtract complex backgrounds and highlight the target in the fiber forming area at the nozzle. To address the Gaussian noise present in infrared thermal images and visible light images, a two-dimensional Gaussian filter is used for smoothing. The filter kernel function is shown in formula (2): (2) Where (x,y) are the relative coordinates of the pixels within the filter kernel, and σ is the Gaussian standard deviation (set according to the image noise intensity, preferably σ=1.5~2.5). The filter kernel is convolved with the original image through convolution operation to obtain the denoised images I_IR' and I_VIS'. The convolution operation is shown in formula (3): (3) Where * represents convolution operation, k is half the size of the filter kernel (preferably k=1~2, i.e. 3×3 or 5×5 filter kernel), I(x,y) is the pixel value of the original image, and I'(x,y) is the pixel value of the denoised image.

[0028] An adaptive histogram equalization algorithm is used to enhance image contrast and highlight the difference between the fiber forming area at the nozzle and the background. For each local region of the image, its histogram is calculated and equalization is performed to limit contrast gain and avoid local overexposure or underexposure. The enhanced images are denoted as I_IR'' and I_VIS''.

[0029] To address complex background interference from factors such as temperature fluctuations and lighting changes in the production environment, a Gaussian mixture model (Gaussian model) is used to construct a background model and perform background subtraction. It is assumed that the grayscale value of each pixel follows a mixture of K Gaussian distributions, with K preferably being 3-5. The Gaussian model parameters include the mean μ and variance σ. 2 With weight ω, the model update formulas are shown in formulas (4) to (6): (4) (5) (6) Where t is the current frame number, α is the learning rate (preferably α = 0.01~0.05), ρ is the update rate (ρ = α·ρ, η is the matching coefficient), and M_i(t) is the matching flag (M_i(t) = 1 when I(x,y) matches the i-th Gaussian distribution, otherwise it is 0). By calculating the degree of matching between the current pixel value and each Gaussian distribution (the difference between the pixel value and the mean is less than 2.5σ), the foreground (fiber forming area) and background are determined, and the foreground mask images Mask_IR and Mask_VIS are obtained, achieving effective subtraction of complex backgrounds and highlighting the fiber forming area target at the nozzle.

[0030] b) Using threshold-based segmentation and edge detection image segmentation algorithms, the morphology of the fiber forming area at each nozzle is separated from the preprocessed image, and a unique ID number is assigned to each fiber forming area morphology at the nozzle, enabling individual tracking of multi-filament bundles. Specifically, this includes: The Otsu algorithm adaptively determines the segmentation threshold using the maximum inter-class variance method, binarizes the foreground mask image, and obtains the preliminary outline of the fiber forming area. The core of the Otsu algorithm is to find a threshold T that maximizes the inter-class variance between the foreground and the background. The inter-class variance is calculated as shown in formula (7): (7) in, , These represent the pixel percentages of the background and foreground, respectively. , These are the average gray values ​​for the background and foreground, respectively. Let T be the average grayscale value of the entire image. Iterate through all possible thresholds T and select the one that makes T equal to the average grayscale value of the entire image. The largest T is used as the optimal segmentation threshold, and the binarized images are denoted as Bin_IR and Bin_VIS.

[0031] The Canny edge detection algorithm is used to extract the precise edges of the fiber forming shape. The Canny algorithm includes four steps: Gaussian smoothing (which has been completed in the preprocessing stage), calculation of gradient magnitude and direction, non-maximum suppression, and double threshold detection. The gradient magnitude is calculated using the Sobel operator, as shown in formulas (8) to (9): (8) (9) (10) in, , Let G be the gradient components in the x and y directions, respectively, and G be the gradient magnitude. A high threshold T_h and a low threshold T_l are set (preferably T_h = 0.2G_max, T_l = 0.1G_max, where G_max is the maximum gradient magnitude). Edges with gradient magnitudes greater than T_h and edges between T_l and T_h that are connected to the high threshold edges are retained, resulting in precise edge images Edge_IR and Edge_VIS of the fiber forming area.

[0032] Connectivity analysis is performed on the binarized image Bin_IR (or Bin_VIS), and a seed filling algorithm is used to label each independent connected region (each connected region corresponds to the fiber forming area morphology at a nozzle). A unique ID number is assigned to each connected region (denoted as ID=1,2,...,N, where N is the total number of nozzles), and the coordinates of the minimum bounding rectangle (x_min, y_min, x_max, y_max) corresponding to each ID are recorded to achieve individual differentiation of multi-filament bundles.

[0033] Individual tracking is achieved based on the positional continuity and morphological similarity of connected regions in adjacent frames. The matching degree between the i-th connected region (ID=i) in the current frame and the j-th connected region in the previous frame is defined as shown in formula (11): (11) in , Let i be the area of ​​the current frame i and the area of ​​the previous frame j, respectively. The area of ​​intersection between the two regions. Let Euclidean distance be the coordinates of the centers of the two regions. Distance threshold Weighting coefficients (preferred) =0.6). When When the value is ≥0.7, the two regions are determined to be of the same fiber forming form and inherit their ID number; if a new connected region appears in the current frame (without a matching region in the previous frame), a new ID number is assigned; if the region in the previous frame disappears in the current frame (without a matching region in the current frame), it is marked as a suspected break (further judgment is made based on the features).

[0034] c) For the fiber forming area morphology at the nozzle for each ID number, extract temperature features from the preprocessed infrared thermal image, including but not limited to: longitudinal temperature distribution curve, real-time temperature change rate (first derivative), and the area of ​​the low-temperature anomaly region at suspected breakpoints; extract visual features from the preprocessed visible light image, including but not limited to: morphological contour (e.g., width), motion trajectory, and contour continuity. Specifically, this includes: Temperature feature extraction (based on infrared thermal image I_IR''): Longitudinal temperature distribution curve: Using the central longitudinal axis of the fiber forming zone as the sampling line (along the fiber forming and drawing direction, i.e., the y-axis direction), M sampling points (preferably M=20~30) are evenly selected on the sampling line, and the temperature value of each sampling point is read. (k=1~2,...,M), thus obtaining the longitudinal temperature distribution curve T(y).

[0035] Real-time temperature change rate: The temperature change rate (temperature gradient) is obtained by taking the first derivative of the longitudinal temperature distribution curve T(y), as shown in formula (12): (12) Where Δy is the longitudinal distance between adjacent sampling points. The temperature change rate reflects the trend of fiber forming temperature along the drawing direction, and a significant negative abrupt change (sudden temperature drop) occurs when the fiber breaks.

[0036] Area of ​​the low-temperature anomaly region: Set the low-temperature threshold (Based on the fiber molding melt temperature range during normal production, the preferred setting is...) =1250℃), the temperature in the statistical fiber forming morphology region is lower than 1250℃), The number of pixels is the area of ​​the low-temperature anomaly region. As shown in formula (13): (13) in, I is the fiber-forming melt morphology region corresponding to ID, and I(·) is an indicator function (1 when the condition is met, 0 otherwise).

[0037] Visual feature extraction (based on visible light image I_VIS''): Morphological contour features: The contour width W is the maximum pixel distance of the fiber forming melt morphology in the transverse direction (perpendicular to the drawing direction); the contour curvature is calculated by fitting the arcs of each point on the contour using the least squares method, as shown in formula (14): (14) Where x' and y' are the first derivatives of the contour point coordinates with respect to the arc length, and x'' and y'' are the second derivatives; the roundness C reflects the regularity of the melt shape, as shown in formula (15): (15) Where S is the area of ​​the fiber-molded morphological region, and L is the perimeter of the outline.

[0038] Motion trajectory curve: Based on the fiber forming area ID tracking results of multiple frames of images, the coordinate sequence of the center of the fiber forming melt morphology corresponding to each ID is extracted {( , ), ( , ),...,( , The motion trajectory curve is obtained by polynomial fitting, as shown in formula (16): (16) (17) Where t is the frame number. , is the fitting coefficient, and n is the fitting order (preferably n=2~3).

[0039] The continuity is evaluated using the number of contour breakpoints and the length of contour gaps. The extracted edge contours are traversed, and when the Euclidean distance between adjacent edge points is greater than the gap threshold D_gap (preferably D_gap = 3~5 pixels), it is determined as a breakpoint, and the total number of breakpoints N_break is counted. At the same time, the sum of the lengths of all gaps L_gap is calculated. The larger N_break and L_gap are, the worse the contour continuity is.

[0040] Step 3: Spatiotemporally register the temperature and visual features of the fiber forming area at the same nozzle to form a multimodal feature vector for the fiber. Due to potential differences in the installation positions of the infrared thermal imager and the high-speed vision sensor, it is necessary to spatially calibrate the features of the fiber forming area morphology (associated by ID number) at the same nozzle in both modes to ensure that they describe the same location of the fiber forming area morphology at the nozzle. Simultaneously, ensure the consistency of timestamps to guarantee that the features were acquired at the same time or within a very short time interval. Combine the registered temperature and visual feature parameters to construct the multimodal feature vector of the fiber forming area.

[0041] Spatial coordinate mapping of dual-modal images is achieved based on the calibration board. Using the mapping relationship from step 1, the pixel coordinates of the infrared image are established. , ) and visible light image pixel coordinates ( , The mapping relationship between the two cameras is established. The intrinsic parameters of the two cameras are calibrated using Zhang Zhengyou's calibration method, yielding the intrinsic parameter matrix. , And distortion coefficients D_IR, D_VIS; then solve the extrinsic parameter matrices R, T (rotation matrix and translation vector) through multiple sets of calibration plate corresponding points to achieve coordinate transformation, as shown in formula (18): (18) For each ID corresponding to the fiber-forming melt morphology region, the coordinates of the temperature feature sampling points in the infrared image are converted into visible light image coordinates according to the above mapping relationship, ensuring that the temperature features and visual features describe the same spatial location of the fiber-forming melt morphology.

[0042] Based on the timestamp synchronization mechanism in step S1, it is ensured that the extracted temperature features and visual features come from dual-modal image frames within the same time or a very short time interval. A time interval threshold Δt_max is set (preferably Δt_max = 50 μs). When the temperature feature extraction time t_IR and the visual feature extraction time t_VIS corresponding to the same ID satisfy... If the time synchronization is valid, the frame features are discarded and the features of the adjacent frames are extracted again.

[0043] The temperature and visual features, after spatiotemporal registration, are normalized (to eliminate dimensional differences) and then concatenated to form a multimodal feature vector. The normalization method is min-max normalization, as shown in formula (19): (19) Where f is the original feature value, , These are the minimum and maximum values ​​of the feature in the training set, respectively. These are the normalized feature values. The final constructed multimodal feature vector is shown in formula (20): (20) in, ~ For m normalized temperature features (such as longitudinal temperature distribution sample values, temperature change rate, area of ​​low temperature anomaly region, etc.). ~ are n normalized visual features (such as contour width, curvature, motion trajectory parameters), and the dimension of F is m + n.

[0044] Step 4: Based on the threshold method, make a preliminary judgment on the multi-modal feature vectors of the fiber-forming melt morphology at each nozzle, and mark the suspected broken-end fibers; set a reasonable threshold range. When the slope (temperature change rate) of the longitudinal temperature distribution curve of the fiber-forming melt morphology at a certain nozzle at a certain position is greater than the first threshold (that is, a temperature decrease > 50 °C / mm indicates a sudden temperature drop), or there is an obvious low-temperature abnormal area at this position, and at the same time the contour width shows an abnormal jump or break such as > 1.8 mm, then mark this fiber as a suspected broken-end fiber. The primary judgment can quickly screen out most normal fibers, reduce the computational amount of the subsequent machine learning model, and improve the real-time performance of the system.

[0045] a) Set the threshold range of each key feature: Based on the statistical analysis of a large amount of normal production data, determine the normal threshold range of each key feature, including: b) Temperature change rate threshold T1: The temperature change rate range of normal fibers is [T1_min, T1_max]. When the temperature change rate (indicating a sudden temperature drop), trigger an abnormal mark; c) Low-temperature abnormal area threshold T2: When S_low > T2, trigger an abnormal mark; d) Contour width abnormal threshold T3: The contour width range of normal fibers is [T3_min, T3_max]. When the width < T3_min or the width > T3_max, trigger an abnormal mark; e) Contour break point threshold T4: When N_break > T4, trigger an abnormal mark.

[0046] f) Preliminary judgment rule: When the multi-modal feature vector F corresponding to a certain ID satisfies any of the following conditions, mark it as a suspected broken-end fiber: 1) Temperature change rate and S_low > T2; 2) Contour width abnormal (< T3_min or > T3_max) and N_break > T4; 3) Temperature change rate < and the contour break point N_break > T4.

[0047] g) Threshold optimization: Use statistical methods to dynamically optimize the threshold, and update the normal threshold range of each feature every N frames (preferably N = 1000 frames) to ensure that the threshold adapts to temperature fluctuations, equipment vibrations, etc. in the production process. Step 5: Input the multimodal feature vectors of suspected broken fibers sequentially into the trained machine learning model for further identification to obtain the breakage confidence score. The input to the machine learning model is the multimodal feature vector registered in Step 3, such as temperature values, temperature gradients, fiber width, contour continuity indices, trajectory coordinate sequences, etc., for a specific region. The output is a breakage confidence score between 0 and 1, representing the probability that the fiber has broken. The machine learning model preferably uses a convolutional neural network, especially a network structure designed for sequence data or fused features. The performance of the trained model is evaluated using a test set, including metrics such as accuracy, precision, recall, and F1 score. Specifically, this includes: a) Data Acquisition and Labeling: On the actual inorganic fiber drawing production line, a large amount of dual-modal image data containing normal fibers and broken fibers is collected simultaneously (preferably ≥1000 samples). Professional operators label the image data, marking the location and type of breakage (complete breakage / partial breakage) and the corresponding fiber ID, generating a label vector Y (normal fibers Y=0, broken fibers Y=1).

[0048] b) Dataset construction and preprocessing: The collected raw images and labeled data were organized and divided into training set, validation set and test set in a ratio of 7:2:1. The same preprocessing operations (Gaussian filtering, contrast enhancement, background subtraction) were performed on all dataset images as in online monitoring, and multimodal feature vectors were extracted and normalized using formula (19).

[0049] c) Model training: Input the feature vector F and label Y of the training set into the CNN model, and use the cross-entropy loss function as the training objective, as shown in formula (21): (twenty one) in, The number of samples in the training set. Let i be the true label of the i-th sample. The model's predicted head loss confidence score is used. The Adam optimizer is employed to minimize the loss function. The learning rate η (initially η=0.001, dynamically adjusted using an exponential decay strategy), batch size (preferably batch size=32), and number of training epochs (preferably epochs=50~100) are set. During training, the model performance is evaluated using a validation set after each epoch, and hyperparameters (learning rate, batch size, number of network layers, etc.) are adjusted until the model's loss function value on the validation set stabilizes and no longer decreases.

[0050] d) Model Evaluation and Optimization: The performance of the trained model is evaluated using the test set, with accuracy, precision, recall, and F1 score as evaluation metrics, as shown in formulas (22) to (25): (twenty two) (twenty three) (twenty four) (25) In this model, TP represents a true positive (a severed head is correctly predicted as a severed head), TN represents a true negative (a normal body is correctly predicted as normal), FP represents a false positive (a normal body is incorrectly predicted as a severed head), and FN represents a false negative (a severed head is incorrectly predicted as normal). The model is required to achieve an accuracy ≥98%, a recall ≥97%, and an F1 score ≥97.5% on the test set. If these metrics are not met, the model should be optimized by increasing the number of training samples, adjusting the network structure, and using L2 regularization to suppress overfitting until the performance requirements are met.

[0051] Step 6: If the confidence level of the broken fiber exceeds the set threshold (e.g., 0.95), it is determined to be a broken fiber, and an alarm and location processing are triggered. The central processing unit records the ID number of the broken fiber, the precise timestamp of the breakage, and combines it with its position information in the image. The alarm execution unit issues an audible and visual alarm and highlights the ID of the broken fiber bundle and its approximate physical location (e.g., the area corresponding to the leak number) on the monitoring interface so that the operator can quickly locate and handle it.

[0052] Example 2 Embodiment 2 of the present invention provides a multimodal sensing online monitoring system for inorganic fiber breakage during drawing, comprising: The infrared sensing module includes at least one infrared thermal imager, which is installed below the stencil at an angle of 10-25° to the stencil. It is used to collect infrared thermal images of the stencil and the vicinity of the nozzle in real time to monitor the temperature field distribution at the drawing nozzle and the fiber forming area.

[0053] The visual perception module includes at least one high-speed visual sensor, which is installed in a position close to the infrared thermal imager to synchronously acquire visible light images of the fiber bundles, in order to capture the morphological changes and fiber forming contour features of the melt during the fiber formation process at the nozzle.

[0054] The central processing unit is connected to the infrared sensing module and the visual sensing module respectively. It is used to receive and fuse infrared thermal image data and visible light image data, and execute a comprehensive algorithm that combines primary judgment based on threshold and fine judgment based on machine learning model to identify fiber breakage events in real time and accurately.

[0055] An alarm execution unit, connected to the central processing unit, is used to receive a trigger signal and issue an alarm when the central processing unit determines that a fiber breakage has occurred, while simultaneously locating the breakage position and providing a numerical indication of the current fiber breakage rate.

[0056] like Figure 1 As shown, an online monitoring system for inorganic fiber breakage based on multimodal sensing mainly consists of an infrared thermal imager 5, a vision sensor 6, a central processing unit 7, and an alarm execution unit 8. Both the infrared thermal imager 5 and the vision sensor 6 are installed below the outlet of the sprue plate 1, near the fiber forming area 3, as close as possible to the nozzles 2, but without obstructing normal fiber drawing and equipment maintenance. The infrared thermal imager 5 and the vision sensor 6 are at a 15° angle to the sprue plate 1, ensuring coverage of the field of view of all nozzles (typically a single sprue plate contains 200-400 nozzles), and both cover all the fibers 4 to be monitored. The infrared thermal imager 5 uses a long-wave infrared camera with a resolution of 640×512, a frame rate of 120 fps, and a working wavelength of 8-14μm, with a temperature measurement range of 800-1600℃, and is equipped with a water-cooled protective cover to adapt to high-temperature environments. The vision sensor 6 uses a high-speed industrial camera with a resolution of 1280×1024 and a frame rate of 600 fps, and a specially designed high-temperature resistant filter is installed in front of the lens. The central processing unit 7 uses a high-performance computer, equipped with a multi-core processor (16-core industrial-grade CPU, 3.5GHz) and a dedicated graphics card (24GB professional computing graphics card) to accelerate image processing and model inference. The data interface between the infrared thermal imager 5 and the vision sensor 6 and the central processing unit 7 uses Gigabit Ethernet ×2 and Camera Link interface ×2, with a transmission rate of ≥1Gbps to ensure no data packet loss. The alarm execution unit 8 includes an audible and visual alarm (sound frequency 1.5kHz, red flashing frequency 8Hz) and an interface that displays alarm information and the location of the severed head on the monitoring screen.

[0057] like Figure 3 As shown, the monitoring method flow based on the above system is as follows: Step 1: After system startup, the infrared thermal imager 5 and the visual sensor 6 achieve time synchronization under the control of the central processing unit 7, and begin synchronously acquiring infrared thermal images and visible light images of the nozzle 2 and the fiber forming area 3. Synchronization accuracy is achieved through timestamp calibration to ensure that the time difference between the two modal images is less than 1 second.

[0058] Step 2: Image preprocessing and feature extraction.

[0059] a) Infrared thermal image preprocessing: The infrared image is denoised using a 3×3 Gaussian filter with σ=1.5-2.5; adaptive histogram equalization is used to enhance contrast. Using the full-filament state image of the drawing furnace during stable operation as the background template, the background radiation of the high-temperature furnace body and the nozzle is subtracted by the frame difference method to obtain the foreground region containing only the fiber forming area 3.

[0060] b) Visible light image preprocessing: Gaussian filtering is also performed on the visible light image for noise reduction. For auxiliary light sources of specific wavelengths, threshold segmentation is used to initially extract the fiber forming region, such as... Figure 2 As shown. The Canny edge detection algorithm is then used to optimize the melt profile in the fiber forming zone. Similarly, background subtraction techniques are employed to remove static background elements such as the furnace body and equipment.

[0061] c) Feature extraction: Temperature characteristics: For each fiber ID, extract the longitudinal temperature distribution curve along the fiber forming area skeleton in the infrared image (take a temperature value every 5 pixels). Calculate the real-time temperature change rate of this curve (the ratio of the temperature difference between two adjacent points to their distance), and the temperature standard deviation of the entire curve. Search the curve for areas where the temperature value is more than 20% lower than the average temperature of the surrounding 10 pixels; mark these as suspected low-temperature anomaly areas and record their area and center location.

[0062] Visual features: For each fiber ID, extract the width (along the direction perpendicular to the fiber direction) and continuity (number of break points) of its fiber-forming melt morphology profile from the visible light image. Fit its motion trajectory by the positional changes of the fiber centroid in 5 consecutive frames of images, and calculate the radius of curvature of the trajectory.

[0063] Step 3: Spatiotemporal Registration. For fiber-molded melt at the same leak point with the same ID, a spatial transformation relationship is established based on its center coordinates and contour shape in the infrared and visible light images. Temperature feature points in the infrared image are mapped to their corresponding positions in the visible light image, ensuring that the temperature and visual features of the fiber-molded melt at the same leak point describe the same physical location on the melt. Temporally, the temperature and visual features used for registration are ensured to come from image frames from the same moment or strictly synchronized. Parameters such as the temperature change rate, low-temperature anomaly area, contour width, and contour continuity of the fiber-molded melt at the leak point are combined to form a multimodal feature vector.

[0064] Step 4: Initial Judgment. Set the initial judgment thresholds: The first threshold (temperature change rate threshold) is -50℃ / mm (the negative sign indicates a decrease in temperature). When the temperature change rate at a point on the longitudinal temperature distribution curve is less than this value, it is considered that there is a sudden drop in temperature. The second threshold (outline width) is 1.8 mm. When the outline width of the fiber forming area is greater than this value, it is considered that the shape has changed drastically, and the fiber is marked as a suspected broken fiber.

[0065] Step 5: Fine-tuning using the machine learning model. The multimodal feature vectors of the labeled suspected severed fibers are input into a trained CNN model. This CNN model contains 3 convolutional layers (for extracting local features), 2 pooling layers (for dimensionality reduction and feature selection), 2 fully connected layers (for feature integration and classification), and 1 output layer (outputting 0-1 severed fiber confidence scores via a sigmoid activation function). The model was trained on a dataset containing 1000 labeled samples (70% normal samples and 30% severed fiber samples), achieving an accuracy of 98.5% and a recall of 97.8% on the test set.

[0066] Step 6: Decision and Execution. Set the fiber breakage confidence threshold to 0.9. If the model outputs a fiber breakage confidence score exceeding 0.9, the central processing unit determines that the fiber ID has broken. Immediately record the fiber ID, the precise timestamp of the breakage, and its coordinates in the image. Trigger the alarm execution unit: the audible and visual alarm emits a rapid beep and a red flashing light; an alarm window pops up on the monitoring interface, displaying the fiber breakage ID, the time of occurrence, and marking the location of the broken fiber with a prominent red box on the real-time monitoring screen.

[0067] Through the detailed embodiments described above, this invention can stably, efficiently, and accurately achieve online monitoring of fiber breakage during the inorganic fiber drawing process, providing strong technical support for large-scale industrial production.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for online monitoring of inorganic fiber breakage based on multimodal sensing, comprising: Step 1: Based on the time synchronization mechanism, acquire infrared thermal images and visible light images of the wire drawing furnace stencil; Step 2: Perform preprocessing to extract the temperature and visual characteristics of the fiber forming area at each nozzle; Step 3: Spatiotemporally register the temperature characteristics and visual characteristics of the fiber forming area at the same nozzle to form a multimodal feature vector for the fiber; Step 4: Based on the threshold method, make a preliminary judgment on the multimodal feature vector of each fiber forming area and mark suspected broken fibers; Step 5: Input the multimodal features of the severed fiber into the trained severed fiber detection model for further identification to obtain the severed fiber confidence score; Step 6: If the confidence level of the broken fiber exceeds the set threshold, it is determined to be a broken fiber, and an alarm and location processing are triggered.

2. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 1, characterized in that, The time synchronization mechanism in step 1 includes: The infrared thermal image and the visible light image are acquired simultaneously by using a synchronous trigger signal. Add a uniform timestamp to each frame of the acquired infrared thermal image and visible light image; An image frame index mapping table is established to associate and store infrared thermal images and visible light images corresponding to the same timestamp.

3. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 1, characterized in that, The preprocessing in step 2 includes: Noise is removed by Gaussian filtering, contrast is enhanced, and complex background is subtracted by Gaussian mixture model to highlight the target fiber forming area at the nozzle. The fiber forming area morphology at each nozzle is separated using threshold-based segmentation and edge detection image segmentation algorithms, and a unique ID number is assigned.

4. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 1, characterized in that, Step 2 involves extracting the temperature and visual features of the fiber forming area at each nozzle, including: extracting the longitudinal temperature distribution curve for each ID number of the fiber forming area from the preprocessed infrared thermal image, and calculating the real-time temperature change rate, temperature standard deviation, and low-temperature anomaly area at suspected breakpoints; and extracting the forming area contour width and motion trajectory for each ID number of the fiber forming area from the preprocessed visible light image.

5. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 4, characterized in that, The preliminary judgment in step 4 includes: when the slope of the longitudinal temperature distribution curve is greater than the first threshold or the width of the fiber forming area is greater than the second threshold, then suspected broken fibers are marked.

6. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 5, characterized in that, The method further includes: updating the first threshold and the second threshold using a statistical method every set number of frames.

7. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 1, characterized in that, The decapitation detection model in step 5 is a convolutional neural network. The input includes multimodal features such as temperature value, temperature gradient, contour width and trajectory coordinates, and the output is a decapitation confidence score between 0 and 1.

8. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 1, characterized in that, The method also includes a training step for the decapitation detection model, which uses the cross-entropy loss function as the training objective and minimizes the loss function through the Adam optimizer until the loss function value tends to stabilize and no longer decreases, thus obtaining a trained decapitation detection model.

9. The online monitoring method for inorganic fiber breakage based on multimodal sensing according to claim 1, characterized in that, Step 6 includes: recording the fiber's ID number and timestamp, triggering an alarm, and marking the location of the broken end and leak.

10. An online monitoring system for inorganic fiber breakage based on multimodal sensing, characterized in that, include: The infrared sensing module is installed below the spindle of the drawing furnace and aligned with the fiber forming area below the furnace outlet to collect infrared thermal images of the fiber bundles during operation. The visual perception module is used to acquire visible light images of the fiber forming area at the nozzle. The central processing unit is used to extract the temperature and visual characteristics of the fiber forming area at each nozzle. The temperature and visual features of the fiber forming area at the same nozzle are spatiotemporally registered to form a multimodal feature vector for the fiber. The multimodal feature vector of each fiber forming area is initially judged based on the threshold method, and suspected broken fibers are marked. The multimodal features of the broken fibers are sequentially input into the trained broken fiber detection model for further identification to obtain the broken fiber confidence. and The alarm execution unit is used to determine the confidence level. If the confidence level of the broken fiber exceeds the set threshold, it is determined to be a broken fiber and an alarm and location processing are triggered.

11. The online monitoring system for inorganic fiber breakage based on multimodal sensing according to claim 10, characterized in that, The infrared sensing module uses an infrared thermal imager, and the visual sensing module uses a visual sensor. Both are installed below the outlet of the stencil and near the fiber forming area, with a field of view covering all the fibers to be monitored. The central processing unit uses a high-performance computer equipped with a multi-core processor. The alarm execution unit includes an audible and visual alarm and a monitoring screen.

Images