Production line blockage early warning and self-healing method based on internet of things

Abstract

This invention provides a method for early warning and self-healing of material blockages on production lines that integrates the Internet of Things (IoT). It relates to the fields of industrial intelligent manufacturing and automation technology. The method involves deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data. This data is then transmitted to edge computing nodes for preprocessing, outputting a multimodal feature dataset. A production line blockage early warning model is constructed based on this multimodal fusion deep learning network model. Blockage monitoring is performed using this model. Based on the monitoring results, corresponding self-healing strategies are executed, generating control commands that are sent to the production line actuators. The entire chain of data for each early warning and self-healing event, along with its final result labels, is collected to form a feedback dataset. This dataset is then used for periodic incremental training and optimization of the production line blockage early warning model. This invention achieves early warning and autonomous handling of blockages, improving production continuity and intelligence.

Description

Technical Field

[0001] This invention relates to the field of industrial intelligent manufacturing and automation technology, and in particular to a method for early warning and self-healing of material blockage in production lines that integrates the Internet of Things. Background Technology

[0002] In modern continuous production processes, such as those in building materials, chemical, and food processing industries, the smooth transport of powdery and granular materials is crucial for ensuring production efficiency and product quality. Material blockage is the most common and highly dangerous type of malfunction in these production lines. At best, it leads to production interruptions and reduced equipment utilization; at worst, it damages drive equipment, generates large amounts of waste, and can even cause production safety accidents.

[0003] Currently, handling material blockages on production lines mainly relies on the following methods: First, traditional manual periodic inspections and experience-based judgment. This method heavily depends on the operator's sense of responsibility and experience, cannot achieve 24-hour uninterrupted monitoring, and is difficult to detect internal blockages or early signs. It often only addresses the issue after blockages have occurred or even caused a shutdown, representing a passive, reactive maintenance approach. Second, methods based on single-sensor threshold alarms, such as installing current sensors on the equipment, triggering an alarm when the motor current exceeds a certain fixed threshold. While simple to implement, this method has a high false alarm rate, cannot distinguish between normal load fluctuations and genuine signs of blockage, cannot pinpoint the fault location, and cannot identify complex or multiple causes of blockages. Furthermore, some more advanced solutions incorporate video monitoring, but this is mostly done manually in the background or using only simple image recognition algorithms, lacking in-depth integration and intelligent analysis of multi-dimensional information such as the dynamic process of material flow, equipment sound, and vibration.

[0004] More importantly, existing technologies generally suffer from the problem of "emphasizing detection but neglecting handling" or "only issuing alarms but not self-healing." Even if the system successfully issues an alert, manual intervention is still required, potentially wasting valuable fault-handling windows and failing to nip problems in the bud. Furthermore, once deployed, traditional models have fixed performance, unable to adapt to changes in production line conditions, equipment wear, and material properties, and lack the ability for continuous optimization.

[0005] Therefore, there is an urgent need in this field for an intelligent material blockage solution capable of early and accurate warning, intelligent decision-making and self-healing, and continuous self-evolution, in order to fundamentally improve the continuity and intelligence level of production lines. In summary, designing an IoT-integrated method for production line material blockage warning and self-healing is essential. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a production line blockage early warning and self-healing method that integrates the Internet of Things.

[0007] To achieve the above objectives, the present invention provides the following solution: This invention provides a method for production line blockage early warning and self-healing that integrates the Internet of Things, comprising: Step 1: Build an IoT sensing layer by deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data, including visual data, acoustic data, vibration data, process parameter data, and environmental data; Step 2: Transmit the multi-source heterogeneous data to the edge computing node, perform preprocessing on each data source, and output a multimodal feature dataset; Step 3: Construct a production line blockage early warning model based on a multimodal fusion deep learning network model, and perform production line blockage early warning monitoring based on the production line blockage early warning model; Step 4: Based on the monitoring results, execute the corresponding self-healing strategy and generate control commands to send to the production line actuators; Step 5: Collect the full-link data and final result labels of each warning and self-healing event to form a feedback dataset. Based on this dataset, perform periodic incremental training and optimization of the production line blockage warning model to achieve continuous self-evolution of the model.

[0008] Preferably, in step 1, an IoT sensing layer is constructed by deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data, including visual data, acoustic data, vibration data, process parameter data, and environmental data. Specifically: Visual sensors are installed inside the raw material silo and intermediate silo, or in the observation window, the discharge port of the screw feeder and rotary valve, the transparent sight glass section of the pneumatic conveying pipeline, the transfer chute and the three-way distribution valve. Acoustic sensors are installed on the outer wall of the drive motor housing, the outer wall of the screw feeder barrel and bearing housing, and the outer wall of the pneumatic conveying pipeline. Vibration sensors are installed on the drive electrode housing and reducer housing, screw feeder hanger bearing support and barrel support point, cyclone separator and dust collector housing. Install a motor current sensor in the power supply circuit of the drive motor; Install level gauges on the top or side of the silo; Install pipeline pressure sensors in the pipeline sections of the conveying system; A speed sensor is installed on the output shaft of the rotary feeder. Environmental sensors were installed inside the workshop and at the material silo ventilation openings. The system collects multi-source heterogeneous data from various sensors, including visual data, acoustic data, vibration data, process parameter data, and environmental data. The process parameter data includes motor current data, material height data in the silo, gas pressure data inside the pipeline, and real-time rotational speed data of the equipment.

[0009] Preferably, in step 2, the multi-source heterogeneous data is transmitted to the edge computing node and preprocessed to output a multimodal feature dataset, specifically: The visual data is processed sequentially by keyframe extraction, image denoising and enhancement, region of interest cropping, and image normalization. The acoustic and vibration data are sequentially sliced ​​and filtered, and the Mel frequency cepstral coefficients of the acoustic data and the time and frequency domain features of the vibration data are extracted to generate a comprehensive vibration feature vector. The process parameter data and environmental data were sequentially processed by time alignment, outlier removal, and data standardization.

[0010] Preferably, in step 3, a production line blockage early warning model is constructed based on a multimodal fusion deep learning network model, and production line blockage early warning monitoring is performed based on the production line blockage early warning model, specifically as follows: The production line blockage early warning model includes a visual feature extraction subnetwork, an acoustic vibration feature extraction subnetwork, a process parameter time series analysis subnetwork, and a multimodal fusion and decision layer.

[0011] Preferably, the visual feature extraction subnetwork is based on an improved multi-scale convolutional neural network, which is composed of alternating multi-scale Inception modules and residual modules.

[0012] Preferably, the acoustic vibration feature extraction subnetwork is composed of an improved one-dimensional convolutional neural network, which employs a large convolutional kernel design, a weakening pooling structure design, and a batch normalization design.

[0013] Preferably, the process parameter timing analysis subnetwork is composed of an improved long short-term memory network, which adopts a bidirectional LSTM design and adds an attention mechanism.

[0014] Preferably, in step 4, the corresponding self-healing strategy is executed based on the monitoring results, and control commands are generated and sent to the production line actuators, specifically as follows: Build a layered strategy generation library; Based on the logic pre-stored in the layered strategy generation library, the corresponding self-healing strategy is executed according to the monitoring results; The decision logic for the layered strategy generation library is as follows: If the monitoring results reach the first-level threshold, the first-level strategy is executed, and instructions are sent to the production line control system to adjust the process parameters of the relevant equipment. If the monitoring result reaches the second-level threshold, the second-level strategy is executed, and an instruction is sent to the production line control system to execute the active unblocking operation sequence. If the monitoring results reach the third-level threshold, the third-level strategy will be executed, sending an orderly shutdown command to the production line control system and an alarm message to the maintenance personnel.

[0015] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention provides a method for production line blockage early warning and self-healing that integrates the Internet of Things (IoT). The method includes constructing an IoT sensing layer, deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data, including visual data, acoustic data, vibration data, process parameter data, and environmental data. This multi-source heterogeneous data is transmitted to edge computing nodes, preprocessed, and output as a multimodal feature dataset. A production line blockage early warning model is constructed based on a multimodal fusion deep learning network model. Blockage early warning monitoring is performed based on this model. Based on the monitoring results, a corresponding self-healing strategy is executed, generating control commands which are sent to the production line actuators. The entire chain of data for each early warning and self-healing event, along with its final result labels, is collected to form a feedback dataset. This dataset is then used for periodic incremental training and optimization of the production line blockage early warning model, enabling continuous self-evolution of the model. Compared to existing technologies, this invention offers significant and multifaceted advantages: 1. Early warning foresight and accuracy: By integrating multimodal information such as vision, acoustics, vibration, and process parameters, and using deep learning methods to explore their deep correlations, the system can capture early and weak fault signs that cannot be detected by the human eye or a single sensor, realizing a fundamental shift from "post-fault processing" to "pre-fault early warning" and significantly reducing the false alarm rate. 2. Timeliness and Autonomy of Response: The system innovatively combines artificial intelligence early warning with automated execution mechanisms to form a complete closed loop of "perception-analysis-decision-execution". The system can automatically take the most appropriate intervention measures in the early stage of risk and resolve the fault before it worsens, which greatly reduces the reliance on external manual intervention and improves the autonomy of the production line. 3. Robustness of multimodal fusion: The multimodal fusion strategy using the attention mechanism enables the system to dynamically adjust the weights of different information when faced with the loss or interference of some sensor signals (such as when the camera is temporarily blocked by dust), and still maintain high judgment accuracy, demonstrating strong system robustness. 4. Preventative nature of independent environmental control: Environmental data is used in an independent control loop, enabling proactive intervention in key causes of material blockage. Together with the core early warning and self-healing system, this forms a dual guarantee of "prevention + treatment," reducing overall risk from the source. 5. System Self-Evolution Capability: Through the closed-loop feedback and incremental learning mechanism in step 5, the system is no longer a static software, but an intelligent agent capable of continuously learning from historical data, especially from its own experiences of success and failure. It can continuously adapt to changes in the production line, becoming more intelligent with use, maintaining and optimizing its performance over the long term, and achieving a leap from "intelligentization" to "smartization". Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the multi-scale convolution module structure; Figure 3 A schematic diagram of an improved multi-scale convolutional neural network structure; Figure 4 A schematic diagram of an improved one-dimensional convolutional neural network structure; Figure 5 This is a schematic diagram of the improved Long Short-Term Memory (LSTM) network. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The purpose of this invention is to provide a production line blockage early warning and self-healing method integrating the Internet of Things (IoT). Through the fusion of multimodal perception and deep learning, it achieves early and accurate warnings and intelligent hierarchical decision-making for blockage risks. The system can automatically execute multi-level self-healing strategies, from parameter fine-tuning to proactive blockage clearing, forming a closed-loop control of "perception-decision-execution," significantly reducing manual intervention. An independent environmental control mechanism prevents blockage triggers at the source, while continuous incremental learning enables the system to self-evolve, effectively improving the continuity, intelligence, and adaptability of the production line.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, this invention provides a production line blockage early warning and self-healing method integrating the Internet of Things, including: Step 1: Build an IoT sensing layer by deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data, including visual data, acoustic data, vibration data, process parameter data, and environmental data; Step 2: Transmit the multi-source heterogeneous data to the edge computing node, perform preprocessing on each data source, and output a multimodal feature dataset; Step 3: Construct a production line blockage early warning model based on a multimodal fusion deep learning network model, and perform production line blockage early warning monitoring based on the production line blockage early warning model; Step 4: Based on the monitoring results, execute the corresponding self-healing strategy and generate control commands to send to the production line actuators; Step 5: Collect the full-link data and final result labels of each warning and self-healing event to form a feedback dataset. Based on this dataset, perform periodic incremental training and optimization of the production line blockage warning model to achieve continuous self-evolution of the model.

[0022] In step 1, an IoT sensing layer is constructed by deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data, including visual data, acoustic data, vibration data, process parameter data, and environmental data. Specifically: 1. Vision sensor Deployment location: (1) Inside the raw material silo and intermediate silo or through the observation window, used to monitor the rate of change of material level and whether structural blockage precursors such as "bridging" or "rat hole" are formed; (2) The discharge port of the screw feeder and rotary valve is used to monitor the continuity, uniformity and flow rate of the discharge flow; (3) The transparent sight glass section of the pneumatic conveying pipeline is used to directly observe the fluidization state of the powdery materials inside the pipeline; (4) Inside the key transfer chute and the three-way distribution valve, used to monitor whether the material accumulates or gets blocked at the transfer point; Output data: Video stream data and keyframe image data at specified resolution and frame rate; 2. Acoustic sensor Deployment location: (1) Drive motor housing, used to collect different sound characteristics emitted by the motor under no-load, normal load and overload (material blockage) conditions; (2) The outer wall of the screw feeder barrel and the bearing seat are used to collect the sound of the internal screw blades rubbing and squeezing with the material and the bearing operation. (3) The outer wall of the pneumatic conveying pipeline is used to collect the sound of material friction with the pipe wall and airflow. The sound spectrum will change significantly before the material is blocked. Output data: Audio waveform data with a specific sampling rate and precision.

[0023] 3. Vibration sensor Deployment location: (1) Drive motor housing and reducer housing, used to monitor the vibration spectrum changes caused by sudden load changes; (2) The screw feeder hanger bearing support and the key support point of the barrel are used to monitor abnormal vibrations caused by uneven material distribution or jamming. (3) Cyclone separator and dust collector housing, used to monitor unbalanced vibrations caused by material accumulation; Output data: Vibration acceleration waveform data in three axes (X, Y, Z directions); 4. Process parameter sensors Deployment location and output data: (1) The motor current sensor is installed in the power supply circuit of the drive motor and outputs the real-time current value of the motor. When the load increases due to material blockage, the current value rises significantly; (2) The level gauge is installed on the top or side of the silo and outputs the height value of the material in the silo. By calculating the height drop rate, it can be indirectly judged whether the feeding is smooth. (3) Pipeline pressure sensors are installed in key sections of the pneumatic conveying system to output the gas pressure value inside the pipeline. In the early stage of material blockage, the pressure will rise abnormally; (4) The speed sensor is installed on the output shaft of the rotary feeder and outputs the real-time speed value of the equipment. It can be compared with the set value to determine whether there is slippage or jamming. (5) Environmental sensors Deployment location: Install near the vent of the silo or in the workshop environment.

[0024] Output data: ambient temperature and relative humidity values, used to monitor environmental conditions. Some materials are prone to absorbing moisture and clumping in high temperature and high humidity environments, which is an important cause of material blockage. 5. Final output dataset All sensor data is collected through an industrial gateway and tagged with a unified and accurate timestamp and device location identifier, forming a multimodal, time-synchronized data set.

[0025] The final output of step 1 is a continuously generated, time-aligned multimodal dataset. This dataset fully represents all aspects of the production line, from equipment operating status and material flow to environmental conditions, providing a solid and rich data foundation for subsequent dynamic assessment of material blockage risk based on deep learning.

[0026] In step 2, the multi-source heterogeneous data is transmitted to edge computing nodes and preprocessed to output a multimodal feature dataset, specifically: The visual data is processed sequentially as follows: 1. Data verification and decoding: Verify the integrity of the video stream data and decode it into a continuous sequence of image frames; 2. Keyframe extraction: Based on a preset sampling frequency (e.g., 1 frame per second or 1 frame per 10 seconds), keyframe images are systematically extracted from the video stream to balance data processing load and information integrity. 3. Image denoising and enhancement: Applying mean value filtering or Gaussian filtering algorithms to suppress noise in the image caused by light fluctuations and dust interference. At the same time, histogram equalization technology is used to enhance image contrast and make the boundary between materials and background clearer. 4. Region of Interest (ROI) cropping: Based on pre-calibrated parameters, the system automatically crops a rectangular region from the full-frame image that contains only the key monitoring targets (such as the discharge port and sight glass), removes irrelevant background, and significantly reduces the amount of data to be processed in subsequent steps. 5. Image standardization processing: Normalize the pixel values ​​of the cropped image to the floating-point range of [0, 1], and scale its size to the fixed resolution required by the model (e.g., 224×224 pixels) to meet the input requirements of the deep learning model.

[0027] Preprocessing of acoustic and vibration data: 1. Data Slicing: Cutting continuous audio and vibration waveform data into data segments of fixed duration (e.g., 1 second), with each segment being an independent processing unit.

[0028] 2. Signal Filtering: Apply bandpass filters to eliminate frequency band interference in the signal that is unrelated to equipment operation and material flow. For example, retain the main characteristic frequency band of 100Hz to 8kHz in the audio signal, and filter out low-frequency hum and high-frequency shrieks in the industrial environment.

[0029] 3. Feature Extraction - Acoustic Data: For each audio data segment, calculate its Mel-frequency cepstral coefficients (MFCCs). The specific process includes: pre-emphasis, framing, windowing, Fast Fourier Transform (FFT), Mel-filter bank filtering, and finally, Discrete Cosine Transform (DCT). The final output is an MFCC feature matrix, which compactly represents the spectral characteristics of the audio signal.

[0030] 4. Feature Extraction - Vibration Data: For each axis of the triaxial vibration acceleration data, perform the following operations: Calculate time-domain characteristics, including root mean square value, peak factor, and amplitude.

[0031] Calculate frequency domain characteristics: Perform a fast Fourier transform on the signal to obtain its spectrum and extract the amplitude energy of a specific frequency band (such as the frequency band where the bearing passes or the gear meshing frequency is located).

[0032] Finally, the characteristics of all axes are merged to form a comprehensive vibration characteristic vector.

[0033] Preprocessing of process and environmental data: 1. Data Alignment: Given that different sensors have different sampling frequencies, based on the global unified timestamp of the previous step, all low-frequency process and environmental data (current, material level, pressure, speed, temperature and humidity) are resampled to a unified time point through linear interpolation to ensure that all data streams are strictly synchronized on the time axis.

[0034] 2. Outlier Removal: Based on common sense physics and equipment limits (e.g., current cannot be negative or exceed the motor stall current), outliers that clearly exceed reasonable ranges are identified and removed. For such missing points, valid data from nearby time points are used to fill in the gaps.

[0035] 3. Data standardization: For each type of process parameter, calculate the mean and standard deviation of its historical data, and perform z-score standardization on the current data to transform it into a distribution with a mean of 0 and a standard deviation of 1, so as to eliminate the influence of different units on the model.

[0036] It should be noted that environmental data does not need to be input into the production line blockage early warning model. It uses independent control logic, sets temperature and humidity thresholds. If the ambient temperature is higher than the temperature threshold, the air conditioning system or refrigeration unit is started. If the ambient temperature is lower than the temperature threshold, the air conditioning system or heater is started. If the humidity is lower than the humidity threshold, no action is generally taken, and the material is dry enough for transportation. If the humidity is higher than the preset threshold, the dehumidifier or drying device is turned on.

[0037] In step 3, a production line blockage early warning model is constructed based on a multimodal fusion deep learning network model. The production line blockage early warning model is then used for early warning monitoring of production line blockages. Specifically: The production line blockage early warning model includes a visual feature extraction subnetwork, an acoustic vibration feature extraction subnetwork, a process parameter time series analysis subnetwork, and a multimodal fusion and decision layer.

[0038] The visual feature extraction subnetwork is based on an improved multi-scale convolutional neural network. Its input is a standardized keyframe image array, and its output is a high-dimensional, fixed-length visual feature vector, which comprehensively represents the macroscopic morphology and microscopic texture state of the material in the monitoring area. The improved multi-scale convolutional neural network consists of alternating multi-scale Inception modules and residual modules, as shown in the schematic diagram below. Figure 3As shown, it will be described in detail: To some extent, the deeper the convolutional neural network, the stronger its learning ability. Increasing the number of convolutional kernels (Conv) can extract feature information at more scales in an image. The core idea of ​​multi-scale convolutional neural networks is to integrate multi-scale feature learning capabilities into traditional convolutional neural networks, using convolutional kernels of different sizes to extract different features, and then performing feature fusion. Therefore, constructing a multi-scale convolutional neural network... Figure 2 The multi-scale convolutional modules shown have kernel sizes of 1×1, 3×3, and 5×5, and each module uses a rectified linear unit (ReLU) as its activation function. Introducing a 1×1 kernel into the multi-scale convolutional Inception module enables dimensionality reduction of the feature maps, which facilitates feature fusion and reduces the number of model parameters, thereby improving network training speed. Convolutional layers, batch normalization, and nonlinear activation functions are introduced into the residual module, such as... Figure 3 As shown, the convolutional layer is used to further extract local features of the image, which helps to obtain richer high-level semantic information when performing information fusion in the deep layers of the network; the batch normalization layer is used to alleviate the gradient vanishing or gradient explosion problems that may occur due to the increase of convolutional layers. The improved residual module calculates the input information as shown in Equation (1): (1) (2) In the formula, H(x) is the output of the residual module; x is the input of the previous layer; F(x) is the feature extraction and nonlinear transformation of x; Conv(·) is the convolution operation; BN(·) is the batch normalization operation; ReLU(·) is the nonlinear activation function; To ensure that the residual modules have the same number of channels as the corresponding convolutional modules, residual modules 1, 2, and 3 are selected with 128, 192, and 256 convolutional kernels, respectively. Each convolutional kernel has a size of 1×1 and a stride of 1. The activation function is ReLU. This invention constructs a multi-scale convolutional neural network model (Inception-Residualnet, IRnet) based on multi-scale Inception modules and residual modules. The IRnet model consists of 3 Inception modules and 3 residual modules, as follows: Figure 3As shown, firstly, the image is input into a convolutional layer, followed by a max pooling layer to suppress noise and reduce redundant features. The Inception module is then used to extract multi-scale features from the image. Secondly, the local features of the image are combined through the residual module to extract deeper features. After the output values ​​of the Inception and residual modules are added together, a max pooling layer is introduced to prevent the loss of image edge and texture features. Finally, based on the feature fusion of the Inception and residual modules, a global average pooling layer is used to process the output features and output a visual feature vector.

[0039] The acoustic vibration feature extraction subnetwork is composed of an improved one-dimensional convolutional neural network. Its input is the acoustic MFCC feature matrix and the vibration comprehensive feature vector (including time domain and frequency domain features) from step 2, and its output is a comprehensive acoustic vibration feature vector.

[0040] The improved one-dimensional convolutional neural network employs a large convolutional kernel design, a weakening pooling structure design, and a batch normalization design. Its structural diagram is shown below. Figure 4 As shown, the traditional one-dimensional convolutional neural network will not be introduced here. Its specific improvements include: 1. Use large convolutional kernels: Convolutional kernels perform convolution operations by sliding across the input data. The size and number of convolutional kernels can be adjusted. Generally speaking, larger convolutional kernels represent a wider receptive field, which is beneficial for convolutional operations to capture rich global features. In contrast, smaller convolutional kernels tend to capture microscopic features. Figure 4 The model shown uses a large 64×1 convolutional kernel in the first layer and small 3×1 convolutional kernels in the remaining layers to improve the accuracy of fault identification. 2. Simplify Pooling Structure: Pooling operations are typically added between consecutive convolution operations to compress data. However, common pooling methods have significant shortcomings. Max pooling only considers the maximum active value within a region, ignoring other active values, meaning a large amount of information is lost. Mean pooling assigns the same weight to each active value within a region, inevitably introducing errors and causing important features to become blurred. Therefore, Figure 4 The model shown adopts a weak pooling structure, retaining only one layer of pooling operations; 3. Add a batch normalization layer: A batch normalization layer can solve the problem of the input distribution of each layer changing with the parameters of the previous layer, thus ensuring that the input signal falls within the desired sensitive region. Figure 4 The model shown adds a batch normalization layer after each convolution operation, which enhances the model's ability to adapt to different data distributions and significantly improves the model's generalization ability and computation speed.

[0041] The process parameter time series analysis subnetwork is composed of an improved long short-term memory network. Its input is the standardized and time-aligned process and environment multidimensional time series matrix from step 2, and its output is a condensed time series context feature vector.

[0042] The improved Long Short-Term Memory (LSTM) network employs a bidirectional LSTM design and incorporates an attention mechanism. The specific structure is as follows: Figure 5 As shown, the traditional Long Short-Term Memory (LSTM) network will not be introduced here. Its specific improvement is as follows: 1. Introduce attention mechanism Traditional LSTM models have limitations in capturing key temporal features when processing long time series. To address this issue, this invention introduces an attention mechanism on top of the traditional LSTM model. This allows the model to automatically assign different weights to the data at different time steps, thereby focusing on the moments that are more important for fault prediction. Specifically, the attention mechanism calculates the attention score at each time step and performs a weighted sum to help the network select the most relevant input features for processing. This is particularly effective in improving fault prediction performance at critical moments of data change. 2. Use bidirectional LSTM Traditional LSTM models process data sequences in a single direction, using only past time step information to predict the current state. However, many time series problems (such as transformer fault prediction) involve dependencies between consecutive time steps. To address this, this invention employs bidirectional long short-term memory (BiLSTM), a structure that allows the model to process sequence data simultaneously in both forward and reverse directions, thereby fully utilizing historical and future information and enhancing the model's contextual understanding capabilities. 3. Enhance the gating mechanism LSTM models control information flow through forget gates, input gates, and output gates to avoid the vanishing gradient problem, thus effectively handling long-term time series data. However, traditional LSTMs have certain limitations when dealing with complex time series patterns. To address this issue, this invention introduces an enhanced gating mechanism. By combining more refined gating structures such as gated recurrent units (GRUs), the information selection and forgetting strategies are improved. Compared to LSTMs, GRUs have a simpler structure, using only update and reset gates, making them more computationally efficient while accurately retaining important information. The update gate determines the amount of information retained at the current time step, while the reset gate controls the memory capacity at each time step. This gives GRUs greater flexibility and robustness when handling complex fault modes. Furthermore, this invention optimizes the LSTM's forget gate, enabling the model to more accurately forget and select information in complex time series data, helping the model better identify features helpful for fault prediction. By reducing the number of gating mechanisms and computational complexity, GRU improves training efficiency, especially when dealing with long sequence data. It reduces computation while maintaining the ability to extract key information, thereby accelerating model convergence and reducing the risk of overfitting.

[0043] The multimodal fusion and decision layer is the brain of the entire main model. Its core task is to solve the spatial alignment and contribution weighting problems of heterogeneous features, and to establish a precise mapping relationship from fused features to risk index. Its input consists of three feature vectors from the three sub-networks, assuming their dimensions are: visual feature vector V (dimension D). v Acoustic vibration feature vector A (dimension D) a Temporal context feature vector T (dimension D) t ); Its specific structure is as follows: 1. Feature normalization and dimensionality reduction: First, the three feature vectors V, A, and T from different sources and with different distributions are transformed nonlinearly and their dimensions are unified through independent fully connected layers. This aims to map the features to the same high-dimensional semantic space and unify them to the same dimension, for example, all of them are converted to 512 dimensions. Subsequently, the transformed features are normalized using the L2 norm to make them unit vectors, in order to eliminate the bias caused by the different original dimensions and numerical ranges of each mode, laying the foundation for subsequent weighted fusion. 2. Weighted feature fusion: An attention-weighted fusion mechanism is adopted to dynamically calculate the weights of each modality feature. The specific implementation is as follows: Each normalized feature vector is input into an attention network, which typically consists of one or more fully connected layers. Finally, a softmax function is used to output the attention score (α) for that modality.v α a α t ), and α v +α a +α t =1; These scores reflect the relative importance of each data type for assessing the risk of material blockage at the current moment. For example, when visual information is obscured, the weights of acoustic vibration and process parameters automatically increase. The final weighted fusion feature vector F is calculated using the following formula: F = α v *V'+α a *A'+α t *T', where V', A', and T' are normalized features; 3. Deep Decision Networks: The fused feature vector F is input into a multilayer perceptron (MLP) for deep decision-making. This MLP consists of at least two fully connected layers. The first fully connected layer receives the fused feature F, has 256 neurons, and uses the ReLU activation function to perform non-linear transformation to extract high-order interaction features. The second fully connected layer has 64 neurons and also uses the ReLU activation function to further abstract and compress information. Dropout layer: A dropout layer can be introduced between two layers, with a dropout rate set to 0.3 to 0.5, to prevent the model from overfitting and enhance its generalization ability; Finally, the features are mapped to the range of 0 to 1 through an output layer (a single neuron using the Sigmoid activation function).

[0044] Final output: The scalar output value of the Sigmoid neuron is the blockage risk index R defined in this model. This index is a continuous, dimensionless value with a range of [0, 1]. In practical applications, a specific threshold range can be set to classify it into levels. For example, [0, 0.3) is "normal", [0.3, 0.6) is "low risk", [0.6, 0.8) is "medium risk", and [0.8, 1.0] is "high risk".

[0045] The training of subnetworks will not be discussed in detail here; they can be trained using a pre-set dataset in a conventional manner.

[0046] In step 4, based on the monitoring results, the corresponding self-healing strategy is executed, and control commands are generated and sent to the production line actuators, specifically: The core design of rule-based expert systems lies in transforming domain knowledge (i.e., experience in handling various material blockage scenarios) into precise and executable logical rules. Input: Blockage risk index R and its corresponding risk level from the main model; Decision-making mechanism and strategy library (executable logic): This invention maintains a hierarchical strategy knowledge base whose decision-making process follows the "condition-action" rule. The following is a detailed logical description of this knowledge base, which can be directly programmed and implemented by those skilled in the art: 1. IF R < 0.3 THEN; Strategy: Maintain routine monitoring; Action executed: No specific control commands are output. The system only records data, and the model continues to run and be monitored. 2. IF 0.3≤R<0.6 THEN; Strategy: Primary strategy - parameter fine-tuning intervention; Perform the following actions: Send a command to the programmable logic controller (PLC) to increase the power of the associated vibrator by 10%-20% for 10 seconds. At the same time, the speed of the screw feeder will be slightly increased by 5% based on the set value; Delay and Feedback: After the action is completed, the system waits for 15 seconds and then reads the latest risk index R again. new ; Subsequent assessment: IF R new <0.3: Self-healing is considered successful; return to normal monitoring. ELSE: If the initial intervention is deemed ineffective, the risk level is automatically escalated, triggering the next level of strategy; 3. IF 0.6≤R<0.8 THEN Strategy: Secondary strategy - proactive congestion clearing operation; Perform the following actions: Immediately send a reverse operation command to the PLC, for example, control the screw feeder to rotate in the reverse direction 3 to 5 times; Trigger the high-pressure pulse purging solenoid valve installed at the prone blockage point to perform a powerful purging for 0.5 seconds; Delay and Feedback: After all actions are completed, the system waits for 20 seconds and monitors R. new ; Subsequent assessment: IF R new <0.6: The blockage is deemed cleared effectively, the system status is downgraded, and monitoring continues; ELSE: If the active clearing of congestion fails, the highest level alarm and shutdown strategy will be triggered; 4. IF R≥0.8 THEN Strategy: Three-level strategy - safe shutdown and alarm.

[0047] Perform the following actions: Orderly shutdown: Immediately send an interlock shutdown signal to the production line control system, first stop the upstream feeding equipment, and after a 2-second delay, stop the main motor of the currently faulty equipment; Audible and visual alarm: Activate the on-site audible and visual alarm; Remote alarm: Sends structured alarm information to preset maintenance personnel mobile terminals and central monitoring system. This information includes at least: [alarm time], [equipment number and location], [fault type: blockage], [final risk index: R], [recommended handling measures: "Please check and clear the blockage inside the XX equipment"]. Final output: The output of the self-healing strategy decision model consists of specific control commands that conform to their communication protocols (such as Modbus TCP, OPC UA) sent to the production line programmable logic controller (PLC) or distributed control system (DCS), as well as alarm information data packets sent to the monitoring and data acquisition system (SCADA) and personnel terminals.

[0048] In step 5, the entire chain data and its final result labels for each warning and self-healing event are collected to form a feedback dataset. Based on this dataset, the production line blockage warning model is periodically incrementally trained and optimized to achieve continuous self-evolution of the model. Specifically: This step is the core of the system's intelligent evolution. It aims to establish an automated and continuously optimized learning loop by collecting feedback data from the system's actual operation, so that the entire early warning and self-healing system can continuously adapt to new production conditions, material characteristics and equipment status, and achieve self-evolution from "excellent" to "superior". 1. Construction of the feedback dataset After each decision-making cycle, the system automatically generates and stores a structured feedback data record. The fields in this record are designed to comprehensively capture the complete context of a single event, specifically including: (1) Event unique identifier and timestamp: used to uniquely identify and trace each early warning-decision event; (2) Model input snapshot: The multimodal feature data (e.g., keyframe images, MFCC features, vibration feature vectors, process parameter sequences, etc.) output from step 2 that triggers this decision. (3) Model output and decision: The blockage risk index R and risk level calculated by the main model; The specific self-healing strategy instructions selected by the self-healing strategy decision model (such as "increase the vibrator power by 20%" or "execute reverse rotation"). (4) System execution log: Details of the control commands sent to the actuator (such as a PLC) and the execution time; (5) Final result labels (key): These are the supervision signals for model optimization, generated automatically or semi-automatically by the system based on subsequent monitoring, and are categorized as follows: Successful early warning and self-healing: Within a period of time after the self-healing strategy is implemented (e.g., 3 minutes), the risk index R continuously decreases and stabilizes at a normal level (e.g., R < 0.3), and no production line downtime occurs. This record is marked as a positive sample; Early warning successful but self-healing failed: The system executed the self-healing strategy, but the risk index did not decrease effectively, and the level 3 strategy (safe shutdown) was eventually triggered. This record is marked as a self-healing failure sample. It is still a valid positive sample for the main model (because of its accurate early warning), but a negative sample for the decision model. Missed report: The system did not issue a warning (R is always below the threshold), but the production line actually experienced a blockage and shutdown. Through manual confirmation or shutdown alarm records afterward, the data in step 2 during this period can be traced back and marked as a negative sample. False Alarm: The system issued a warning and may have executed a self-healing strategy, but subsequent analysis showed that the production line should have been operating normally, and the risk was actually a misjudgment. This record is marked as a negative sample. 2. Model Incremental Training and Optimization Process The system periodically (e.g., weekly or monthly) or after accumulating a sufficient amount of new feedback data (e.g., 1000 data points) initiates the incremental training process of the model; (1) Training data preparation: A balanced dataset is extracted from the historical feedback database, which contains long-term accumulated samples covering various working conditions, and then merged with the most recently collected feedback data. Data is cleaned and augmented to ensure the quality and diversity of the training set; (2) Incremental training of the main model: Training objective: To optimize the parameters of the main model (multimodal fusion network) to make it more effective in distinguishing between "successful warnings" and "false alarms / missed warnings," and to make the output of the risk index R more accurate; Training methods: Use the model input snapshot in the feedback data record as the training input; Using the final result label (considering "successful warning" and "self-healing failure" as high-risk outputs, and "missed report" and "warning" as low-risk outputs) as monitoring signals, a loss function (such as binary cross-entropy loss) is constructed. The transfer learning and fine-tuning strategy is adopted. Instead of training from scratch, the weights of the currently running online model are used as the initial values. The model is trained using a new mixed dataset with a small learning rate (such as 1 / 10 of the original learning rate) to ensure that the model knowledge is updated smoothly and to avoid catastrophic forgetting. (3) Optimization of the self-healing strategy decision model: Optimization objective: Based on the "self-healing failure" samples, optimize the strategy selection logic to improve the success rate of self-healing actions; Optimization methods: Rule base optimization: Analyze "self-healing failure" cases, and have domain experts manually review, revise or expand the "IF-THEN" rules. For example, if it is found that the "reverse rotation" of the secondary strategy is ineffective under certain material characteristics, a new specific unblocking strategy for that material may be added. Strategy performance statistics: The system automatically calculates the historical success rate of each self-healing strategy under different risk levels and operating conditions, and provides operation and maintenance personnel with visual reports as a basis for optimization decision-making rules; 3. Model evaluation and deployment (1) Offline assessment: After the model completes incremental training, its performance is rigorously evaluated on an independent, labeled test set. Evaluation metrics include, but are not limited to, accuracy, precision, recall, F1 score, and area under the AUC-ROC curve. The core principle is: the overall performance of the new model must be no lower than that of the currently running online model before it can enter the deployment process; (2) Shadow mode and smooth transition: Before the formal replacement, the new model can be run in shadow mode, which means that the new model receives real-time data in parallel and makes predictions, but its prediction results are not used for actual control, but only for comparison with the results of the old online model to further verify its stability and superiority. Finally, through the system management interface, the verified new model was smoothly switched to the online activity model to take on actual early warning and self-healing tasks, while the old version model was retained as a backup. 4. Continuous accumulation of knowledge base The entire process in step 5 constitutes a self-evolutionary engine. Every warning, every decision, every success or failure is transformed into valuable experience data by the system and fed back into the AI ​​model. This enables the system to not only perform well in the early stages of installation, but also to become increasingly proficient in the specific production line it serves over time, eventually becoming an intelligent entity with "domain expert experience".

[0049] To verify the effectiveness of this invention, it was implemented and applied in the auxiliary system of a rubber products manufacturing company's internal mixer. This system is responsible for automatically and accurately feeding various powder and granular raw materials such as carbon black and fillers into the internal mixer. The risk of material blockage directly affects the continuity of production and the accuracy of batching.

[0050] I. System Deployment According to step 1 of this invention, sensors are deployed at key nodes: a visual sensor and a radar level gauge are installed on the top of the main carbon black silo; a visual and acoustic sensor is installed at the discharge port of the screw feeder; a current and vibration sensor is installed on the feeder drive motor; a pressure sensor is installed in the pneumatic conveying pipeline; and a temperature and humidity sensor is installed in the raw material storage area. All data are aggregated through an industrial gateway.

[0051] II. Typical Operating Scenarios During a certain batch of production, environmental monitoring showed high humidity. The system detected the following multimodal anomalies in the carbon black screw feeder: Visual data: The discharge port image shows that the carbon black feed flow is uneven in a "pulsating" manner, and the fluidity is reduced.

[0052] Acoustic data: MFCC feature analysis shows that the spectral energy of the material friction sound inside the barrel is abnormally increased in a specific frequency band.

[0053] Vibration data: The root mean square value of the vibration acceleration at the drive end increased by 20%, showing regular impact characteristics.

[0054] Process parameters: The motor operating current rises slowly and continuously, approaching the rated value.

[0055] III. Early Warning and Self-Healing Process Step 3 uses a multimodal fusion model to comprehensively analyze the above characteristics and calculates the blockage risk index R=0.72 in real time, which the system determines to be "medium risk".

[0056] Step 4's self-healing decision engine immediately activates the secondary strategy: Send a command to the PLC to control the screw feeder to perform "short-term reverse rotation 2 revolutions" to mechanically break up the initially formed blockage structure.

[0057] The high-frequency air hammer associated with the feeding unit is triggered to vibrate the cone section of the hopper, assisting in breaking the arch.

[0058] IV. Effect Verification After the self-healing process was completed, the system monitoring data returned to normal: the material output became uniform and stable, abnormal noises and vibrations disappeared, and the motor current returned to a safe range. The risk index R decreased to 0.22 within 30 seconds. This invention successfully completed unmanned intervention and recovery before material blockage worsened, leading to batch interruption and scrapping, thus ensuring the continuous stability and batching accuracy of the mixing process.

[0059] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0060] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A production line blockage early warning and self-healing method integrating the Internet of Things, characterized in that, include: Step 1: Build an IoT sensing layer by deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data, including visual data, acoustic data, vibration data, process parameter data, and environmental data; Step 2: Transmit the multi-source heterogeneous data to the edge computing node, perform preprocessing on each data source, and output a multimodal feature dataset; Step 3: Construct a production line blockage early warning model based on a multimodal fusion deep learning network model, and perform production line blockage early warning monitoring based on the production line blockage early warning model. The production line blockage early warning model includes a visual feature extraction sub-network, an acoustic vibration feature extraction sub-network, and a process parameter time series analysis sub-network. The visual feature extraction sub-network is based on an improved multi-scale convolutional neural network, which is composed of alternating multi-scale Inception modules and residual modules. The acoustic vibration feature extraction sub-network is composed of an improved one-dimensional convolutional neural network, which adopts a large convolutional kernel design, a weakening pool structure design, and a batch normalization design. The process parameter time series analysis sub-network is composed of an improved long short-term memory network, which adopts a bidirectional LSTM design and adds an attention mechanism. Step 4: Based on the monitoring results, execute the corresponding self-healing strategy and generate control commands to send to the production line actuators; Step 5: Collect the full-link data and final result labels of each warning and self-healing event to form a feedback dataset. Based on this dataset, perform periodic incremental training and optimization of the production line blockage warning model to achieve continuous self-evolution of the model.

2. The method according to claim 1, characterized in that, In step 1, an IoT sensing layer is constructed by deploying multiple types of sensors at key equipment nodes on the production line to collect multi-source heterogeneous data, including visual data, acoustic data, vibration data, process parameter data, and environmental data. Specifically: Visual sensors are installed inside the raw material silo and intermediate silo, or in the observation window, the discharge port of the screw feeder and rotary valve, the transparent sight glass section of the pneumatic conveying pipeline, the transfer chute and the three-way distribution valve. Acoustic sensors are installed on the outer wall of the drive motor housing, the outer wall of the screw feeder barrel and bearing housing, and the outer wall of the pneumatic conveying pipeline. Vibration sensors are installed on the drive electrode housing and reducer housing, screw feeder hanger bearing support and barrel support point, cyclone separator and dust collector housing. Install a motor current sensor in the power supply circuit of the drive motor; Install level gauges on the top or side of the silo; Install pipeline pressure sensors in the pipeline sections of the conveying system; A speed sensor is installed on the output shaft of the rotary feeder. Environmental sensors were installed inside the workshop and at the material silo ventilation openings. The system collects multi-source heterogeneous data from various sensors, including visual data, acoustic data, vibration data, process parameter data, and environmental data. The process parameter data includes motor current data, material height data in the silo, gas pressure data inside the pipeline, and real-time rotational speed data of the equipment.

3. The method according to claim 2, characterized in that, In step 2, the visual data is preprocessed, specifically as follows: The visual data is processed sequentially by keyframe extraction, image denoising and enhancement, region of interest cropping, and image normalization.

4. The method according to claim 3, characterized in that, In step 2, the acoustic and vibration data are preprocessed, specifically as follows: The acoustic and vibration data are sequentially sliced ​​and filtered, and the acoustic Mel frequency cepstral coefficient features and the time and frequency domain features of the vibration are extracted to generate a comprehensive vibration feature vector.

5. The method according to claim 4, characterized in that, In step 2, the process parameter data and environmental data are preprocessed, specifically as follows: The process parameter data and environmental data were sequentially processed by time alignment, outlier removal, and data standardization.

6. The method according to claim 5, characterized in that, In step 3, a production line blockage early warning model is constructed based on a multimodal fusion deep learning network model. This model is then used for production line blockage early warning monitoring. Specifically: The production line blockage early warning model also includes a multimodal fusion and decision layer.

7. The method according to claim 6, characterized in that, In step 4, based on the monitoring results, the corresponding self-healing strategy is executed, and control commands are generated and sent to the production line actuators, specifically: Build a layered strategy generation library; Based on the logic pre-stored in the hierarchical strategy generation library, the corresponding self-healing strategy is executed according to the monitoring results; The decision logic for the layered strategy generation library is as follows: If the monitoring results reach the first-level threshold, the first-level strategy is executed, and instructions are sent to the production line control system to adjust the process parameters of the relevant equipment. If the monitoring result reaches the second-level threshold, the second-level strategy is executed, and a command is sent to the production line control system to execute the active unblocking operation sequence. If the monitoring results reach the third-level threshold, the third-level strategy will be executed, sending an orderly shutdown command to the production line control system and an alarm message to the maintenance personnel.

Images