Method and system for detecting heat sealing abnormalities in industrial heat sealing equipment

Abstract

This application relates to a method and apparatus for detecting heat sealing abnormalities in industrial heat sealing equipment. [Solution] The method includes the steps of: collecting a real-time temperature matrix of the heat sealing area of ​​an industrial heat sealing apparatus; detecting the real-time temperature matrix using a pre-trained time-series model to determine whether or not abnormal pixels exist; and, if present, performing clustering and morphological processing to determine whether or not a heat sealing anomaly exists. This application provides an end-to-end automated detection solution from raw temperature data collection to final determination of heat sealing anomalies by utilizing a pre-trained time-series model. During detection, the pre-trained model can quickly and automatically screen for abnormal pixels in the input real-time temperature matrix, without requiring complex feature engineering or artificial intervention.

Description

[Technical Field]

[0001] This application relates to the field of image recognition, and more particularly to the field of industrial automation testing and intelligent manufacturing, and more specifically to a method and system for detecting heat sealing abnormalities in industrial heat sealing equipment. [Background technology]

[0002] For example, the heat sealing process for vertical roll film packaging machines (VFFS) and batteries is a core process that ensures the sealing performance, barrier properties, and extended shelf life of the packaging, and its quality directly impacts the safety of the final product. Common defects include jamming (intrusion of particulate matter or powder into the sealing area), wrinkles, uneven pressure, and insufficient temperature due to heater tube degradation.

[0003] However, conventional detection methods have significant limitations. On the one hand, many packaging films are semi-transparent, highly reflective, or monochromatic, making it difficult to produce clear contrast with visible light, and visible light cannot penetrate the surface to detect the heat-seal state inside the film. Many conventional infrared solutions directly analyze "absolute temperature." However, industrial production lines are dynamic equilibrium processes, and preheating of equipment, changes in airflow within the factory, and diurnal changes in ambient temperature cause continuous drift of the infrared pixel baseline. Using an absolute temperature threshold directly for determination leads to an extremely high false alarm rate. To pursue accuracy, some solutions attempt to introduce deep learning (such as the YOLO series) or autoencoders (VAEs) for anomaly recognition. However, these models typically require enormous video memory and computing resources, and on embedded controllers commonly found in industrial heat-sealing equipment (e.g., ARM platforms), the inference delay often exceeds 2ms, making it impossible to meet the real-time requirements of high-speed production lines of 60-120 units per minute. Furthermore, in industrial settings, it is difficult to collect negative sample data that covers all defect types, and conventional supervised learning models have extremely weak generalization capabilities when faced with unknown anomalies. [Overview of the project] [Problems that the invention aims to solve]

[0004] To at least partially solve the technical problems in related technologies, this application provides a method and system for detecting heat seal abnormalities in industrial heat seal equipment. [Means for solving the problem]

[0005] In one embodiment, this application, A step of collecting a temperature matrix of the heat sealing region of an industrial heat sealing device, wherein the temperature matrix includes the temperature values ​​of a plurality of pixels in the heat sealing region. A step of extracting spatiotemporal feature information having baseline drift invariance from the temperature matrix, A step of training a dynamic time series model based on the aforementioned spatiotemporal feature information to learn the distribution law of the spatiotemporal feature information of qualified samples, The process involves detecting a real-time temperature matrix using the dynamic time-series model and evaluating the degree to which the spatiotemporal characteristic information of the test sample deviates from the distribution law to determine whether or not abnormal pixels exist. The present invention provides a method for detecting heat seal abnormalities in an industrial heat seal apparatus, which includes the step of determining whether or not a heat seal abnormality exists by performing clustering and morphological processing on abnormal pixels if abnormal pixels are present.

[0006] In another embodiment, this application provides a heat seal abnormality detection device for industrial heat seal equipment, employing the following technical solution.

[0007] An infrared thermometer for collecting the temperature matrix of the heat sealing area of ​​an industrial heat sealing device, A processing module connected to the infrared temperature measurement module, which extracts spatiotemporal feature information from the temperature matrix, trains a dynamic time series model based on the spatiotemporal feature information, and uses the dynamic time series model to detect the real-time temperature matrix to determine whether or not abnormal pixels exist, and if abnormal pixels exist, performs clustering and morphological processing on the abnormal pixels to determine whether or not a heat seal abnormality exists, The system includes an output module connected to the processing module, which outputs control signals to the industrial heat sealing device to adjust its operating parameters or eliminate defective products when a heat sealing abnormality is present. [Effects of the Invention]

[0008] As described above, this application has at least one beneficial technical effect as described below.

[0009] 1. By utilizing a pre-trained time-series model, we provide an end-to-end automated detection solution from raw temperature data collection to final determination of heat seal anomalies. The core of this solution is that the dynamic time-series model, through unsupervised learning using only qualified samples, grasps the deep laws of normal heat seal conditions in advance, thus solving the challenge of comprehensively collecting defective samples in industrial settings. During detection, the pre-trained model can rapidly and automatically screen for anomaly pixels on the input real-time temperature matrix, without requiring complex feature engineering or artificial intervention. The subsequent combination with clustering and morphological processing transforms the pixel-level preliminary screening results into more practical defect evaluations at the region level, effectively integrating scattered information, suppressing noise interference, and ultimately improving the generalization ability and accuracy of detection, while significantly increasing the level of automation and deployment efficiency of the entire detection process.

[0010] 2. By collecting and fusing a temperature matrix with spatiotemporally aligned second modality data, multimodal fused feature data is constructed, supplementing the dimensional deficiencies of single infrared thermal image information. This enables comprehensive analysis of the state of the heat-sealed area from multiple physical dimensions (e.g., internal thermal state, external morphology, contact pressure, etc.), thereby improving the accuracy and reliability of identifying complex defects (e.g., pinching or pressure unevenness within a transparent film).

[0011] 3. By processing multimodal fused feature data using a multi-channel neural network model, the complex relationships between the internal state of the heat seal region (e.g., temperature, ultrasonic signals, etc.) and its external representation (e.g., visible light images, etc.) can be automatically learned and analyzed. This allows the model to make decisions based on more comprehensive information, significantly improving the detection performance for heat seal defects where features are not prominent with a single modality.

Brief Description of Drawings

[0012] [Figure 1] FIG. 1 is a flowchart showing a heat seal abnormality detection method for an industrial heat seal device in an embodiment of the present application. [Figure 2] FIG. 2 is a flowchart showing a heat seal abnormality detection method for another industrial heat seal device in an embodiment of the present application. [Figure 3] FIG. 3 is a module block diagram showing a heat seal abnormality detection device for an industrial heat seal device in an embodiment of the present application.

Modes for Carrying Out the Invention

[0013] Hereinafter, the present application will be described in more detail with reference to FIGS. 1 to 3.

[0014] Embodiments of the present application analyze relative change values rather than absolute temperature values of an infrared temperature matrix, effectively overcoming the influence of temperature drift, and detecting known or unknown abnormalities through unsupervised learning using only a small number of positive example samples. A heat seal abnormality detection method and system for an industrial heat seal device are disclosed. In addition, embodiments of the present application adopt a multi-layer technical architecture from basic feature extraction to efficient statistical modeling, and provide a deployable solution for adapting to different application scenarios.

[0015] FIG. 1 is a flowchart showing a heat seal abnormality detection method for an industrial heat seal device in an embodiment of the present application. Referring to FIG. 1, the method includes the following steps.

[0016] S1: Collect a temperature matrix of the heat seal area of the industrial heat seal device.

[0017] S2: Extract spatio-temporal feature information having baseline drift invariance from the temperature matrix.

[0018] S3: Train a dynamic time series model based on spatiotemporal feature information to learn the distribution laws of spatiotemporal feature information for eligible samples.

[0019] S4: A dynamic time-series model is used to detect a real-time temperature matrix, and the degree to which the spatiotemporal characteristic information of the test sample deviates from the distribution law is evaluated to determine whether or not abnormal pixels exist.

[0020] S5: If abnormal pixels are present, clustering and morphological processing are performed on the abnormal pixels to determine whether or not a heat seal anomaly exists.

[0021] In step S1, a temperature matrix of the heat-sealing area of ​​an industrial heat-sealing apparatus is collected. The industrial heat-sealing apparatus includes, for example, a packaging machine and a battery. During the heat-sealing process of the industrial heat-sealing apparatus, the temperature matrix of the heat-sealing area is collected in response to a trigger signal. The trigger signal may be, for example, an external input / output (IO) signal or a trigger signal based on an internal feature image. For example, an infrared camera with a resolution of 256 × 192 can be used to collect the temperature matrix of the heat-sealing area. Each shot taken by the infrared camera can capture an array of 256 rows × 192 columns of temperature measurement points, where each point in the array (which can be understood as a "thermal image pixel") corresponds to an accurate temperature reading at a minute location in the heat-sealing area. In this case, the temperature matrix is ​​represented as a two-dimensional array containing 256 × 192 independent temperature values. For initial training, multiple frames (e.g., 77-10 frames) of positive example sample data can be collected sequentially. In some embodiments, to improve computational efficiency, temperature values ​​can be converted to an integer format with a predetermined scaling ratio (for example, multiplied by 10 and saved in int16 format, maintaining an accuracy of 0.1°C). It is also possible to eliminate the effects of temperature drift using dynamic baseline correction techniques. The temperature gradient is calculated using a 16-pixel span spatial difference, and temperature drift is converted from additive noise to a multiplicative scaling factor. The system can cache the temperature matrix of the most recent N cycles for model input, constructing a sequence of depth N.

[0022] In step S2, spatiotemporal feature information is extracted from the temperature matrix. This spatiotemporal feature information is spatiotemporal feature information extracted from the temperature matrix that represents the temperature relationship between at least one pixel in the matrix and other neighboring pixels, or spatiotemporal feature information of a spatial structure that is invariant to the overall translation and scaling of the temperature matrix.

[0023] In some embodiments, the gradient amplitude and direction of the temperature matrix can be calculated using the Sobel operator to generate an edge feature map. This method has the advantages of being simple and fast, but is relatively sensitive to noise. In some other embodiments, a PatchCore-based feature extraction method can be employed to divide the temperature matrix into multiple local blocks (e.g., 8x8), extract statistical features (mean, variance, etc.) from each block, and construct a feature memory bank. In the real-time detection process, the distance between the real-time feature and its nearest neighbor in the memory bank is calculated, and if the distance is too large, it is determined to be an anomaly. Specifically, Euclidean distance or Mahalanobis distance may be used as the distance calculation method. Taking Euclidean distance as an example, the real-time feature vector is x, and the k-th feature vector in the memory bank is m. k In that case, the calculation formula is as follows:

number

[0024] In several other embodiments, the two-dimensional temperature matrix can be transformed into the frequency domain by the Fourier transform (FFT) or discrete cosine transform (DCT). A normal heat seal should exhibit a stable and concentrated energy distribution in the frequency domain. On the other hand, defects such as wrinkles and pinching introduce high-frequency noise or alter the spectral structure. In several other embodiments, wavelet transforms can also be used to perform multiscale analysis on the temperature matrix. Wavelet coefficients can reflect both the frequency and spatial information of the signal and are highly sensitive to the detection of localized and transient anomaly signals (such as minute pinching).

[0025] In step S3, a dynamic time series model is trained based on the spatiotemporal feature information. This dynamic time series model is either a statistical model or a machine learning model and can learn the distribution laws of spatiotemporal feature information of eligible samples. At this time, the spatiotemporal feature information is accumulated to form a positive example sample dataset (including spatiotemporal feature information corresponding to temperature matrices of multiple frames), and the dynamic time series model can be trained based on this positive example sample dataset. In the accumulation process, to prevent overflow, the sum S and sum of squares array Q can be stored using an int32 array. In addition, a history time series window can be maintained for each pixel, and the model parameters can be updated based on the data in the history time series window.

[0026] In some examples, multivariate statistical methods (such as the Mahalanobis distance) can be used to calculate the deviation between each sample and the positive example sample dataset, but this increases computational complexity.

[0027] In another embodiment, a univariate statistical model (mean and variance) can be constructed independently for each pixel, and the parameters can be dynamically updated using the cumulative sequence method, which supports online learning. Specifically, for each pixel (i,j), the mean μ{i,j} and variance σ are obtained from a positive sample set. 2Calculate {i,j}. Also, maintain the sum array S and the sum-of-squares array Q using the cumulative array method, and avoid repeated calculations by updating μ and σ online. It is represented by the following formula, where μ {i,j} =S {i,j} / N, σ {i,j} 2 =(Q i,j -S i,j 2 / N) / N, where N is the number of samples, S {i,j} represents the sum Sum of the temperature values of pixel (i,j) in N samples, and Qi,j represents the sum of squares of the temperature values of pixel (i,j) in N samples.

[0028] In another embodiment, an Equivalent σ-multiple (ESM) model can be trained, which includes calculating the skewness and kurtosis for each pixel to determine the distribution type. The mean μ and standard deviation σ for each pixel are calculated from the historical data, and the method for calculating the mean μ and standard deviation σ here is consistent with the method for calculating them in the univariate statistical model described above. The required percentiles, e.g., the 2.5th percentile (Q_{2.5%}^{empirical}) and the 97.5th percentile (Q_{97.5%}^{empirical}) are calculated from the historical data sequence. To eliminate variability due to limited samples, the empirical quantiles are corrected using a quantile smoothing method, which is expressed by the following equation: Q_{p}^{smooth}=α·Q_{p}^{empirical}+(1-α)·Q_{p}^{ Q_{p}^{smooth} is the p-quantile after smoothing, Q_{p}^{empirical} is the p-quantile calculated based on empirical data, and Q_{p}^{normal} represents the theoretical p-quantile assuming the data follows a normal distribution. Here, the theoretical p-quantile Q_{p}^{normal} can be estimated by applying extreme value theory, for example, as shown in the following equation. Therefore, Q_{97.5%}^{theory} = μ + 2.24σ, where Q_{97.5%}^{theory} is the theoretical quantile at the theoretical 97.5th percentile. Subsequently, the ESM value is calculated based on the smoothed quantile and is expressed by the following formula: ESM = (Q_{97.5%}^{smooth} - Q{2.5%}^{smooth}) / (2σ). Finally, it can also be fitted according to the type of data distribution. If the data distribution is judged to be biased, a two-sided ESM strategy should be adopted, that is, by calculating ESM_{high} to judge upper limit anomalies and ESM_{low} to judge lower limit anomalies, the asymmetric data distribution can be fitted more accurately, improving the sensitivity and accuracy of detection. Specifically, in industrial heat sealing processes, the temperature distribution is often biased to the left or right due to asymmetric heat conduction of the heating system and differences in ambient heat dissipation.The role of ESM_{high} is to set dynamic criteria for temperature rise anomalies higher than the average value (e.g., localized heat accumulation due to jamming), while the role of ESM_{low} is to set criteria for temperature drop anomalies lower than the average value (e.g., partial failure of heating tubes or failure to achieve sealing).

[0029] In another embodiment, multiple models are aggregated based on data similarity (e.g., by heating block ID or data distribution), the data is clustered into multiple groups, and an independent model is trained for each group. During the training process, the ability to detect low-probability anomalies is improved by adjusting the σ threshold.

[0030] In step S4, the dynamic time series model is used to detect a real-time temperature matrix, and the dynamic time series model is used to evaluate the degree to which the features of the test sample deviate from the distribution law, thereby determining whether or not abnormal pixels exist. The model not only learns the normal spatial distribution of features, but also learns how the feature values ​​change within a series of cycles, for example, the cooling curve pattern of a successful seal, and a multi-algorithm parallel fusion method can be employed in the detection process.

[0031] In some embodiments, ESM adaptive detection is performed, which involves loading pre-calculated ESM model parameters (μ, σ, ESM, etc.) and calculating an adaptive threshold. The adaptive threshold is expressed as follows: threshold = ESM × adapt_factor, where ESM represents the ESM value and adapt_factor represents a pre-set adaptive sensitivity coefficient. A standard score is calculated, and the standard score, the Z score, is determined as z = |G - μ| / σ, where z is the standard score and G represents the feature value of a single pixel collected in real time (e.g., temperature value or its relative change feature value). If z > threshold and the connected component after the closing operation on the abnormal pixel is 10 or more, it is determined to be abnormal. The closing operation here is one of the basic operations in mathematical morphology and is defined as first performing a dilation operation on the image, followed by an erosion operation. The role of the closing operation is to fill in small cavities within an object and connect adjacent fracture regions while essentially preserving the object's original area and shape. Connected components refer to sets of pixels in an abnormal pixel image that are adjacent in position and all have abnormal pixel values. By setting an area threshold for connected components, scattered, isolated noise points (for example, those with fewer than 10 pixels in the connected component) can be effectively filtered out, and abnormal pixel clusters that have true cohesiveness and represent heat seal defects can be retained. This improves detection accuracy and reduces the false alarm rate. In this process, the Z-score of real-time feature values ​​against the historical time-series distribution can be calculated to make the determination.

[0032] In some embodiments, kernel density estimation (KDE) detection is performed, a temperature history sequence is maintained independently for each pixel, a bandwidth h: h = 1.06 × σ × n^(-0.2) is selected using Silverman's law (where n is the number of samples in the history sequence used for kernel density estimation), and the kernel density is calculated to determine anomalies. The kernel density is calculated using the following formula:

number

[0033] Preferably, the ESM algorithm and the KDE algorithm are executed simultaneously to improve the detection efficiency. Here, intelligent fusion can be adopted, and a dynamic fusion strategy based on confidence and scenarios can be adopted. The aggregation process supports multiple aggregation strategies such as logical AND, logical OR, weighted sum, etc. The intelligent fusion is, for example, a trained machine learning model, which takes the outputs of ESM and KDE, as well as the features representing the current driving situation scenario as input features, and outputs the final fusion result after learning. The fusion result is the final abnormal probability score P final of the pixel point to be measured, or a binary abnormal flag (0 / 1). As a specific implementation of its fusion logic, in the weighted sum mode, P final = w1·P ESM + w2·P KDE where P ESM is the abnormal probability normalization value calculated based on the ESM model (the Z-score mapped to the 0-1 interval by the sigmoid function), and P KDEis the anomaly probability (i.e., 1-f(x) or confidence converted from the probability density) calculated based on the KDE model. w1 and w2 are weight coefficients, where w1+w2=1. The weights w1 and w2 are dynamically adjusted according to the current scenario characteristics. For example, in a transient state scenario immediately after the production line starts up, w2 (KDE weight) is increased to take advantage of a higher distribution capture capability, and in a steady state scenario during high-speed operation, w1 (ESM weight) is increased to ensure real-time accuracy. If the confidence index indicates that the current temperature fluctuations are severe, and the results of ESM and KDE contradict each other, the KDE judgment result is given priority. The confidence index here refers to the degree of stability of the current temperature data and is obtained by calculating the variance Var of the temperature data within the most recent fixed time window. If Var is greater than a preset jitter threshold, it is determined that the temperature fluctuations are severe, and in this case, the confidence index decreases. This dynamic fusion allows the system to combine the efficient statistical properties of ESM with the accurate description of complex distributions by KDE, producing more robust decision results than a single algorithm and effectively solving the problem of single algorithms being susceptible to noise interference in complex operating conditions. For gradual temperature drift or periodic fluctuations (hole-axis fitting misalignment) caused by mechanical vibrations of heater tubes, dynamic time-series models can provide earlier and more stable alerts by comparing real-time sequences with historical normal sequence patterns, whereas static spatial models may become obsolete due to baseline drift.

[0034] In step S5, if abnormal pixels are present, clustering and morphological processing are performed on the abnormal pixels to determine whether or not a heat seal abnormality exists. Here, a layer-space clustering algorithm may be used to concatenate the abnormal pixels fragmented by a 3x3 structural element closing operation. By reducing the minimum cluster size to 10 pixels, it is possible to improve the ability to detect minute defects by 40%. In addition, concatenated component analysis can be used to calculate the area, shape, and other characteristics of the abnormal region. After detecting the abnormal region, the morphological or texture characteristics of the abnormal region can be extracted, and based on these characteristics, the abnormal region can be classified into one of the predefined defect types (e.g., jamming, wrinkles, incomplete sealing, etc.). Based on the location, size, or type of the detected abnormal region, one or more control signals can be generated. These control signals can be transmitted to the industrial heat seal device via GPIO or a communication interface (e.g., Ethernet) to automatically adjust the operating parameters of the industrial heat seal device (e.g., heat seal temperature, heat seal pressure, heat seal time, film feed speed, etc.).

[0035] The following describes specific embodiments of this application by combining specific parameters and model settings. In one basic embodiment of this application, temperature data is integerized (temperature values ​​are multiplied by 10 and stored in int16 format), and S and Q are stored in the cumulative array using an int33 array to prevent overflow. The number of training samples is 77 frames. An ESM model is employed, using quantile smoothing and extreme value theory, with α = 0.7. The pixel-level ESM threshold is dynamically adjusted, and the overall anomaly rate threshold is 1%. The detection time is less than 12.5 ms / frame. In one preferred embodiment of this application, a solution combining ESM adaptive detection and pixel-level KDE detection is employed. The fusion strategy employs confidence-based intelligent fusion. The KDE parameters are set to a maximum historical sample count of 1, and Silverman's law is used for the bandwidth method. Performance indicators are a total L1 time of less than 8 ms, a detection rate of over 99.5%, and a false positive rate of less than 0.08%. Memory usage is as follows: ESM model (~60KB+KDE), history buffer (~1KB+), fusion settings (~20KB=~180KB). Another preferred embodiment of this application provides support for multiple heating blocks. An ESM model is trained independently for each heating block of an industrial heat sealing device, and the corresponding model is selected based on the heating block ID. Each model is 60KB, and four heating blocks occupy a total of 240KB of storage. Switching time is less than 1ms. Another preferred embodiment of this application optimizes fixed-point arithmetic. The temperature value adopts the Q8.8 format, with an integer part of 8 bits, a fractional part of 8 bits, and a precision of 1 / 256 ≈ 0.39. The temperature conversion process is represented as temp_q88 = (temp_original * 256), where multiplying by 256 corresponds to an 8-bit left shift (<<8), converting the floating-point temperature value to a fixed-point integer representation. Here, temp_q88 represents the original floating-point temperature value (e.g., 180.5°C), and temp_q88 represents the converted Q8.8 format fixed-point number. In this format, the data length is 16 bits in total, with the upper 8 bits representing the integer part and the lower 8 bits representing the fractional part.z_score_q88 = (abs_diff<<8) / sigma_q88, where abs_diff <<8 ensures precision before division and prevents loss of fractional parts due to integer division. Here, abs_diff represents the absolute difference value, sigma_q88 represents the standard deviation expressed in Q8.8 format, and z_score_q88 represents the standard score output in Q8.8 format, which is used to evaluate how much the current observation deviates from the normal distribution. threshold_q88 = (esm_q88 * adapt_factor_q88)>>8, where >>8 (8-bit right shift) is performed because multiplying two numbers in Q8.8 format results in a result in Q16.16 format (16-bit fractional part), and it is necessary to right shift it by 8 bits back to Q8.8 format to facilitate future comparisons. Here, esm_q88 represents the ESM value expressed in Q8.8 format, adapt_factor_q88 represents the adaptive sensitivity factor expressed in Q8.8 format, and threshold_q88 represents the final adaptive decision threshold.

[0036] Embodiments of this application further disclose a method for detecting heat seal anomalies in industrial heat sealing equipment. Compared with the heat seal anomaly detection method for industrial heat sealing equipment in the previously described embodiments, this method employs a pre-trained time-series model and provides an end-to-end heat seal quality detection solution. The solution will be described in detail below with reference to the drawings.

[0037] Figure 2 is a flowchart showing a method for detecting a heat seal abnormality in another industrial heat seal apparatus according to an embodiment of this application. Referring to Figure 2, the method includes the following steps.

[0038] S10: A step of collecting a real-time temperature matrix of a heat-sealing region of an industrial heat-sealing apparatus, wherein the real-time temperature matrix includes temperature values ​​of a plurality of pixels in the heat-sealing region.

[0039] S20: The real-time temperature matrix is ​​detected using a pre-trained dynamic time series model to determine whether or not abnormal pixels exist.

[0040] S30: If abnormal pixels are present, clustering and morphological processing are performed on the abnormal pixels to determine whether or not a heat seal abnormality exists.

[0041] In step S10, a real-time temperature matrix of the heat sealing region of an industrial heat sealing device is collected, wherein the real-time temperature matrix includes the temperature values ​​of multiple pixels in the heat sealing region. This step is similar to step S1 of the heat sealing anomaly detection method for an industrial heat sealing device. In a preferred embodiment, a second modality data (e.g., visible light image, pressure distribution map, or ultrasonic data) spatiotemporally aligned with the real-time temperature matrix is ​​collected, and multimodal fused feature data can be generated by fusing the real-time temperature matrix and the second modality data at the feature level or by splicing them at the data level. The multimodal fused feature data can be used in future multimodal integrated detection and analysis. Taking feature-level fusing as an example, spatiotemporal alignment is required before fusing. In spatial alignment, a pre-defined homography matrix is ​​used to transform the visible light image into the coordinate system of the infrared temperature matrix, and in temporal alignment, nearest neighbor matching is performed based on the timestamp of the hardware trigger signal. In the fusion stage, a channel concatenation method is employed, merging the temperature matrix feature map (e.g., C1×H×W) and the visible light feature map (e.g., C2×H×W) in the channel dimension to form a fused feature tensor of (C1+C2)×H×W, which is then input into a multichannel neural network for collaborative inference. This fusion method allows for the use of high-resolution edge information from visible light images to assist in fuzzy thermal boundary positioning in infrared images.

[0042] In step S20, the presence or absence of abnormal pixels is determined by detecting the real-time temperature matrix using a pre-trained dynamic time series model. Here, the dynamic time series model is trained using an unsupervised learning method with a temperature matrix set containing only qualified samples. This pre-trained dynamic time series model may be one trained in steps S1 to S3 of the heat seal anomaly detection method for industrial heat seal equipment, or other models trained using an unsupervised learning method may be used, such as a convolutional autoencoder model, a generative adversarial network (GAN) model, and a multi-channel neural network model. When employing a convolutional autoencoder model, step S20 may include the steps of: inputting the real-time temperature matrix into the convolutional autoencoder model to generate a reconstructed temperature matrix corresponding to the real-time temperature matrix; calculating the reconstruction error between the real-time temperature matrix and the reconstructed temperature matrix to generate an error matrix; and comparing the error values ​​in the error matrix with a preset error threshold, and determining that the pixel at the corresponding position is an abnormal pixel if the error value is greater than the error threshold. In a specific embodiment of this application, the input to the training process of the convolutional autoencoder model is not a single-frame temperature matrix, but a three-dimensional tensor of dimensions (N, H, W), where N is the length of the time sliding window, and H and W are the height and width of the heat seal region. The convolutional autoencoder may employ a 3D convolutional layer (Conv3D) as the encoder to simultaneously capture spatial texture features and temporal evolution features, or it may employ a CNN+LSTM architecture, first extracting the spatial feature vector of each frame using a 2D-CNN, and then inputting the feature vector sequence within the time window into the LSTM network to learn the dynamic laws associated with the cooling of the temperature distribution over time.

[0043] The generative adversarial network model comprises a generator and a discriminator. When the generative adversarial network model is adopted, step S20 may include the steps of: inputting the real-time temperature matrix into the discriminator, outputting a truth score from the discriminator, and determining whether or not abnormal pixels exist by comparing the truth score with a preset confidence threshold; or generating a reference normal temperature matrix using the generator, calculating a similarity difference value between the reference normal temperature matrix and the real-time temperature matrix, and determining whether or not abnormal pixels exist by comparing the similarity difference value with a preset difference threshold. When a multi-channel neural network model is used, step S20 may further input the multimodal fused feature data extracted in step S10 into the multi-channel neural network model and determine whether or not abnormal pixels exist by analyzing the internal state and external representation of the heat-sealed region.

[0044] In step S30, if abnormal pixels are present, clustering and morphological processing are performed on the abnormal pixels to determine whether or not a heat seal anomaly exists. Morphological processing first involves performing a closing operation (a process that performs corrosion after expansion), using a 3x3 or 5x5 rectangular kernel as a structural element to fill the void inside the abnormal pixel and connect the fragmented crack features, and then performing an opening operation to remove isolated noise points. In the clustering process, a density-based DBSCAN algorithm or a Connected Component Labeling algorithm is employed, a minimum connected component area threshold (e.g., 10 pixels) is set, and abnormal regions below this threshold are treated as random noise and removed by filtering.

[0045] The embodiments of this application further disclose a heat sealing abnormality detection device for industrial heat sealing equipment.

[0046] Figure 3 is a modular block diagram showing a heat seal anomaly detection device for an industrial heat seal apparatus in an embodiment of this application. The device comprises an infrared temperature measurement module 10, a trigger module 20, a processing module 30, and an output module 40. The infrared temperature measurement module 10 can employ an infrared camera with a resolution of 256 × 192, has a field of view covering a heat seal area with a width of 1 mm, and is used to collect the temperature matrix of the heat seal area of ​​the industrial heat seal apparatus. The trigger module 20 is connected to the infrared temperature measurement module 10 and the processing module 30, and the trigger module 20 activates the subsequent anomaly detection flow in response to an external I / O signal (e.g., a synchronization signal output from the PLC of the industrial heat seal apparatus) or an internal trigger signal (e.g., a specific feature such as a temperature peak detected from the temperature matrix). In systems employing an internal trigger mode, the trigger module 20 may be omitted. The processing module 30 is connected to the infrared temperature measurement module 10 and may employ an embedded device such as an ARM or FPGA. It detects the real-time temperature matrix using a pre-trained dynamic time-series model to determine whether or not abnormal pixels exist. Here, the dynamic time-series model is obtained by training with a temperature matrix set containing only qualified samples using an unsupervised learning method. The output module 40 is connected to the processing module 30 and outputs a control signal via the system's I / O interface to automatically adjust at least one operating parameter (e.g., heat seal temperature, heat seal pressure, heat seal time, film feed speed, etc.) in the industrial heat seal device or to eliminate defective products when a heat seal abnormality is detected.

[0047] All of the above are preferred embodiments of the present application and do not limit the scope of protection of this application; therefore, any equivalent modifications made based on the structure, shape, and principle of this application shall be included within the scope of protection of this application. [Explanation of Symbols]

[0048] 10. Infrared temperature measurement module 20 Trigger Modules 30 Processing Modules 40 Output Modules

Claims

1. A method for detecting heat sealing abnormalities in an industrial heat sealing device, A step of collecting a temperature matrix of the heat sealing region of an industrial heat sealing device, wherein the temperature matrix includes the temperature values ​​of a plurality of pixels in the heat sealing region. A step of extracting spatiotemporal feature information having baseline drift invariance from the temperature matrix, A step of training a dynamic time series model based on the aforementioned spatiotemporal feature information to learn the distribution law of the spatiotemporal feature information of qualified samples, The process involves detecting a real-time temperature matrix using the dynamic time-series model and evaluating the degree to which the spatiotemporal characteristic information of the test sample deviates from the distribution law to determine whether or not abnormal pixels exist. The process includes, if abnormal pixels are present, performing clustering and morphological processing on the abnormal pixels to determine whether or not a heat seal abnormality exists. A method for detecting heat sealing abnormalities in an industrial heat sealing apparatus, characterized by the features described herein.

2. In collecting the temperature matrix of the heat sealing area of ​​the industrial heat sealing apparatus, the temperature values ​​are converted to an integer format with a preset scaling ratio. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 1.

3. The step of extracting spatiotemporal feature information having baseline drift invariance from the temperature matrix is: A process of calculating the magnitude and direction of the gradient of the temperature matrix using the Sobel operator and generating an edge feature map, or, A step of dividing the temperature matrix into multiple local blocks and extracting statistical characteristic information from each local block, or The process involves flattening the temperature matrix into a temperature feature vector and performing dimensionality reduction on the temperature feature vector using the PCA algorithm, or A step of calculating the difference in temperature values ​​between each pixel and adjacent pixels in the temperature matrix and generating a relative change matrix, or, A step of converting the temperature matrix from the spatial domain to the frequency domain using a fast Fourier transform or discrete cosine transform to generate a spectral matrix, and extracting feature information representing the energy distribution or structure from the spectral matrix, or, The process includes: decomposing the temperature matrix into multiple wavelet coefficient subbands covering different scales and directions using a discrete wavelet transform; and extracting feature information representing local spatiotemporal characteristics from the multiple wavelet coefficient subbands. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 1.

4. In the process of training a dynamic time series model based on the spatiotemporal feature information, the spatiotemporal feature information corresponding to the temperature matrix of multiple frames is accumulated to form a positive example sample dataset, and the dynamic time series model is trained based on the positive example sample dataset. Here, a history time series window is maintained for each pixel, and the model parameters are updated based on the data within the history time series window. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 1.

5. The process of training a dynamic time series model based on the aforementioned spatiotemporal feature information is as follows: A step of constructing a univariate statistical model for each of the aforementioned pixels and updating the model parameters using the cumulative sequence method, or The skewness and kurtosis are calculated for each pixel to determine the distribution type, and the ESM value is calculated by applying quantile smoothing technology and extreme value theory for correction. Here, the process involves using two-sided ESM for biased distributions, or The process includes a step of aggregating multiple dynamic time series models based on the similarity of the data. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 4.

6. The step of detecting a real-time temperature matrix using the aforementioned dynamic time series model and determining whether or not abnormal pixels exist is as follows: A process for performing ESM adaptive detection, which involves loading pre-calculated ESM model parameters, calculating an adaptive threshold, and then calculating a Z-score to determine whether or not abnormal pixels exist, and further calculating and determining the Z-score of real-time feature values ​​against the historical time series distribution, or The process involves performing KDE detection, maintaining a temperature history sequence for each pixel, selecting a bandwidth using Silverman's Law, calculating the nuclear density, and determining an anomaly, where, for each pixel, the probability of a real-time temperature value appearing is calculated based on the temperature history sequence, and if the probability is below a threshold, it is determined to be an anomaly, or The process includes performing ESM adaptive detection and KDE detection in parallel, inputting the first stochastic component output from ESM adaptive detection, the second stochastic component output from KDE detection, and scenario feature components representing the current production line speed and ambient temperature into a pre-trained intelligent fusion system, and outputting comprehensive anomaly judgment information. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 5.

7. If abnormal pixels are present, the step of performing clustering and morphological processing on the abnormal pixels to determine whether or not a heat seal anomaly exists includes a step of concatenating the fragmented abnormal pixels by closing operations using 3x3 structural elements based on a layer space clustering algorithm, and calculating the area and shape of the abnormal region using concatenated component analysis. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 6.

8. The process further includes extracting morphological or textured characteristic information of the abnormal region, and classifying the abnormal region into a predefined defect type based on the morphological or textured characteristic information. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 7.

9. The process further includes generating a control signal based on the location, size, or type of the abnormal region, wherein the control signal is used to control the industrial heat sealing apparatus to adjust its operating parameters or to eliminate defective products. A method for detecting a heat seal abnormality in an industrial heat seal apparatus according to feature 8.

10. A heat sealing abnormality detection system for industrial heat sealing equipment, An infrared thermometer for collecting the temperature matrix of the heat sealing area of ​​an industrial heat sealing device, A processing module connected to the infrared temperature measurement module, which extracts spatiotemporal feature information from the temperature matrix, trains a dynamic time series model based on the spatiotemporal feature information, and uses the dynamic time series model to detect the real-time temperature matrix to determine whether or not abnormal pixels exist, and if abnormal pixels exist, performs clustering and morphological processing on the abnormal pixels to determine whether or not a heat seal abnormality exists, The system includes an output module connected to the processing module, which outputs control signals to the industrial heat sealing device to adjust its operating parameters or eliminate defective products when a heat sealing abnormality is present. A heat seal abnormality detection system for industrial heat seal equipment, characterized by the following features.

Images