Heat sealing quality detection method and device based on unsupervised learning

Abstract

The invention relates to the technical field of image recognition, in particular to a heat sealing quality detection method and device based on unsupervised learning. The method comprises the following steps: acquiring a real-time temperature matrix of a heat sealing area of industrial heat sealing equipment; and detecting the real-time temperature matrix through a pre-trained time sequence model to judge whether abnormal pixel points exist or not, and if yes, performing clustering and morphological processing to judge whether heat sealing abnormality exists or not. According to the invention, by using the pre-trained time sequence model, an end-to-end automatic detection scheme from direct acquisition of the original temperature data to final judgment of the heat sealing abnormity is provided. During detection, the pre-training model can perform rapid and automatic abnormal pixel screening on an input real-time temperature matrix, and complex feature engineering or manual intervention is not needed. While the generalization ability and accuracy of detection are improved, the automation degree and deployment efficiency of the whole detection process are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to the technical field of industrial automation detection and intelligent manufacturing, and especially to a heat sealing quality detection method and device based on unsupervised learning. BACKGROUND

[0002] Industrial heat sealing equipment, such as vertical form fill sealing (VFFS) and heat sealing process of batteries, as the core process to ensure the sealing, barrier and shelf life of the product, its quality directly affects the safety of the final product. Common defects include material inclusion (particulate matter, powder into the sealing area), wrinkles, uneven pressure, and insufficient temperature caused by aging of the heating tube.

[0003] However, the existing detection methods have significant limitations. On the one hand, many packaging films have the characteristics of semi-transparency, high reflectivity or single color, and it is difficult to form clear contrast under visible light; in addition, visible light cannot penetrate the surface to detect the heat sealing state inside the film. Most existing infrared solutions directly analyze the "absolute temperature". However, the industrial production line is a dynamic balance process, and the preheating of the equipment, the change of the workshop airflow and the day-night alternation of the environmental temperature will cause the baseline of the infrared pixels to drift constantly. Directly using absolute temperature threshold judgment will result in a very high false positive rate. In order to pursue accuracy, some solutions try to introduce deep learning (such as YOLO series) or autoencoder (VAE) for anomaly recognition. However, these models usually require huge video memory and computing resources, and on the embedded controller (such as ARM platform) commonly used in industrial heat sealing equipment, the inference delay is often more than 200ms, which cannot meet the real-time requirements of high-speed production lines with 60-120 packages per minute. In addition, it is difficult to collect negative samples covering all defect types in the industrial field, and traditional supervised learning models have very weak generalization ability when facing unknown anomalies. SUMMARY

[0004] In order to at least partially solve the technical problems in the related art, the present application provides a heat sealing and sealing anomaly detection method and system for industrial heat sealing equipment.

[0005] In one aspect, the present application provides a heat sealing and sealing anomaly detection method for industrial heat sealing equipment, comprising the following steps: A heat sealing quality detection method based on unsupervised learning, comprising the following steps: Collecting a real-time temperature matrix of a heat sealing area of an industrial heat sealing equipment, the real-time temperature matrix comprising temperature values of a plurality of pixel points of the heat sealing area; detecting the real-time temperature matrix by a pre-trained dynamic temporal model to determine whether there is an abnormal pixel point; wherein the dynamic temporal model is trained on a temperature matrix set containing only qualified sample by unsupervised learning; the pre-trained dynamic temporal model includes a convolutional autoencoder model, wherein the step of detecting the real-time temperature matrix by the pre-trained convolutional autoencoder model includes: inputting the real-time temperature matrix into the convolutional autoencoder model to generate a reconstructed temperature matrix corresponding to the real-time temperature matrix; calculating the reconstruction error between the real-time temperature matrix and the reconstructed temperature matrix to generate an error matrix; comparing the error value in the error matrix with a preset error threshold value, if the error value is greater than the error threshold value, the pixel point at the corresponding position is determined as an abnormal pixel point; If there is an abnormal pixel point, the abnormal pixel point is clustered and morphologically processed to determine whether there is a heat sealing abnormality; wherein the clustering and morphological processing includes using a 3x3 structure element closed operation to connect fragmented abnormal pixel points based on a layer space clustering algorithm, and using connected component analysis to calculate the area and shape of the abnormal region.

[0006] By using the above technical solution, the dynamic temporal model is trained by only using qualified samples in an unsupervised learning manner, which solves the problem that all defect types of negative samples cannot be collected in an industrial field, so that the model can learn the temperature distribution rule in a normal heat sealing state, thereby effectively identifying known and unknown heat sealing abnormalities, and improving the generalization ability and robustness of the detection model. At the same time, by clustering and morphologically processing the abnormal pixel points, the scattered abnormal points can be connected into meaningful defect regions, effectively reducing the false positives caused by single pixel point noise, and improving the accuracy of defect determination. In particular, the real-time temperature matrix is reconstructed by using the convolutional autoencoder model, and the deviation of the current heat sealing state from the normal state is quantified by calculating the reconstruction error, thereby realizing an efficient and pixel-level abnormality detection mechanism. This method can accurately locate the pixel point position where the abnormality occurs, providing accurate basis for subsequent defect analysis and positioning, and the model structure is relatively lightweight and easy to realize real-time detection on an embedded platform with limited computing power.

[0007] Optionally, the real-time temperature matrix is detected by a pre-trained dynamic temporal model to determine whether there is an abnormal pixel point, wherein the dynamic temporal model is configured to learn the heat distribution manifold of qualified samples and generate a standard reconstructed image corresponding to the input.

[0008] By adopting the technical scheme, the dynamic time sequence model can deeply learn the internal law and complex structure of the qualified sample on the temperature distribution, instead of only relying on the absolute temperature value. By comparing the real-time temperature matrix with the standard image reconstructed by the model, the slight change deviating from the normal mode can be accurately identified, so that the influence of baseline drift caused by environmental temperature change, equipment preheating and other factors can be effectively overcome, and the stability and sensitivity of detection are significantly improved.

[0009] Optionally, the dynamic time sequence model further comprises a generative adversarial network (GAN) model, the generative adversarial network model comprising a generator and a discriminator, wherein the step of detecting the real-time temperature matrix by the pre-trained dynamic time sequence model comprises: inputting the real-time temperature matrix into the discriminator, outputting a authenticity score by the discriminator, and comparing the authenticity score with a preset confidence threshold to determine whether there is an abnormal pixel point; or generating a reference normal temperature matrix by the generator, calculating a similarity difference value between the reference normal temperature matrix and the real-time temperature matrix, and comparing the similarity difference value with a preset difference threshold to determine whether there is an abnormal pixel point.

[0010] By introducing the generative adversarial network model, the powerful data distribution learning ability and discrimination ability of the generative adversarial network model are utilized to provide two high-precision implementation paths for the heat sealing quality detection: on the one hand, the discriminator can be directly used to evaluate the authenticity of the real-time temperature matrix to quickly determine whether it conforms to the normal sample distribution; on the other hand, the reference normal sample generated by the generator can be compared to quantize the difference more finely, and both ways can effectively improve the detection ability of weak abnormalities or new unknown defects.

[0011] Optionally, the step of collecting the real-time temperature matrix of the heat sealing area of the industrial heat sealing equipment further comprises: collecting second modality data aligned with the real-time temperature matrix in space and time, and performing feature-level fusion or data-level splicing on the real-time temperature matrix and the second modality data to generate multi-modality fusion feature data.

[0012] By collecting and fusing the second modality data aligned with the temperature matrix in space and time, the multi-modality fusion feature data is constructed, the deficiency of single infrared thermal imaging information dimension is made up, the state of the heat sealing area can be comprehensively analyzed from multiple physical dimensions (such as internal thermal state, external morphology, contact pressure, etc.), and thus the recognition accuracy and reliability of complex defects (such as clamped materials wrapped by transparent film, uneven pressure, etc.) are improved.

[0013] Optionally, the second modality data includes visible light images, pressure distribution maps, or ultrasonic data.

[0014] The above technical solution specifies the specific form of visible light images, pressure distribution maps, or ultrasonic data as second modality data, provides a specific and effective technical path for multi-modal fusion, enables the detection system to selectively fuse the most relevant data sources according to the characteristics of different heat sealing processes and defect types, and further enhances the relevance and effectiveness of the detection scheme.

[0015] Optionally, the dynamic time series model includes a multi-channel neural network model, and the step of detecting the real-time temperature matrix by the pre-trained dynamic time series model to determine whether there is an abnormal pixel point further includes: inputting the multi-modal fusion feature data into the multi-channel neural network model to analyze the internal state and external representation of the heat sealing area to determine whether there is an abnormal pixel point.

[0016] The above technical solution uses a multi-channel neural network model to process multi-modal fusion feature data, which can automatically learn and analyze the complex correlation between the internal state (such as temperature, ultrasonic signal) and external representation (such as visible light image) of the heat sealing area, enabling the model to make judgments based on more comprehensive information, thereby significantly improving the detection performance of heat sealing defects that are not obvious in a single modality.

[0017] Optionally, the heat sealing quality detection method based on unsupervised learning further includes: Extracting morphological feature information or texture feature information of the abnormal area, and dividing the abnormal area into predefined defect types according to the morphological feature information or the texture feature information.

[0018] The above technical solution further extracts the morphological or texture features of the abnormal area after detection, and automatically classifies it into predefined defect types (such as material clamping, wrinkles, etc.), realizing the classification from "defect or not" to "what kind of defect", and providing more specific and valuable information for subsequent quality tracing, process optimization, and automatic waste rejection.

[0019] Optionally, the heat sealing quality detection based on unsupervised learning further includes: Generating a control signal based on the location, size, or type of the abnormal area, the control signal being used to control the industrial heat sealing equipment to adjust operation parameters or reject defective products.

[0020] Adopting the technical scheme, based on the analysis result of the position, size or type of the abnormal area, a control signal is automatically generated to adjust the operation parameter of the industrial heat sealing equipment or to reject defective products, and a closed-loop system from real-time detection to feedback control is constructed; the system can realize dynamic and intelligent adjustment of the production process, help to correct the production deviation in time, reduce the generation of continuous defective products, and thus improve the automation level of the production process and the overall good product rate.

[0021] In another aspect, the application also provides a heat sealing quality detection device based on unsupervised learning, which adopts the following technical scheme.

[0022] A heat sealing quality detection device based on unsupervised learning comprises: An infrared temperature measurement module is configured to collect a temperature matrix of a heat sealing area of an industrial heat sealing equipment. A processing module is connected to the infrared temperature measurement module and is configured to detect the real-time temperature matrix by a pre-trained dynamic time sequence model to determine whether there is an abnormal pixel point, and if there is an abnormal pixel point, to perform clustering and morphological processing on the abnormal pixel point to determine whether there is a heat sealing and sealing abnormality; wherein the dynamic time sequence model is trained on a temperature matrix set containing only qualified product samples by an unsupervised learning method. An output module is connected to the processing module and is configured to output a control signal to control the industrial heat sealing equipment to adjust the operation parameter or reject defective products if there is a heat sealing and sealing abnormality.

[0023] Adopting the technical scheme, through the cooperative work of the infrared temperature measurement module, the processing module and the output module, a complete and automatic heat sealing and sealing abnormality detection and control device is formed; the device can automatically collect data and perform real-time analysis in response to the working rhythm of the industrial heat sealing equipment, and can output a control signal in time to realize equipment linkage when an abnormality is detected, realizing closed-loop management from data collection, analysis and processing to control feedback, and having the advantages of high integration, strong automation, fast response, easy deployment and application on industrial production lines.

[0024] In summary, the application has at least one of the following beneficial technical effects: 1. By utilizing a pre-trained dynamic temporal model, an end-to-end automated detection scheme is provided for directly finalizing the judgment of heat sealing abnormalities from raw temperature data acquisition. The core of the scheme lies in that the dynamic temporal model, by performing unsupervised learning only on qualified sample, has pre-mastered the deep law of normal heat sealing state, thereby solving the problem that defect samples are difficult to exhaustively collect in industrial field. In detection, the pre-trained model can quickly and automatically screen abnormal pixel points for the input real-time temperature matrix without complex feature engineering or manual intervention. Subsequently, combined with clustering and morphological processing, the pixel-level preliminary screening result is converted into a region-level, more practically meaningful defect judgment, effectively integrating scattered information and suppressing noise interference, ultimately improving the detection generalization ability and accuracy while greatly improving the automation degree and deployment efficiency of the entire detection process.

[0025] 2. By collecting and fusing the second modality data aligned with the temperature matrix in space and time, multi-modal fusion feature data is constructed, which makes up for the deficiency of single infrared thermal imaging information dimension, and can comprehensively analyze the state of the heat sealing area from multiple physical dimensions (such as internal thermal state, external morphology, contact pressure, etc.), thereby improving the recognition accuracy and reliability of complex defects (such as sandwiched material wrapped by transparent film, uneven pressure, etc.).

[0026] 3. By using a multi-channel neural network model to process multi-modal fusion feature data, the complex correlation between internal state (such as temperature, ultrasonic signal) and external representation (such as visible light image) of the heat sealing area can be automatically learned and analyzed, so that the model can make judgments based on more comprehensive information, thereby significantly improving the detection performance of those heat sealing defects whose features are not obvious in a single modality. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 A flowchart illustrating a heat sealing abnormality detection method for an industrial heat sealing device is shown.

[0028] Figure 2 A flowchart illustrating a heat sealing quality detection method based on unsupervised learning is shown.

[0029] Figure 3 A module diagram of a heat sealing quality detection device based on unsupervised learning is shown.

[0030] BRIEF DESCRIPTION OF DRAWINGS DETAILED DESCRIPTION

[0031] The following will be described in detail Figure 1 - Figure 3 The present application is further described in detail.

[0032] The embodiment of the present application discloses a heat sealing abnormality detection method and system for industrial heat sealing equipment. By analyzing the relative change value of the infrared temperature matrix instead of the absolute temperature, the influence of temperature drift is effectively overcome. Only a small amount of positive samples are needed for unsupervised learning to detect known or unknown abnormalities. Moreover, the embodiment of the present application adopts a multi-level technical architecture from basic feature extraction to efficient statistical modeling, and provides a configurable solution to adapt to different application scenarios.

[0033] Figure 1 A flowchart of a heat sealing abnormality detection method for industrial heat sealing equipment is shown. Referring to Figure 1 , the method comprises the following steps: S1, collecting a temperature matrix of a heat sealing area of an industrial heat sealing equipment.

[0034] S2, extracting time-space feature information with baseline drift invariance from the temperature matrix.

[0035] S3, training a dynamic time series model based on the time-space feature information to learn the distribution rule of the time-space feature information of qualified samples.

[0036] S4, using the dynamic time series model to detect a real-time temperature matrix, evaluating the degree of deviation of the time-space feature information of the to-be-tested sample from the distribution rule, and judging whether there is an abnormal pixel point.

[0037] S5, if there is an abnormal pixel point, clustering and morphological processing are performed on the abnormal pixel point to determine whether there is a heat sealing abnormality.

[0038] In step S1, a temperature matrix of a heat sealing area of an industrial heat sealing device is collected. The industrial heat sealing device includes, for example, a packaging machine and a battery, etc. In the heat sealing process of the industrial heat sealing device, the temperature matrix of the heat sealing area is collected in response to a trigger signal. The trigger signal can be, for example, an input / output (IO) signal outside the system or a feature image-based trigger signal inside the system. The temperature matrix of the heat sealing area can be collected by using, for example, an infrared camera with a resolution of 256*192. Each shot of the infrared camera can capture an array composed of 256 rows and 192 columns of temperature measurement points. Each point in the array (which can be understood as a “thermal imaging pixel”) corresponds to an accurate temperature reading of a very small position on the heat sealing area. At this time, the temperature matrix is a two-dimensional array of 256*192, which contains 256*192 independent temperature values. A plurality of frames of normal sample data (for example, 77-1000 frames) can be continuously collected for initial training. In some embodiments, the temperature values can be converted into an integer format of a preset scaling ratio (for example, multiplied by 10 and stored in int16 format, maintaining an accuracy of 0.1°C) to improve calculation efficiency. In addition, a dynamic baseline compensation technique can also be used to eliminate the influence of temperature drift. The temperature drift is converted from additive noise to multiplicative scaling factor by calculating the temperature gradient through spatial difference of 16 pixels. The system can cache the temperature matrix of the last N periods to form a sequence with a depth of N for model input.

[0039] In step S2, spatiotemporal feature information is extracted from the temperature matrix. The spatiotemporal feature information is spatiotemporal feature information extracted from the temperature matrix for representing the temperature relationship between at least one pixel point and other pixel points in its neighborhood, or spatial structure spatiotemporal feature information that is invariant to the overall translation or scaling of the temperature matrix.

[0040] In some embodiments, the Sobel operator can be used to calculate the gradient amplitude and direction of the temperature matrix, and then generate an edge feature map. This method has the advantages of simplicity and speed, but is sensitive to noise. In other embodiments, the PatchCore feature extraction method can be used to divide the temperature matrix into a plurality of local blocks (for example, 8*8), extract the statistical features (such as mean and variance) of each block, and construct a feature memory library in this way. In the real-time detection process, the distance between the real-time feature and the nearest neighbor in the memory library is calculated. If the distance is too large, it is determined to be abnormal. Specifically, the distance can be calculated by using the Euclidean distance or the Mahalanobis distance. Taking the Euclidean distance as an example, let the real-time feature vector be x, and the kth feature vector in the memory library be m k The calculation formula is: where d(x, m k ) represents the distance between the real-time feature vector x and the kth feature vector m kThe Euclidean distance between them, where D represents the feature dimension of the feature space, and x j m represents the component value of the real-time feature vector in the j-th dimension. kj Let d represent the component value of the k-th eigenvector in the memory database along the j-th dimension. Select the eigenvector in the memory database that minimizes d as its nearest neighbor, and then set this minimum d... min The distance is compared with a preset threshold; if the distance is too large, it is considered an anomaly. In other embodiments, the temperature matrix can be flattened into a vector, and principal component analysis (PCA) can be used to reduce its dimensionality to 50 dimensions, retaining the principal components, and anomaly scoring can be performed based on the reconstruction error. In another preferred embodiment, the difference between each pixel and its neighboring pixels can be directly calculated to form a relative change matrix; this scheme has the highest computational efficiency.

[0041] In other embodiments, the two-dimensional temperature matrix can be transformed to the frequency domain using Fourier transform (FFT) or discrete cosine transform (DCT). A normal heat seal should exhibit a stable and concentrated energy distribution in the frequency domain. Defects such as wrinkles and inclusions can introduce high-frequency noise or alter the spectral structure. In other embodiments, wavelet transform can be used to perform multi-scale analysis of the temperature matrix. Wavelet coefficients can simultaneously reflect the frequency and spatial information of the signal, making them highly sensitive to detecting local, transient abnormal signals (such as minute inclusions).

[0042] In step S3, a dynamic time-series model is trained based on the spatiotemporal feature information. This dynamic time-series model is a statistical model or a machine learning model that can learn the distribution patterns of the spatiotemporal feature information of qualified samples. Specifically, spatiotemporal feature information can be accumulated to form a positive sample dataset (containing spatiotemporal feature information corresponding to temperature matrices across multiple frames), and the dynamic time-series model is trained based on this positive sample dataset. During the accumulation process, an int32 array can be used to store the sum S and the sum of squares array Q to prevent overflow. Furthermore, a historical time-series window can be maintained for each pixel, and the model parameters are updated based on the data within the historical time-series window.

[0043] In some embodiments, multivariate statistical methods (such as Mahalanobis distance) can be used to calculate the deviation of each sample from the positive sample dataset, but this has high computational complexity.

[0044] In another embodiment, a univariate statistical model (mean and variance) can be independently built for each pixel, and the parameters can be dynamically updated using the cumulative array method. This method supports online learning. Specifically, for each pixel (i,j), the mean μ{i,j} and variance σ are calculated from the positive sample set. 2 {i,j}. The sum array S and the sum of squares array Q are maintained using the cumulative array method, and μ and σ are updated online to avoid redundant calculations, expressed as follows: μ {i,j} =S {i,j} / N,σ {i,j} 2 =(Q i,j -S i,j 2 / N) / N, where N is the number of samples, S {i,j} Sum represents the sum of temperature values ​​of pixel (i,j) across N samples, and Qi,j represents the sum of squares of temperature values ​​of pixel (i,j) across N samples.

[0045] In another embodiment, an equivalent multiple of σ (ESM) model can be used for training, including: calculating skewness and kurtosis for each pixel and determining the distribution type. The mean μ and standard deviation σ of each pixel are calculated from historical data, where the mean μ and standard deviation σ are calculated in the same way as in the aforementioned univariate statistical model. The required percentiles are calculated from the historical data sequence, such as the 2.5th percentile (Q_{2.5%}^{empirical}) and the 97.5th percentile (Q_{97.5%}^{empirical}). To eliminate fluctuations caused by finite samples, quantile smoothing techniques are used to correct the empirical quantiles, expressed as: Q_{p}^{smooth}=α·Q_{p}^{empirical}+(1-α)·Q_{p}^{normal}, where p represents the percentile. Quantile, α represents the smoothing coefficient, Q_{p}^{smooth} represents the smoothed p-quantile, Q_{p}^{empirical} represents the p-quantile calculated based on empirical data, and Q_{p}^{normal} represents the theoretical p-quantile assuming the data follows a normal distribution. The theoretical p-quantile Q_{p}^{normal} can be estimated using extreme value theory, for example, expressed as: Q_{97.5%}^{theory}=μ+2.24σ, Here, Q_{97.5%}^{theory} represents the theoretical quantile of the 97.5% quantile. Then, the ESM value is calculated based on the smoothed quantile, expressed as: ESM = (Q_{97.5%}^{smooth} - Q_{2.5%}^{smooth}) / (2σ). Finally, it can be adapted according to the data distribution type. If the data distribution is determined to be skewed, a two-sided ESM strategy should be adopted, i.e., calculating ESM_{high} for judging upper limit anomalies and ESM_{useful} for judging upper limit anomalies. The ESM_{low} parameter, used to identify abnormalities at the lower limit, more accurately fits asymmetric data distributions, thereby improving detection sensitivity and accuracy. Specifically, in industrial heat sealing processes, due to asymmetric heat conduction in the heating system or differences in environmental heat dissipation, temperature distributions often exhibit left- or right-skewed characteristics. The ESM_{high} parameter sets a dynamic discrimination benchmark for abnormal temperature rises above the mean (such as localized heat accumulation caused by material clamping); the ESM_{low} parameter sets a benchmark for abnormal temperature drops below the mean (such as partial failure or leaks in the heating element).

[0046] In another embodiment, multiple models can be aggregated based on data similarity (e.g., by heating block ID or data distribution), clustering the data into multiple groups, with each group training an independent model. During training, the ability to detect low-probability anomalies is improved by adjusting the σ threshold.

[0047] In step S4, the dynamic time series model is used to detect the real-time temperature matrix, and the dynamic time series model is utilized to evaluate the degree to which the features of the sample to be measured deviate from the distribution law to determine whether there are abnormal pixel points. This model not only learns the normal distribution of features in space but also learns how the feature values change within consecutive periods, such as the cooling curve pattern of a qualified seal. The detection process can adopt a multi-algorithm parallel fusion approach.

[0048] In some embodiments, ESM adaptive detection is performed, including: loading pre-computed ESM model parameters (μ, σ, ESM, etc.); calculating an adaptive threshold, expressed as the following formula threshold = ESM × adapt_factor, where ESM represents the ESM value and adapt_factor represents a preset adaptive sensitivity coefficient. Calculate the standard score and perform a standard score Z-score determination, expressed as z = |G - μ| / σ, where z represents the standard score and G represents the feature value (such as the temperature value or its relative change feature value) of a single pixel point collected in real time; if z > threshold and the connected component of the abnormal pixel after closing operation ≥ 10, it is determined as abnormal. The closing operation here is a basic operation in mathematical morphology, defined as first performing a dilation operation on the image and then an erosion operation. The role of the closing operation is to fill the small holes inside the object, connect adjacent broken regions, and at the same time keep the original area and shape of the object basically unchanged. The connected component refers to a set of pixel points in the abnormal pixel image that are adjacent in position and have abnormal pixel values. By setting the connected component area threshold, scattered and isolated noise points (such as the number of connected component pixels is usually less than 10) can be effectively filtered out, while the abnormal pixel clusters that are truly aggregative and represent heat-sealing defects are retained, thereby improving the detection accuracy and reducing the false alarm rate. Among them, the Z-score relative to the historical time series distribution of the real-time feature value can be calculated for determination.

[0049] In some embodiments, kernel density estimation (KDE) detection is performed, including: independently maintaining a temperature history sequence for each pixel; using the Silverman rule to select the bandwidth h: h = 1.06 × σ × n^(-0.2), where n represents the number of samples in the historical sequence used for kernel density estimation; calculating the kernel density for abnormal determination, and the kernel density calculation is expressed as the following formula: , where K() represents the kernel function, x represents the real-time value, and x i represents the historical sample. The determination condition is expressed as the following formula: f(x) < threshold. Among them, the occurrence probability of the real-time temperature value can be calculated for each pixel point based on the temperature history sequence, and if the probability is lower than the threshold, it is determined as abnormal. Preferably, the ESM and KDE algorithms are executed simultaneously to improve detection efficiency. A dynamic fusion strategy based on confidence level and scene can be employed using an intelligent fusion processor. The aggregation process supports various aggregation strategies, such as logical AND, logical OR, and weighted summation. This intelligent fusion processor can be, for example, a trained machine learning model that takes the outputs of ESM and KDE, along with features representing the current working scene, as input features, and outputs the final fusion result after learning. The fusion result is the final anomaly probability score P for the pixel being tested. final Or, a binary anomaly label (0 / 1). Its fusion logic is specifically implemented as follows: weighted summation mode: P final =w1·P ESM +w2·P KDE , where P ESM P is the normalized anomaly probability calculated based on the ESM model (by mapping the Z-score to the 0-1 interval using the Sigmoid function). KDE The anomaly probability (i.e., 1 - f(x) or confidence level converted from probability density) is calculated based on the KDE model. w1 and w2 are weighting coefficients, and w1 + w2 = 1. The weights w1 and w2 are dynamically adjusted according to the characteristics of the current scenario. For example, in the non-steady-state scenario when the production line has just started, w2 (KDE weight) is increased to utilize its stronger distribution capture capability; in the steady-state scenario of high-speed operation, w1 (ESM weight) is increased to ensure real-time performance. When the confidence index shows that the current temperature fluctuates drastically, if the ESM and KDE results conflict, the KDE judgment result is adopted first. Here, the confidence index refers to the stability measure of the current temperature data, which can be obtained by calculating the variance Var of the temperature data within the recent time window. If Va is greater than the preset jitter threshold, the temperature fluctuation is judged to be drastic, and the confidence index is reduced. Through this dynamic fusion, the system can combine the efficient statistical characteristics of ESM with the accurate description of complex distributions by KDE to output a more robust judgment result than a single algorithm, effectively solving the problem that a single algorithm is susceptible to noise interference under complex working conditions. For slow temperature drift or periodic fluctuations (hole-shaft misalignment) caused by mechanical vibration of the heating tube, the dynamic time series model can issue an early and more stable warning by comparing the real-time sequence with the historical normal sequence pattern, while the static spatial model may fail due to baseline drift.

[0050] In step S5, if abnormal pixels are found, clustering and morphological processing are performed on these pixels to determine if heat sealing anomalies exist. Layered spatial clustering algorithms can be used, employing 3×3 structuring element closing operations to connect fragmented abnormal pixels. The minimum cluster size can be reduced to 10 pixels, improving the detection capability of subtle defects by 40%. Connected component analysis can also be used to calculate the area, shape, and other features of the abnormal region. After detecting an abnormal region, its morphological or texture features can be extracted, and the abnormal region can be classified into one of the predefined defect types (e.g., material clamping, wrinkles, incomplete sealing, etc.) based on these features. Based on the location, size, or type of the detected abnormal region, one or more control signals can be generated. These control signals can be sent to the industrial heat sealing equipment via GPIO or a communication interface (e.g., Ethernet) to automatically adjust the operating parameters of the industrial heat sealing equipment (e.g., heat sealing temperature, heat sealing pressure, heat sealing time, film conveying speed, etc.).

[0051] The following details the implementation examples of this application with specific parameters and model settings. In the basic implementation example of this application, temperature data is integerized (temperature value multiplied by 10, stored as int16 type), and the cumulative array uses an int32 array to store S and Q to prevent overflow. The number of training samples is 77 frames. An ESM model is adopted, using quantile smoothing and extreme value theory, with α=0.7. The pixel-level ESM threshold is dynamically adjusted, and the global outlier proportion threshold is 1%. Detection time: <12.5ms / frame. In the optimized implementation example of this application, an ESM adaptive detection + pixel-level KDE detection scheme is adopted. The fusion strategy adopts confidence-based intelligent fusion. The KDE parameter is the maximum historical sample size of 100, and the bandwidth method is the Silverman rule. Performance indicators are: total L1 latency <8ms, detection rate >99.5%, and false detection rate <0.08%. Memory usage is ~60KB+ for the ESM model, ~100KB+ for the historical buffer, and ~20KB=~180KB for the fusion processor configuration. In another optimized embodiment of this application, support for multiple heating blocks is provided. An ESM model is independently trained for each heating block of the industrial heat sealing equipment, and the corresponding model is selected based on the heating block ID. Each model is 60KB, and the total storage for the four heating blocks is 240KB. Switching time is <1ms. In another preferred embodiment of this application, fixed-point arithmetic optimization is performed. The temperature value adopts the Q8.8 format, with an 8-bit integer part and an 8-bit fractional part, with a precision of 1 / 256≈0.0039. The temperature conversion process is represented by the following formula: temp_q88 = (temp_original * 256), where multiplying by 256 is equivalent to left-shifting by 8 bits (<<8), converting the floating-point temperature value to a fixed-point integer representation. Here, temp_q88 represents the original floating-point temperature value (e.g., 180.5℃), and temp_q88 represents the converted Q8.8 format fixed-point number. In this format, the total data length is 16 bits, with the high 8 bits representing the integer part and the low 8 bits representing the fractional part. `z_score_q88 = (abs_diff<<8) / sigma_q88`, where `abs_diff<<8` is used to improve precision before division to prevent the loss of decimal parts due to integer division. Here, `abs_diff` represents the absolute difference, `sigma_q88` represents the standard deviation in Q8.8 format, and `z_score_q88` represents the standard score output in Q8.8 format, used to measure the degree to which the current observation deviates from the normal distribution. `threshold_q88 = (esm_q88 * adapt_factor_q88)>>8`, where `>>8` (right shift by 8 bits) is used because multiplying two numbers in Q8.8 format results in Q16.16 format (16 decimal places), requiring a right shift of 8 bits to restore it to Q8.8 format for subsequent comparisons.Where esm_q88 represents the ESM value in Q8.8 format, adapt_factor_q88 represents the adaptive sensitivity factor in Q8.8 format, and threshold_q88 represents the final generated adaptive decision threshold.

[0052] The implementation principle of the heat sealing anomaly detection method for industrial heat sealing equipment in this application embodiment is as follows: by collecting the temperature matrix of the heat sealing area and extracting its spatiotemporal feature information to train and use a dynamic time series model, it can effectively overcome the temperature baseline drift problem caused by factors such as equipment preheating, workshop airflow, and changes in ambient temperature, and avoid the high false alarm rate of traditional methods based on absolute temperature thresholds. At the same time, this scheme only requires data (positive samples) under normal sealing conditions for model training, which solves the problem that it is difficult to collect negative samples covering all defect types in industrial sites, and realizes effective detection of unknown anomalies. Thus, under the premise of ensuring low resource consumption and high real-time performance, it significantly improves the accuracy, robustness and generalization ability of heat sealing anomaly detection in industrial heat sealing equipment.

[0053] This application also discloses a heat seal quality inspection method based on unsupervised learning. Compared with the heat seal anomaly detection method for industrial heat sealing equipment in the above embodiments, this method uses a pre-trained time series model to provide an end-to-end heat seal quality inspection scheme. The scheme will be described in detail below with reference to the accompanying drawings.

[0054] Figure 2 A flowchart illustrating a heat seal quality inspection method based on unsupervised learning, as described in an embodiment of this application, is shown. (Refer to...) Figure 2 The method includes the following steps: S10. Collect the real-time temperature matrix of the heat-sealing area of ​​the industrial heat-sealing equipment, wherein the real-time temperature matrix includes the temperature values ​​of multiple pixels in the heat-sealing area.

[0055] S20. Detect the real-time temperature matrix using a pre-trained dynamic time-series model to determine if there are any abnormal pixels.

[0056] S30. If abnormal pixels exist, cluster and morphological processing are performed on the abnormal pixels to determine whether there is an abnormal heat sealing.

[0057] In step S10, a real-time temperature matrix of the heat-sealing area of ​​the industrial heat-sealing equipment is acquired. The real-time temperature matrix includes the temperature values ​​of multiple pixels in the heat-sealing area. This step is similar to step S1 of the method for detecting abnormal heat-sealing seals in industrial heat-sealing equipment. Furthermore, in a preferred embodiment, second modal data (e.g., visible light images, pressure distribution maps, or ultrasonic data) aligned spatially and temporally with the real-time temperature matrix can also be acquired. The real-time temperature matrix and the second modal data are then fused at the feature level or stitched together at the data level to generate multimodal fusion feature data. This multimodal fusion feature data can be used for subsequent multimodal comprehensive detection and analysis. Taking feature-level fusion as an example, spatiotemporal alignment is required before feature-level fusion. For spatial alignment, the visible light image is transformed to the coordinate system of the infrared temperature matrix using a pre-calibrated homography matrix. For temporal alignment, nearest neighbor matching is performed based on the timestamp of the hardware trigger signal. During the fusion stage, a channel concatenation method is used to merge the temperature matrix feature map (e.g., C1×H×W) and the visible light feature map (e.g., C2×H×W) along the channel dimension, forming a fused feature tensor of (C1+C2)×H×W. This tensor is then input into a multi-channel neural network for joint inference. This fusion method can utilize the high-resolution edge information of the visible light image to assist in the localization of the blurred thermal field boundary of the infrared image.

[0058] In step S20, the real-time temperature matrix is ​​detected using a pre-trained dynamic temporal model to determine whether any abnormal pixels exist. The dynamic temporal model is trained using unsupervised learning on a temperature matrix set containing only qualified product samples. This pre-trained dynamic temporal model can be trained using steps S1-S3 of a heat-sealing anomaly detection method for industrial heat-sealing equipment, or it can employ other models trained using unsupervised learning methods, such as convolutional autoencoders, generative adversarial networks (GANs), and multi-channel neural network models.

[0059] When a convolutional autoencoder model is used, step S20 may include: inputting the real-time temperature matrix into the convolutional autoencoder model to generate a reconstructed temperature matrix corresponding to the real-time temperature matrix; calculating the reconstruction error between the real-time temperature matrix and the reconstructed temperature matrix to generate an error matrix; comparing the error value in the error matrix with a preset error threshold, and if the error value is greater than the error threshold, then the pixel at the corresponding position is determined to be an abnormal pixel. In a specific embodiment of this application, the input to the training process of the convolutional autoencoder model is not a single-frame temperature matrix, but a three-dimensional tensor with dimensions (N, H, W), where N is the length of the time sliding window, and H and W are the height and width of the heat-sealed region. The convolutional autoencoder uses a 3D convolutional layer (Conv3D) as the encoder to simultaneously capture spatial texture features and temporal evolution features; or it uses a CNN+LSTM architecture, first extracting the spatial feature vector of each frame through a 2D-CNN, and then inputting the feature vector sequence within the time window into the LSTM network to learn the dynamic law of temperature distribution cooling over time.

[0060] The generative adversarial network (GAN) model includes a generator and a discriminator. When using a GAN model, step S20 may include: inputting the real-time temperature matrix into the discriminator, having the discriminator output a truth score, and comparing the truth score with a preset confidence threshold to determine whether abnormal pixels exist; or generating a baseline normal temperature matrix through the generator, calculating the similarity difference value between the baseline normal temperature matrix and the real-time temperature matrix, and comparing the similarity difference value with a preset difference threshold to determine whether abnormal pixels exist. When using a multi-channel neural network model, step S20 may further input the multimodal fusion feature data extracted in step S10 into the multi-channel neural network model to analyze the internal state and external characterization of the heat-sealed area to determine whether abnormal pixels exist.

[0061] In step S30, if abnormal pixels exist, clustering and morphological processing are performed on these abnormal pixels to determine if heat sealing anomalies exist. Morphological processing includes first performing a closing operation (dilation followed by erosion), using a 3×3 or 5×5 rectangular kernel as the structuring element to fill the voids inside the abnormal pixels and connect fractured crack features; then performing an opening operation to remove isolated noise points. In the clustering step, a density-based DBSCAN algorithm or a connected component labeling algorithm is used, setting a minimum connected component area threshold (e.g., 10 pixels). Abnormal regions smaller than this threshold are considered random noise and filtered out.

[0062] The implementation principle of the heat-sealing quality inspection method based on unsupervised learning in this application is as follows: By utilizing a pre-trained dynamic temporal model, an end-to-end automated inspection scheme is provided, directly from the acquisition of raw temperature data to the final judgment of heat-sealing anomalies. The core of this scheme lies in the fact that the dynamic temporal model, through unsupervised learning only on qualified samples, pre-masters the deep patterns of normal heat-sealing states, thus solving the problem of exhaustively collecting defect samples in industrial settings. During inspection, this pre-trained model can quickly and automatically screen for abnormal pixels in the input real-time temperature matrix without complex feature engineering or manual intervention. Subsequently, combined with clustering and morphological processing, the pixel-level preliminary screening results are transformed into region-level, more practically meaningful defect judgments, effectively integrating scattered information and suppressing noise interference. Ultimately, while improving the detection generalization ability and accuracy, it greatly enhances the automation level and deployment efficiency of the entire inspection process.

[0063] This application also discloses a heat-sealing quality inspection device based on unsupervised learning.

[0064] Figure 3 This diagram illustrates a module schematic of a heat-sealing quality inspection device based on unsupervised learning, according to an embodiment of this application. The device includes an infrared temperature measurement module 10, a trigger module 20, a processing module 30, and an output module 40. The infrared temperature measurement module 10 can employ a 256x192 resolution infrared camera, with a field of view covering a 100mm wide heat-sealing area, for acquiring the temperature matrix of the heat-sealing area of ​​the industrial heat-sealing equipment. The trigger module 20 is connected to the infrared temperature measurement module 10 and the processing module 30. The trigger module 20 can initiate subsequent anomaly detection processes in response to external I / O signals (e.g., synchronization signals output by the PLC of the industrial heat-sealing equipment) or internal trigger signals (e.g., detecting specific features such as temperature peaks from the temperature matrix). Furthermore, for systems employing an internal triggering mode, the trigger module 20 can be omitted. The processing module 30 is connected to the infrared temperature measurement module 10 and can be an embedded device such as an ARM or FPGA. It is used to detect the real-time temperature matrix through a pre-trained dynamic time-series model to determine whether there are abnormal pixels. The dynamic time-series model is trained on a temperature matrix set containing only qualified product samples through unsupervised learning. The output module 40 is connected to the processing module 30 and is used to output a control signal through the system's I / O interface when an abnormality in heat sealing is detected. This control allows the industrial heat sealing equipment to automatically adjust at least one of its operating parameters (e.g., heat sealing temperature, heat sealing pressure, heat sealing time, film speed, etc.) or reject defective products.

[0065] The implementation principle of the heat sealing anomaly detection system for industrial heat sealing equipment in this application embodiment is as follows: Through the coordinated work of infrared temperature measurement module 10, trigger module 20, processing module 30 and output module 40, a complete and automated heat sealing anomaly detection system is formed. The system can automatically collect data and perform real-time analysis in response to the working cycle of industrial heat sealing equipment. When an anomaly is detected, it can output control signals in a timely manner to control the linkage of industrial heat sealing equipment, realizing closed-loop control from data acquisition, analysis and processing to control signal output. It has the advantages of high integration, strong automation and fast response speed, and is easy to deploy and apply on industrial production lines.

[0066] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A heat seal quality inspection method based on unsupervised learning, characterized in that, Includes the following steps: The real-time temperature matrix of the heat-sealing area of ​​the industrial heat-sealing equipment is collected, and the real-time temperature matrix includes the temperature values ​​of multiple pixels in the heat-sealing area. The real-time temperature matrix is ​​detected using a pre-trained dynamic temporal model to determine whether there are any abnormal pixels. The dynamic temporal model is trained using unsupervised learning on a set of temperature matrices containing only qualified samples. The pre-trained dynamic temporal model includes a convolutional autoencoder model. The step of detecting the real-time temperature matrix using the pre-trained convolutional autoencoder model includes: inputting the real-time temperature matrix into the convolutional autoencoder model to generate a reconstructed temperature matrix corresponding to the real-time temperature matrix; calculating the reconstruction error between the real-time temperature matrix and the reconstructed temperature matrix to generate an error matrix; comparing the error value in the error matrix with a preset error threshold, and if the error value is greater than the error threshold, determining the pixel at the corresponding position as an abnormal pixel. If abnormal pixels are found, clustering and morphological processing are performed on the abnormal pixels to determine whether there is an abnormality in the heat sealing process. The clustering and morphological processing includes using a layer space clustering algorithm, connecting the fragmented abnormal pixels using a 3×3 structuring element closing operation, and using connected component analysis to calculate the area and shape of the abnormal region.

2. The heat seal quality inspection method based on unsupervised learning according to claim 1, characterized in that, The real-time temperature matrix is ​​detected by a pre-trained dynamic temporal model to determine whether there are abnormal pixels. The dynamic temporal model is configured to learn the thermal distribution manifold of qualified samples and generate a standard reconstructed image corresponding to the input.

3. The heat seal quality inspection method based on unsupervised learning according to claim 1, characterized in that, The dynamic time series model further includes a generative adversarial network (GAN) model, which comprises a generator and a discriminator. The step of detecting the real-time temperature matrix using the pre-trained dynamic time series model includes: The real-time temperature matrix is ​​input to the discriminator, which outputs a fidelity score. This fidelity score is then compared to a preset confidence threshold to determine if any abnormal pixels exist. The generator generates a baseline normal temperature matrix, calculates the similarity difference value between the baseline normal temperature matrix and the real-time temperature matrix, and compares the similarity difference value with a preset difference threshold to determine whether there are abnormal pixels.

4. The heat seal quality inspection method based on unsupervised learning according to claim 1, characterized in that, The step of collecting the real-time temperature matrix of the heat-sealing area of ​​the industrial heat-sealing equipment further includes: collecting second modal data that is spatially and temporally aligned with the real-time temperature matrix, and performing feature-level fusion or data-level splicing of the real-time temperature matrix and the second modal data to generate multimodal fusion feature data.

5. The heat seal quality inspection method based on unsupervised learning according to claim 4, characterized in that, The second modal data includes visible light images, pressure distribution maps, or ultrasonic data.

6. The heat seal quality inspection method based on unsupervised learning according to claim 4, characterized in that, The dynamic time series model includes a multi-channel neural network model. The step of detecting the real-time temperature matrix through the pre-trained dynamic time series model to determine whether there are abnormal pixels also includes: inputting the multimodal fusion feature data into the multi-channel neural network model to analyze the internal state and external characteristics of the heat-sealed area to determine whether there are abnormal pixels.

7. The heat seal quality inspection method based on unsupervised learning according to claim 1, characterized in that, Also includes: Extract morphological or texture features from the abnormal region, and classify the abnormal region into a predefined defect type based on the morphological or texture features.

8. The heat seal quality inspection method based on unsupervised learning according to claim 7, characterized in that, Also includes: A control signal is generated based on the location, size, or type of the abnormal area. The control signal is used to control the industrial heat sealing equipment to adjust operating parameters or reject defective products.

9. A heat-sealing quality inspection device based on unsupervised learning, characterized in that, For running the unsupervised learning-based heat seal quality inspection method as described in any one of claims 1-8, the apparatus comprises: Infrared temperature measurement module, used to collect temperature matrix of heat-sealing area of ​​industrial heat-sealing equipment; The processing module, connected to the infrared temperature measurement module, is used to detect the real-time temperature matrix through a pre-trained dynamic time series model to determine whether there are abnormal pixels. If there are abnormal pixels, clustering and morphological processing are performed on the abnormal pixels to determine whether there is an abnormal heat sealing. The dynamic time series model is trained on a temperature matrix set containing only qualified product samples through unsupervised learning. The output module, connected to the processing module, is used to output a control signal in the event of an abnormal heat sealing, so as to control the industrial heat sealing equipment to adjust the operating parameters or reject defective products.

Images