A multimodal data fusion method for online plastic quality monitoring

Abstract

This invention relates to the field of image processing technology and discloses a multimodal data fusion method for online quality monitoring of plastics. The method includes: simultaneously acquiring a reference optical image of the plastic surface and a temporal infrared thermal distribution image containing the current frame and the previous frame infrared image; calculating a spatial temperature gradient matrix; correcting the geometric affine matrix based on the displacement vector normal component determined by the dense optical flow field, constructing a spatiotemporal compensation mapping matrix to achieve alignment of heterogeneous pixel coordinate systems, and generating a structure-guided tensor; applying morphological opening operations to remove low-frequency background thermal fluctuations from the structure-guided tensor to obtain a pure-state high-frequency thermal gradient matrix; adjusting the diffusion coefficient using the pure-state high-frequency thermal gradient matrix as a constraint, applying anisotropic diffusion filtering to the reference optical image, and extracting defect regions. This invention achieves physical-level decoupling between optical artifacts and real defects, eliminates feature mapping deviations caused by thermal conduction hysteresis, and improves the signal-to-noise ratio of image feature extraction.

Description

Technical Field

[0001] This invention relates to a method for online quality monitoring of plastics using multimodal data fusion, belonging to the field of image processing technology. Background Technology

[0002] Currently, the industry typically uses machine vision technology to monitor product surface quality online. This involves acquiring optical images of the plastic surface to extract high-frequency edge features that reflect defects such as microcracks, bubbles, or impurities. This detection method, based on a single-modal pixel gradient, can provide basic quality judgment criteria under normal operating conditions and is the mainstream method for ensuring yield in automated production lines. However, when the production line is in a high-temperature molten continuous extrusion or high-speed blown film production scenario, existing technologies face structural noise interference. The surface of polymer materials exhibits strong reflective properties during dynamic evolution, and is affected by thermal stress during the cooling and solidification stage, resulting in transient water ripples or wrinkles on the surface. Since optical imaging only records the photon reflection information of the surface, the gray-scale abrupt changes generated by these physical deformations in the image pixel matrix are highly similar to the feature abrupt changes caused by real physical damage.

[0003] To address such pseudo-defect interference, conventional approaches attempt to increase the resolution of image sensors or employ deep learning recognition models. However, practical analysis shows that simply increasing imaging resolution increases the data processing load and cannot change the numerical overlap between optical artifacts and real defects in the grayscale space. Furthermore, classification models trained on large-scale samples lack underlying physical mechanisms and struggle to provide stable physical attribution in complex and ever-changing production environments. They may even miss minute real defects due to excessive smoothing. Analysis reveals that the bottleneck in online monitoring of plastic surfaces lies in the low-rank nature of information from a single optical mode, preventing the system from distinguishing between optical fluctuations and material damage at the pixel processing level. For example, publication number CN1208075... Chinese invention patent application 99A discloses an automatic registration method for dual-light imaging using a shared near-infrared and visible light sensor. This method acquires interlaced images by cyclically switching filters and uses the half-path optical flow of adjacent visible light frames for motion estimation. The analysis shows that this technical solution is based on the premise of linear and continuous pixel motion. However, in the complex working conditions involving phase changes in plastic melting and extrusion, due to the intrinsic thermal conduction hysteresis effect of polymer materials, the instantaneous reflection of photons and the evolution of molecular thermal diffusion exhibit asymmetry in time phase. Linear half-path estimation cannot correct the nonlinear feature shift caused by thermal field evolution, which can easily lead to mask misalignment. This type of method does not consider the low-frequency thermal background interference caused by macroscopic thickness fluctuations of materials, resulting in a deterioration of the signal-to-noise ratio for feature extraction in non-uniform cooling scenarios.

[0004] Therefore, the technical problem to be solved by this invention is how to achieve high-fidelity extraction of real physical defects on plastic surfaces by establishing a correlation and spatiotemporal calibration mechanism for heterogeneous physical field information, starting from the logic of image data processing. Summary of the Invention

[0005] To address the problems in the background art, the technical solution of the present invention is as follows: A method for online quality monitoring of plastics based on multimodal data fusion, comprising the following steps:

[0006] Step S101: Simultaneously acquire a reference optical image of the plastic surface and a temporal infrared thermal distribution image containing the current frame infrared image and the previous frame infrared image.

[0007] Step S102: Calculate the spatial temperature gradient matrix of the current frame infrared image;

[0008] Step S103: Based on the common edge contour of the reference optical image, a structure guidance tensor is generated by applying spatiotemporal phase compensation registration to the spatial temperature gradient matrix. This registration includes: determining the pixel displacement vector based on the dense optical flow field from the previous frame infrared image to the current frame infrared image; solving the basic geometric affine matrix based on the common edge contour; extracting the normal gradient component of the pixel displacement vector in the motion direction of the image sequence and superimposing it as a hysteresis compensation factor into the 2D translation vector of the basic geometric affine matrix to construct a spatiotemporal compensation mapping matrix; applying bilinear interpolation resampling to the spatial temperature gradient matrix based on the spatiotemporal compensation mapping matrix to output a structure guidance tensor aligned with the pixel coordinate system of the reference optical image.

[0009] Step S104: Determine the 2D structural element matrix based on the preset pixel span, which is greater than the maximum equivalent pixel diameter of the micro-defect of the plastic target to be tested; apply the morphological opening operation to the structural guidance tensor using the 2D structural element matrix to output the background thermal fluctuation matrix; subtract the background thermal fluctuation matrix pixel by pixel from the structural guidance tensor to output the pure state high-frequency thermal gradient matrix.

[0010] Step S105: Using the pure-state high-frequency thermal gradient matrix as a constraint term, anisotropic diffusion filtering is applied to the reference optical image. The diffusion coefficient is dynamically adjusted according to the amplitude of the element at the corresponding coordinate position in the pure-state high-frequency thermal gradient matrix to suppress optical high-frequency noise in the reference optical image and preserve edge features.

[0011] Step S106: Apply morphological processing to the filtered reference optical image to extract the defect image region.

[0012] Preferably, the calculation of the basic geometric affine matrix in step S103 includes the following steps: step S1031, extracting the common edge contours representing the physical boundary of the plastic in the reference optical image and the current frame infrared image; step S1032, establishing a feature point mapping relationship based on the geometric distribution characteristics of the common edge contours; step S1033, calculating parameters including rotation, scaling and reference translation attributes based on the feature point mapping relationship to construct the basic geometric affine matrix.

[0013] Preferably, the output of the background thermal fluctuation matrix in step S104 includes the following steps: Step S1041, applying 2D structural element matrix to the structure guiding tensor for erosion processing to remove small-scale temperature change features; Step S1042, applying expansion processing to the eroded result to generate the background thermal fluctuation matrix; wherein, the background thermal fluctuation matrix is ​​used to characterize the low-frequency thermal conduction background caused by macroscopic thickness rheology of plastic with a spatial frequency lower than a preset threshold.

[0014] Preferably, before step S102, the method further includes the following preprocessing steps: step S1011, applying medium filtering to the time-series infrared thermal distribution image to filter out isolated noise pixels generated by the infrared detector; step S1012, applying histogram equalization to the current frame infrared image to enhance the local contrast of the temperature distribution and obtain an enhanced infrared image with improved signal-to-noise ratio for extracting the spatial temperature gradient matrix.

[0015] Preferably, step S106, which involves extracting the defective image region, includes the following steps: step S1061, applying binarization segmentation to the reference optical image based on a preset adaptive threshold to identify suspected defect candidate regions; step S1062, applying a closing operation to connect the broken edges within the suspected defect candidate regions and fill the internal voids; step S1063, statistically analyzing the pixel feature parameters of the connected components, and eliminating non-defective interference regions that do not conform to the size specifications based on preset geometric area and aspect ratio constraints.

[0016] Preferably, the extraction of the normal gradient component in step S103 includes the following steps: step S1034, establishing a directional distribution model of the pixel displacement vector; step S1035, decomposing the pixel displacement vector into a longitudinal component along the plastic conveying motion direction and a transverse component perpendicular to the conveying motion direction; step S1036, determining a weighting coefficient based on the real-time speed of the plastic conveying motion direction and the intrinsic thermal diffusivity of the plastic, and applying the weighting coefficient to apply quantization correction to the longitudinal component.

[0017] Preferably, the anisotropic diffusion filtering in step S105 further includes: step S1051, constructing a multi-scale Gaussian pyramid containing multiple pixel sampling spans; step S1052, applying the pure-state high-frequency thermal gradient matrix of the corresponding scale to the reference optical image for filtering processing in each level of the multi-scale Gaussian pyramid; and step S1053, fusing the image features after filtering at each level to enhance the global stability of the detection algorithm for identifying defects with different physical spans.

[0018] Preferably, step S106 further includes the following post-processing steps: step S1064, applying pixel coordinate remapping to the extracted defect image region; step S1065, calculating the actual physical coordinates of the defect in the plastic transport coordinate system based on the centroid position of the defect image region in the reference optical image; step S1066, outputting an online quality monitoring report containing the actual physical coordinates, defect area parameters, and defect type labels.

[0019] Preferably, the method further includes: step S1071, acquiring the global average brightness of the reference optical image in real time; step S1072, when the fluctuation range of the global average brightness exceeds a preset deviation threshold, adjusting the diffusion coefficient function parameter in the anisotropic diffusion filter through a linear proportional function, and applying dynamic gain adjustment to the reference optical image to counteract the negative interference of drastic changes in external light intensity on the accuracy of defect feature extraction.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] 1. In online quality monitoring of plastics, by using the spatial temperature gradient extracted from infrared thermal distribution images as the guiding boundary for anisotropic diffusion filtering of optical images, a correlation determination mechanism between optical abrupt changes and thermal conductivity heterogeneity is established at the pixel processing level. This enables the optical high-frequency reflection signal, which does not have thermal gradient characteristics, to undergo isotropic smoothing during diffusion iteration. As a result, while preserving the edges of real physical defects, structural noise generated by surface reflection is suppressed, and the signal-to-noise ratio in the image feature extraction process is improved.

[0022] 2. Further combining the morphological background stripping mechanism, the guide tensor is operated by structural elements of a specific scale to filter out the low-frequency thermal flow background formed by macroscopic thickness fluctuations of the material. This ensures that the edge stopping function in the diffusion process only takes effect on the high-frequency thermal spikes generated by microscopic damage, solving the problem of prior information contamination caused by uneven thickness under complex working conditions, and making the image enhancement process have stronger physical attribution accuracy.

[0023] 3. By introducing a spatiotemporal phase compensation mechanism based on heat flow vector, the coordinate offset caused by heat conduction hysteresis is compensated to the affine transformation matrix, correcting the time phase difference between instantaneous photon reflection and molecular thermal diffusion, ensuring that cross-modal pixel registration maintains rigid alignment in a dynamic environment, providing an accurate spatial mapping relationship for subsequent structure-guided filtering, and avoiding mask misalignment caused by asynchronous physical evolution. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the heterogeneous data fusion and filtering monitoring steps of the present invention;

[0025] Figure 2 This is a logic diagram of cross-modal data processing guided by the structure of this invention.

[0026] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0028] A method for online quality monitoring of plastics using multimodal data fusion includes the following steps:

[0029] Step S101: Simultaneously acquire a reference optical image of the plastic surface and a temporal infrared thermal distribution image containing the current frame infrared image and the previous frame infrared image.

[0030] Step S102: Calculate the spatial temperature gradient matrix of the current frame infrared image;

[0031] Step S103: Based on the common edge contour of the reference optical image, a structure guidance tensor is generated by applying spatiotemporal phase compensation registration to the spatial temperature gradient matrix. This registration includes: determining the pixel displacement vector based on the dense optical flow field from the previous frame infrared image to the current frame infrared image; solving the basic geometric affine matrix based on the common edge contour; extracting the normal gradient component of the pixel displacement vector in the motion direction of the image sequence and superimposing it as a hysteresis compensation factor into the 2D translation vector of the basic geometric affine matrix to construct a spatiotemporal compensation mapping matrix; applying bilinear interpolation resampling to the spatial temperature gradient matrix based on the spatiotemporal compensation mapping matrix to output a structure guidance tensor aligned with the pixel coordinate system of the reference optical image.

[0032] Step S104: Determine the 2D structural element matrix based on the preset pixel span, which is greater than the maximum equivalent pixel diameter of the micro-defect of the plastic target to be tested; apply the morphological opening operation to the structural guidance tensor using the 2D structural element matrix to output the background thermal fluctuation matrix; subtract the background thermal fluctuation matrix pixel by pixel from the structural guidance tensor to output the pure state high-frequency thermal gradient matrix.

[0033] Step S105: Using the pure-state high-frequency thermal gradient matrix as a constraint term, anisotropic diffusion filtering is applied to the reference optical image. The diffusion coefficient is dynamically adjusted according to the amplitude of the element at the corresponding coordinate position in the pure-state high-frequency thermal gradient matrix to suppress optical high-frequency noise in the reference optical image and preserve edge features.

[0034] Step S106: Apply morphological processing to the filtered reference optical image to extract the defect image region.

[0035] Preferably, the calculation of the basic geometric affine matrix in step S103 includes the following steps: step S1031, extracting the common edge contours representing the physical boundary of the plastic in the reference optical image and the current frame infrared image; step S1032, establishing a feature point mapping relationship based on the geometric distribution characteristics of the common edge contours; step S1033, calculating parameters including rotation, scaling and reference translation attributes based on the feature point mapping relationship to construct the basic geometric affine matrix.

[0036] Preferably, the output of the background thermal fluctuation matrix in step S104 includes the following steps: Step S1041, applying 2D structural element matrix to the structure guiding tensor for erosion processing to remove small-scale temperature change features; Step S1042, applying expansion processing to the eroded result to generate the background thermal fluctuation matrix; wherein, the background thermal fluctuation matrix is ​​used to characterize the low-frequency thermal conduction background caused by macroscopic thickness rheology of plastic with a spatial frequency lower than a preset threshold.

[0037] Preferably, before step S102, the method further includes the following preprocessing steps: step S1011, applying medium filtering to the time-series infrared thermal distribution image to filter out isolated noise pixels generated by the infrared detector; step S1012, applying histogram equalization to the current frame infrared image to enhance the local contrast of the temperature distribution and obtain an enhanced infrared image with improved signal-to-noise ratio for extracting the spatial temperature gradient matrix.

[0038] Preferably, step S106, which involves extracting the defective image region, includes the following steps: step S1061, applying binarization segmentation to the reference optical image based on a preset adaptive threshold to identify suspected defect candidate regions; step S1062, applying a closing operation to connect the broken edges within the suspected defect candidate regions and fill the internal voids; step S1063, statistically analyzing the pixel feature parameters of the connected components, and eliminating non-defective interference regions that do not conform to the size specifications based on preset geometric area and aspect ratio constraints.

[0039] Preferably, the extraction of the normal gradient component in step S103 includes the following steps: step S1034, establishing a directional distribution model of the pixel displacement vector; step S1035, decomposing the pixel displacement vector into a longitudinal component along the plastic conveying motion direction and a transverse component perpendicular to the conveying motion direction; step S1036, determining a weighting coefficient based on the real-time speed of the plastic conveying motion direction and the intrinsic thermal diffusivity of the plastic, and applying the weighting coefficient to apply quantization correction to the longitudinal component.

[0040] Preferably, the anisotropic diffusion filtering in step S105 further includes: step S1051, constructing a multi-scale Gaussian pyramid containing multiple pixel sampling spans; step S1052, applying the pure-state high-frequency thermal gradient matrix of the corresponding scale to the reference optical image for filtering processing in each level of the multi-scale Gaussian pyramid; and step S1053, fusing the image features after filtering at each level to enhance the global stability of the detection algorithm for identifying defects with different physical spans.

[0041] Preferably, step S106 further includes the following post-processing steps: step S1064, applying pixel coordinate remapping to the extracted defect image region; step S1065, calculating the actual physical coordinates of the defect in the plastic transport coordinate system based on the centroid position of the defect image region in the reference optical image; step S1066, outputting an online quality monitoring report containing the actual physical coordinates, defect area parameters, and defect type labels.

[0042] Preferably, the method further includes: step S1071, acquiring the global average brightness of the reference optical image in real time; step S1072, when the fluctuation range of the global average brightness exceeds a preset deviation threshold, adjusting the diffusion coefficient function parameter in the anisotropic diffusion filter through a linear proportional function, and applying dynamic gain adjustment to the reference optical image to counteract the negative interference of drastic changes in external light intensity on the accuracy of defect feature extraction.

[0043] Example 1: In the continuous extrusion inspection of polyester film, the production line conveyor speed is 8m / s, and the film is undergoing a high-temperature cooling process at 160℃. Due to its high reflectivity and transient wrinkles caused by thermal stress fluctuations, optical noise is generated on its surface. Under this condition, single visible light imaging is prone to mapping reflective textures as pseudo-defect features due to the non-determinism of photon reflection. The system simultaneously acquires a reference optical image of the film surface and a temporal infrared thermal distribution image containing the current frame infrared image and the previous frame infrared image. The thermal flow displacement vector of each pixel is determined by the dense optical flow field from the previous frame infrared image to the current frame infrared image. The hysteresis compensation factor is determined based on the normal gradient component of the displacement vector in the direction of motion of the image sequence, and it is superimposed on the basic geometric affine matrix determined by the co-phase edge contour of the reference optical image and the current frame infrared image to generate a spatiotemporal compensation mapping matrix. This achieves pixel alignment between the optical image and the infrared image at the microsecond-level physical evolution scale and generates a structure guiding tensor.

[0044] To eliminate low-frequency background thermal interference caused by thin film thickness fluctuations, the system performs morphological opening operations on the structure-guided tensor using a preset two-dimensional structuring element matrix to extract the background thermal fluctuation matrix. This matrix is ​​then subtracted from the structure-guided tensor to obtain a pure-state high-frequency thermal gradient matrix containing defect characteristics. Specifically, before performing the morphological operations, the system calculates the absolute amplitude of each element in the structure-guided tensor to construct a non-negative gradient envelope matrix. Since the high-frequency thermal abrupt changes caused by microscopic defects spatially manifest as isolated structures with spans much smaller than the two-dimensional structuring element matrix, the system addresses these issues. The erosion step of the morphological opening operation strips away these local peaks, while the dilation step refits large-scale continuous gradient undulations. The resulting background thermal fluctuation matrix physically extracts the low-frequency gradient envelope baseline driven by the macroscopic thickness rheology of the material. This pure-state high-frequency thermal gradient matrix is ​​used to constrain the anisotropic diffusion filtering process. By using the thermal gradient peaks as edge stopping criteria, the reference optical image is isotropically smoothed in the reflective region and anisotropically blocked at the edges of real defects. Its calculation logic follows the following formula: ;in, coordinates First The optical pixel grayscale value of the next iteration. For the first The optical pixel grayscale value after the next iteration. The step size is a constant. For divergence operators, The diffusion coefficient function, For optical image gradient, The amplitude of the element at the corresponding position in the pure-state high-frequency thermal gradient matrix is ​​used. After the above filtering process, the optical artifacts and real defects in the reference optical image are physically decoupled, and the high-frequency noise caused by reflection is suppressed. Finally, morphological analysis is performed on the filtered image to extract the defect image region. Based on Otsu's maximum inter-class variance principle, a preset adaptive threshold for binarization is determined. The global pixel grayscale distribution sequence of the reference optical image after anisotropic diffusion filtering is collected, and the grayscale mean of the foreground pixel set and the background pixel set under different grayscale division levels is calculated. The grayscale order from 0 to 255 is traversed to determine... A specific gray level that maximizes the gray-level variance between the foreground and background pixel sets is set as a preset adaptive threshold. This preset adaptive threshold is applied to the reference optical image after anisotropic diffusion filtering. Low-frequency background pixels are stripped based on their gray-level values, and a binarized candidate region for suspected defects is output. The detection signal-to-noise ratio is improved by more than 15dB compared to the single-modal fusion method. This scheme resolves the conflict between genuine and fake defect identification under high reflectivity by using the thermal conductivity physical properties of the infrared domain as a structural constraint for feature extraction in the optical domain, thus achieving high-fidelity online monitoring of the surface quality of high-speed moving plastics.

[0045] In online monitoring scenarios where the optical imaging optical axis shifts at the micrometer level due to high-frequency mechanical vibration of the production line conveyor, the system constructs a coordinate mapping benchmark using the common edge contours in the reference optical image and the current frame infrared image. By calculating the probability distribution of pixel gradient directions belonging to the physical boundary of the plastic film in the two modal images, the system selects the consistent region of the gradient direction vector as the set of registration control points and extracts its spatial coordinate sequence. The least squares method is then used to solve for the rotation parameters. Scaling parameters and and translation parameters and The transformation parameter set is used to construct the basic geometric affine matrix, thereby converging the Euclidean distance between the infrared feature center and the optical defect centroid to within 1 pixel under physical vibration disturbance.

[0046] Example 2: When the monitoring system is in a polyester film production line environment with a continuous extrusion speed of 8 m / s, the test background includes a non-uniform temperature field generated from the molten state at 160℃ to the cooled state at 40℃, as well as optical reflection noise caused by high-frequency mechanical vibration. To verify the ability of the multimodal data fusion method to identify real defects, a synchronous data acquisition platform was constructed using a linear scan camera with an optical resolution of 0.05 mm / pixel and an infrared sensor with a thermal sensitivity better than 0.03 K. The data source was obtained by injecting Gaussian thermal drift noise with a signal-to-noise ratio of 20 dB and low-frequency thermal texture with a spatial period of 50 mm generated by film thickness fluctuations into the original image stream of the production line. The sampling period was... The determination depends on the thin film linear velocity. The integration time of a linear scan camera needs to strike a balance between avoiding pixel ghosting and ensuring feature sampling density. When increasing, The value tends towards the lower limit of the range to maintain spatial resolution stability, and is set to 1.25ms under the experimental conditions.

[0047] The experiment was divided into two comparative analyses. The experimental group used the fusion method of this invention, which includes morphological background stripping and anisotropic diffusion filtering, while the control group used a pixel-level weighted fusion method. In the original input stage, the micro-scratches on the film surface appear as local thermal spikes in the infrared mode. Since the amplitude is on the same order of magnitude as the background thermal fluctuation amplitude caused by uneven thickness, the target edges in the control group image are aliased with the background noise. The experimental group, by applying a preset two-dimensional structural element matrix... Perform a morphological opening operation on the structure-guided tensor. The dimensions are determined based on the spatial length related to thickness fluctuations. If the diameter is less than 20% of the thickness fluctuation wavelength, low-frequency components cannot be separated. When the diameter exceeds 5 times the defect feature size, it triggers feature smoothing. When the unit is set to 15 pixels, the system separates the background thermal fluctuation matrix from the structure-guided tensor, resulting in the extracted pure-state high-frequency thermal gradient matrix. Only the physical characteristics of the defect edges are preserved.

[0048] According to the anisotropic diffusion formula Implement pixel-level iterative updates with a constant step size. The value is set to 0.15 to ensure numerical convergence, where, coordinates First The optical pixel grayscale value of the next iteration. For the first The optical pixel grayscale value after the next iteration. The step size is a constant. For divergence operators, The diffusion coefficient function, For the spatial gradient of the optical image, For the amplitude of the element at the corresponding position in the pure-state high-frequency thermal gradient matrix, when processing regions containing strong reflectivity, based on... The amplitude approaches 0, and the diffusion coefficient function The driving image is subjected to strong isotropic smoothing, which smooths the reflective texture within 5 iterations, while the diffusion process at the real defect is preserved due to the suppression of large gradient values. Experimental data shows that the defect signal-to-noise ratio (SNR) of the control group after processing the above interference conditions is 12.4 dB, while that of the experimental group reaches 28.7 dB. In the thin film thickness deviation gradient test, when the deviation rate is 1.2%, the SNR of the experimental group is 29.5 dB. When the deviation rate increases to 2.5%, the SNR remains at 29.1 dB. When the deviation rate reaches the extreme value of 4.8%, the SNR is 28.4 dB. In contrast, the SNR of the control group under the above three gradients is 15.2 dB, 10.8 dB, and 7.3 dB, respectively, showing a deteriorating trend. This scheme solves the conflict between macroscopic fluctuation interference and microscopic feature extraction by decoupling the physical parameters of heterogeneous modes, and achieves a low sensitivity response of detection accuracy to physical environment fluctuations.

[0049] Example 3: This example combines Figures 1 to 2 This paper describes a method for online quality monitoring of plastics based on multimodal data fusion. Figure 1 As shown, step S101 synchronously acquires optical images of the plastic surface and temporal infrared thermal distribution images containing the current frame infrared image and the previous frame infrared image to obtain multimodal raw data. Step S102 calculates the spatial temperature gradient matrix of the current frame infrared image and extracts the initial gradient distribution used to characterize the infrared modal thermal diffusion features. Then, step S103 applies spatiotemporal phase compensation registration to the spatial temperature gradient matrix based on the common phase edge contour to construct a structure guiding tensor aligned with the pixel coordinate system of the reference optical image. Step S104 then applies morphological opening operations using 2D structuring elements to strip low-frequency background thermal fluctuations from the structure guiding tensor and outputs a pure-state high-frequency thermal gradient matrix reflecting micro-defect features. Step S105 adjusts the diffusion coefficient with the pure-state high-frequency thermal gradient matrix as a constraint and applies anisotropic diffusion filtering to the reference optical image to suppress optical noise and preserve edge features. Finally, step S106 applies morphological processing to the filtered optical image to achieve the final extraction of the physical-level defect image region of the plastic surface.

[0050] like Figure 2As shown, the temporal infrared thermal distribution image points downwards to the spatial temperature gradient matrix. This spatial temperature gradient matrix transfers its extracted spatial temperature gradient to the spatiotemporal phase compensation registration. Simultaneously, the reference optical image also transfers its associated common-phase edge contours to the spatiotemporal phase compensation registration. After spatiotemporal phase compensation registration, the image points downwards to the structure guiding tensor. This structure guiding tensor points to the background thermal fluctuation matrix through a morphological opening operation path, and simultaneously points to the pure-state high-frequency thermal gradient matrix through a background stripping path. The background thermal fluctuation matrix also points to the pure-state high-frequency thermal gradient matrix. Then, the pure-state high-frequency thermal gradient matrix inputs the diffusion coefficient adjustment constraint conditions to the anisotropic diffusion filter. Combined with the reference optical image, the image path is directly input to the anisotropic diffusion filter. Finally, the anisotropic diffusion filter outputs downwards to the defect image region.

[0051] Example 4: In an online monitoring application scenario involving plastic films with a thickness fluctuation period of 50 mm and interference fringes on the surface, the pixel feature points acquired by heterogeneous sensors experience displacement misalignment due to the spatial diffusion hysteresis effect of heat conduction. The system adjusts the sensor configuration based on the physical distance between the infrared detector and the linear scanning camera. and the real-time transport speed of the thin film The basic inter-frame delay is determined, and the Farneback algorithm is used to solve the dense optical flow field between the current frame infrared image and the previous frame infrared image in order to extract the displacement component of each pixel in the transmission direction. A directional distribution model of pixel displacement vectors is constructed based on the principle of local consistency of optical flow field statistical characteristics, with the pixel span set as... The system uses a two-dimensional sliding sampling window to extract the displacement vectors of all pixels within the window and counts the frequency of the longitudinal component along the conveying direction to generate a local histogram. The system searches for the maximum frequency peak in the histogram and extracts the corresponding displacement amplitude as the normal gradient component. Here, the normal is strictly defined as the direction perpendicular to the transverse physical edge contour of the plastic film. Since the transverse edge of the film is always orthogonal to the production line track under the constraint of the conveyor belt, the geometric normal direction of the transverse contour is strictly parallel to the longitudinal movement direction of the film conveyor belt in physical space. Therefore, the system extracts the magnitude of the displacement vector based on the normal constraint of the contour, which can be mapped and characterized as the true longitudinal physical movement component of the film as a whole in the conveying direction. Under the current physical boundary of the straight plastic conveying, the normal gradient component is equivalent to the longitudinal principal displacement component along the movement direction, eliminating the isolated optical flow noise caused by high-frequency mechanical vibration. The side length of the sliding sampling window is represented by the number of pixels, which is an integer greater than the optical resolution limit; a hysteresis compensation factor is used. Discretization perturbation correction is applied to the translation vector of the fundamental geometric affine matrix, where the hysteresis compensation factor... According to the formula Calculate, where, As a hysteresis compensation factor, This is the material's thermal diffusion correction constant. coordinates The pixel displacement vector component at that location, The physical placement distance of the detector. To achieve the thin film transport speed, the optical image and infrared feature spectrum are rigidly aligned in the pixel coordinate system using the spatiotemporal compensation mapping matrix, thereby realizing the physical in-situ construction of the structure guiding tensor.

[0052] To address the nonlinear control requirements of the diffusion rate in anisotropic diffusion filtering, the system incorporates an edge stopping function. Defined as an exponential decay form, it is used to dynamically adjust the smoothing intensity at the pixel scale, and its calculation follows the formula. ,in, The diffusion coefficient function, Let be the amplitude of the element at the corresponding position in the pure-state high-frequency thermal gradient matrix. The contrast parameter is obtained by adaptively calibrating the global gradient mean of the current structure guiding tensor. The amplitude is greater than the parameter When the function value tends towards 0 to block cross-edge diffusion, when When the amplitude approaches 0, the function value tends to 1 to achieve omnidirectional smoothing. During pixel iteration, the system determines the termination time by monitoring the variance change rate of the total image energy between two adjacent iterations. When the variance change rate is lower than a preset threshold for three consecutive samples... When diffusion reaches a steady state, high-frequency reflective noise in the reference optical image is suppressed, while edge features reflecting the defect morphology are preserved with high fidelity due to being locked by the high-frequency thermal gradient.

[0053] To ensure the adaptability of the morphological background stripping procedure to different extrusion processes, the system pre-calculates the characteristic wavelength of film thickness fluctuations based on the frequency of extruder screw speed and die pressure fluctuations. Based on this, the two-dimensional structural element matrix is ​​calibrated. pixel span , specifically follow The proportional relationship, where, The pixel span of the two-dimensional structuring element matrix. This is the coverage factor, and its value is set to range from 0.2 to 0.3. The characteristic wavelength of thickness fluctuation, For the camera's horizontal resolution, when the system detects that the screw speed increases from 50 rpm to 80 rpm, the characteristic wavelength... The corresponding reduction in pixel span is achieved by simultaneously shortening the pixel span. The value was reduced from 25 pixels to 15 pixels to ensure that the opening operation can accurately capture and separate the background thermal fluctuation matrix representing macroscopic thickness rheology. Ultimately, under the condition of fluctuation in the plastic production line environment, the signal-to-noise ratio of defect feature extraction is stabilized above 28dB. This method eliminates the conflict between macroscopic background interference and microscopic defect extraction by establishing a closed-loop mapping between physical parameters and image processing operators, and realizes accurate online monitoring of plastic surface quality.

[0054] Example 5: In the initial deployment of the polyester film production line, the system calibrates parameters for the physical properties of materials with different thicknesses and molecular weight distributions. This is achieved by calibrating the parameters of materials with different thicknesses and molecular weight distributions under a controlled constant temperature environment. A pulsed thermal excitation was applied to the sample, and the spatiotemporal evolution trajectory of the surface temperature gradient was monitored using an infrared detector. The thermal evolution peak was recorded on the horizontal coordinate. with vertical coordinates The offset displacement and corresponding time delay data at the specified location were used to calculate the linear fit between the displacement and the delay using the least squares method, thereby determining the material's thermal diffusion correction constant. The constant, ranging from 0.85 to 1.15, is used to correct the displacement deviation of the infrared feature spectrum relative to the reference optical image caused by thermal conduction hysteresis. In the underlying physical parameter mapping relationship, this thermal diffusion correction constant... The essence is determined by the intrinsic thermal diffusivity of a specific polymer material. During the initialization phase, the system obtains the standard intrinsic thermal diffusivity value of the polyester plastic at the current operating temperature by calling the internally established physical property parameter table. This value is then divided by the thermal diffusivity displacement obtained by fitting through actual pulse testing. This division operation yields a dimensionless weighting coefficient characterizing the degree of environmental convection interference, and this determined weighting coefficient is directly assigned to the correction constant. To participate in the calculation of delay compensation.

[0055] When the thin film surface is disturbed by optical imaging background noise caused by fluctuations in ambient humidity, the system acquires a reference noise image under defect-free operation and calculates the global gray-level variance of the pixel gray-level distribution. And the contrast parameter in the anisotropic diffusion filter Correlation function value calibrated as noise level ,in, For contrast parameters, This is the noise suppression gain coefficient. To address the global gray-level variance and bridge the gap between the global gray-level variance and the high-frequency thermal gradient in terms of physical dimensions, the system adjusts the gain coefficient in the aforementioned calibration formula. It possesses specific dimensional transformation properties, and its value includes a transformation scaling factor predetermined by blackbody radiation calibration experiments. This scaling factor is used to equivalently map the unit grayscale variance output by the optical sensor to a reference thermal gradient amplitude at a specific temperature / pixel span. Through this dimensionality reduction mapping rule, The value is transformed into a high-frequency thermal gradient of the pure state. A reference gradient threshold with the same dimensions is provided to ensure that the underlying division operation of the exponential decay term in the diffusion function has dimensionless mathematical feasibility. The calibration process involves conducting multiple diffusion step size experiments on a sample with a preset defect. When the ratio of the defect edge sharpness to the background smoothness reaches a local maximum, the specific value of the contrast parameter is determined, so that the filter diffusion rate is controlled by the statistical distribution characteristics of the background noise.

[0056] When the monitoring system faces dynamic operating conditions where the extrusion speed fluctuation rate exceeds 10%, in order to maintain the phase stability of the image sequence on the time axis, the system establishes an external triggering synchronization procedure based on encoder pulse feedback, which monitors the pulse frequency of the main drive shaft transmitted by the thin film. Dynamically adjust the line trigger signal of the linear scan camera and the sampling clock frequency of the infrared sensor. Calculate the sampling frequency calibration formula that satisfies the physical pixel displacement equivalence. ,in, The sampling clock frequency, The scaling factor is determined by the camera resolution and the encoder line count. The pulse frequency of the main drive axis is used to counteract the inter-frame image stretching deformation caused by non-uniform motion, and to provide a linearized spatiotemporal input reference for the subsequent displacement vector calculation of the dense optical flow field.

[0057] Example 6: In the deployment of the front-end system for polyester material production lines with different surface texture characteristics, to address the baseline drift caused by thermal noise accumulation due to long-term continuous operation of the optical imaging system, an online calibration procedure is set to maintain the long-term stability of feature extraction. The local gray-level variance sequence of the defect-free background area in the reference optical image after anisotropic diffusion filtering is acquired, and the statistical root mean square (RMS) of the sequence is calculated within the current sampling period. When the difference between the RMS and the initial calibration reference exceeds 5% of the calibration reference for three consecutive sampling periods, a baseline adaptive update mechanism is triggered. The RMS of the current period is used to replace the historical global brightness mean baseline, reconstructing the contrast parameter space in the anisotropic diffusion filter. During the dynamic reconstruction of the contrast parameter space, the system extracts the difference between the currently acquired global brightness mean and the initial calibration baseline value as a state variable, expressed as: The linear scaling function performs parameter refresh, where, and These are the contrast parameters after the update and the baseline, respectively. and These are the current and baseline global average brightness values, respectively. To establish a deterministic quantization cancellation mechanism between external illumination fluctuations and filter smoothing intensity, a diffusion rate compensation slope constant, pre-calibrated through multiple sets of illumination step attenuation experiments, was created. An initial thermal spectrum benchmark was established using calibration samples with preset thickness steps. Thickness fluctuation samples with wavelengths between 10 mm and 100 mm were generated by adjusting the extrusion pressure, and structural guidance tensors were simultaneously acquired across pixel spans. The local variance gradient of the background thermal fluctuation matrix is ​​calculated by incrementing the matrix in 1-pixel steps. This local variance gradient is specifically determined by... Within the macroscopic sliding neighborhood of a pixel, the gray-level variance of the elements in the background thermal fluctuation matrix within that neighborhood is statistically analyzed, and the span between two adjacent pixels is calculated. The variance value is obtained by taking the first derivative of the difference between iterations. The system continuously monitors the decay trend of the first derivative curve. When the variance gradient curve reaches a local minimum and the ratio of pixel displacement to fluctuation wavelength converges to around 0.22, the pixel span under the current material properties is determined. Calibration values, and the two-dimensional structure element matrix The topology is fixed as a circular symmetric core with isotropic distribution characteristics, thereby establishing a mapping relationship between the physical space characteristic wavelength and the spatial cutoff frequency of the digital filter.

[0058] When the monitoring system needs to maintain the algorithm's convergence reliability in an unsteady environment with ambient temperature fluctuations greater than 5°C, 100 frames of background images are continuously acquired during the production line's idle operation phase. The temporal gray-level correlation coefficient of the pixel sequence is calculated, and an adaptive fault-tolerant benchmark is established using the statistical distribution law of background noise. By testing filtering operators with different iteration depths on defective and defect-free sample groups, the variance of the total image energy is recorded as a function of the number of iterations. The changing monotonically decreasing curve occurs when the ratio of the edge gradient preservation rate of the detected defective region to the smoothness of the background noise reaches its maximum value, and the variance change rate of five consecutive samples is lower than the preset step value. When this situation occurs, the rate of change of variance is set as the judgment threshold. The value of is used to offset the loss of edge information caused by the increase of iteration step size by quantitatively characterizing the dynamic evolution of physical background noise, so as to ensure that the system can stably output quality assessment data with a signal-to-noise ratio of greater than 28dB in the online monitoring of raw materials with different physical properties.

[0059] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for online quality monitoring of plastics using multimodal data fusion, characterized in that, Includes the following steps: Step S101: Simultaneously acquire a reference optical image of the plastic surface and a temporal infrared thermal distribution image containing the current frame infrared image and the previous frame infrared image. Step S102: Calculate the spatial temperature gradient matrix of the current frame infrared image; Step S103: Based on the common edge contour of the reference optical image, a structure guidance tensor is generated by applying spatiotemporal phase compensation registration to the spatial temperature gradient matrix. The registration includes: determining the pixel displacement vector based on the dense optical flow field from the previous frame infrared image to the current frame infrared image; solving the basic geometric affine matrix based on the common edge contour; extracting the normal gradient component of the pixel displacement vector in the motion direction of the image sequence and superimposing it as a hysteresis compensation factor into the 2D translation vector of the basic geometric affine matrix to construct a spatiotemporal compensation mapping matrix; applying bilinear interpolation resampling to the spatial temperature gradient matrix based on the spatiotemporal compensation mapping matrix to output a structure guidance tensor aligned with the pixel coordinate system of the reference optical image. Step S104: Determine the 2D structural element matrix based on the preset pixel span. The preset span is greater than the maximum equivalent pixel diameter of the micro-defect of the plastic target to be tested. Apply morphological opening operation to the structural guidance tensor using the 2D structural element matrix to output the background thermal fluctuation matrix. Subtract the background thermal fluctuation matrix pixel by pixel from the structural guidance tensor to output the pure state high-frequency thermal gradient matrix. Step S105: Using the pure-state high-frequency thermal gradient matrix as a constraint term, anisotropic diffusion filtering is applied to the reference optical image. The diffusion coefficient is dynamically adjusted according to the amplitude of the element at the corresponding coordinate position in the pure-state high-frequency thermal gradient matrix to suppress optical high-frequency noise in the reference optical image and preserve edge features. Step S106: Apply morphological processing to the filtered reference optical image to extract the defect image region.

2. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 1, characterized in that, Step S103 involves solving the fundamental geometric affine matrix, which includes the following steps: Step S1031, extracting the common edge contours representing the physical boundary of the plastic from the reference optical image and the current frame infrared image; Step S1032, establishing a feature point mapping relationship based on the geometric distribution characteristics of the common edge contours; Step S1033, calculating parameters including rotation, scaling, and reference translation attributes based on the feature point mapping relationship to construct the fundamental geometric affine matrix.

3. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 1, characterized in that, Step S104 outputs the background thermal fluctuation matrix, which includes the following steps: Step S1041, applying 2D structural element matrix to the structure guiding tensor for erosion processing to remove small-scale temperature abrupt changes; Step S1042, applying expansion processing to the eroded result to generate the background thermal fluctuation matrix; wherein, the background thermal fluctuation matrix is ​​used to characterize the low-frequency thermal conduction background caused by macroscopic thickness rheology of plastic with a spatial frequency lower than a preset threshold.

4. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 1, characterized in that, Before step S102, the method further includes the following preprocessing steps: Step S1011, applying medium filtering to the time-series infrared thermal distribution image to filter out isolated noise pixels generated by the infrared detector; Step S1012, applying histogram equalization to the current frame infrared image to enhance the local contrast of the temperature distribution and obtain an enhanced infrared image with improved signal-to-noise ratio for extracting the spatial temperature gradient matrix.

5. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 1, characterized in that, Step S106 involves extracting the defective image region, which includes the following steps: Step S1061, applying binarization segmentation to the reference optical image based on a preset adaptive threshold to identify suspected defect candidate regions; Step S1062, applying closing operation to connect the broken edges within the suspected defect candidate regions and filling the internal voids; Step S1063, statistically analyzing the pixel feature parameters of the connected components, and removing non-defective interference regions that do not conform to the size specifications based on preset geometric area and aspect ratio constraints.

6. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 1, characterized in that, The extraction of the normal gradient component in step S103 includes the following steps: Step S1034, establish the directional distribution model of the pixel displacement vector; Step S1035, decompose the pixel displacement vector into a longitudinal component along the plastic conveying motion direction and a transverse component perpendicular to the conveying motion direction. Step S1036: Determine the weighting coefficients based on the real-time speed of the plastic conveying direction and the intrinsic thermal diffusivity of the plastic, and apply the weighting coefficients to quantize and correct the longitudinal component.

7. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 1, characterized in that, The anisotropic diffusion filtering in step S105 further includes: step S1051, constructing a multi-scale Gaussian pyramid containing multiple pixel sampling spans; step S1052, applying the pure-state high-frequency thermal gradient matrix of the corresponding scale to the reference optical image for filtering processing in each level of the multi-scale Gaussian pyramid; and step S1053, fusing the image features after filtering at each level to enhance the global stability of the detection algorithm for identifying defects with different physical spans.

8. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 5, characterized in that, Step S106 further includes the following post-processing steps: Step S1064, applying pixel coordinate remapping to the extracted defect image region; Step S1065, calculating the actual physical coordinates of the defect in the plastic transport coordinate system based on the centroid position of the defect image region in the reference optical image; Step S1066, outputting an online quality monitoring report containing the actual physical coordinates, defect area parameters, and defect type labels.

9. The method for online quality monitoring of plastics based on multimodal data fusion according to claim 1, characterized in that, The method also includes: step S1071, acquiring the global average brightness of the reference optical image in real time; step S1072, when the fluctuation range of the global average brightness exceeds the preset deviation threshold, adjusting the diffusion coefficient function parameter in the anisotropic diffusion filter through a linear proportional function, and applying dynamic gain adjustment to the reference optical image to counteract the negative interference of drastic changes in external light intensity on the accuracy of defect feature extraction.

Images