A robust visual recognition and equivalent unevenness estimation method for creased continuous strip material

Abstract

The present application relates to a kind of continuous strip material wrinkle's reflection light robust visual identification and equivalent unevenness estimation method, including the image frame of continuous strip material is collected and synchronous acquisition metadata, wrinkle identification, segmentation and classification are carried out to preprocessed image, output wrinkle mask, wrinkle category, pseudo-wrinkle mask and confidence degree;From wrinkle mask, parameterized feature of wrinkle is extracted, and parameterized vector is obtained;Based on parameterized vector and metadata, estimated uncertainty is output;When there is external sensor signal, multi-source adaptive fusion is carried out, and the equivalent unevenness of fusion is obtained;And, conflict diagnosis is carried out to visual equivalent unevenness and external signal equivalent unevenness, and according to the diagnosis result, degradation state machine strategy is executed.The present application can realize the robust identification and accurate classification of the surface wrinkle defect of continuous strip material under complex optical interference environment, to provide quantifiable control basis for closed-loop regulation and quality monitoring of production process.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a method for robust visual recognition of reflective properties and estimation of equivalent imbalance in continuous strip-shaped material wrinkles. Background Technology

[0002] Continuous strip materials (such as films, paper, metal foils, and fabrics) are prone to surface defects such as edge wrinkles, transverse wrinkles, periodic wrinkles, wavy edges, and wrinkle clusters during continuous processing. In existing technologies, machine vision-based surface defect detection methods have been widely applied in industrial online quality monitoring. These methods typically include image acquisition, image preprocessing, defect identification, and classification. For image preprocessing, existing technologies employ techniques such as filtering and denoising, grayscale conversion, and binarization to extract defect feature contours. For defect identification, existing technologies use deep learning models such as convolutional neural networks for feature extraction and classification prediction. Furthermore, existing technologies also involve using uncertainty to infer the degree of industrial defects and separating patterns from defects through low-rank matrix and sparse matrix decomposition.

[0003] However, existing technologies have several shortcomings in detecting wrinkle defects in continuous strip materials. First, the surface of continuous strip materials often exhibits optical interference such as reflection, overexposure, and interference fringes, which can easily lead to false detections or missed detections by the detection system. Second, the material surface may contain pseudo-wrinkle features such as joints, stains, scratches, reflective strips, and regular textures, making it difficult for existing technologies to effectively distinguish between genuine wrinkles and pseudo-wrinkles. Third, the output results of existing detection methods are mostly limited to qualitative judgments of "defective / no defect," making it difficult to output quantitatively controllable quantities that can be used for closed-loop adjustment. In addition, when visual inspection signals conflict with external sensor signals, existing technologies lack effective conflict diagnosis and degradation processing mechanisms. Finally, when production conditions change (such as changing materials, changing light sources, or changing operating speeds), the performance of existing detection models is prone to fluctuations, and deployment stability needs to be improved. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a robust visual recognition and equivalent imbalance estimation method for wrinkles in continuous strip materials. This method can achieve robust recognition and accurate classification of wrinkle defects on the surface of continuous strip materials under complex optical interference environments, and parameterize the wrinkle features into equivalent imbalance quantities, thereby providing a quantifiable control basis for closed-loop regulation and quality monitoring of the production process.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] A robust visual recognition method for reflective properties of continuous strip material wrinkles and an equivalent imbalance estimation method, comprising:

[0007] Acquire image frames of continuous strip-shaped materials and simultaneously obtain metadata;

[0008] The image frame is subjected to reflection suppression processing to suppress the highlight or specular reflection components in the image, resulting in a reflection-suppressed image;

[0009] The image after reflection suppression is subjected to interference fringe suppression processing to suppress periodic background texture, resulting in a fringe-suppressed image.

[0010] Motion compensation and region of interest adaptive locking are performed on the image after stripe suppression. The image is aligned to the material coordinate system and the region of interest is dynamically updated to obtain the preprocessed image.

[0011] The preprocessed image is subjected to wrinkle recognition, segmentation and classification, and the wrinkle mask, wrinkle category, pseudo-wrinkle mask and confidence score are output.

[0012] Extract the wrinkle parameterization features from the wrinkle mask to obtain the parameterization vector;

[0013] Based on the parameterized vector and the metadata, the visual equivalent imbalance is estimated through a regression model with physical prior constraints, and the estimation uncertainty is output.

[0014] When external sensor signals are present, the visual equivalent imbalance and the external signal equivalent imbalance are subjected to multi-source adaptive fusion to obtain a fused equivalent imbalance; and...

[0015] The visual equivalent imbalance and the external signal equivalent imbalance are diagnosed for conflict, and a degraded state machine strategy is executed based on the diagnosis results.

[0016] Furthermore, in the above method, the reflection suppression treatment includes at least one of the following:

[0017] Generate a specular mask and perform local texture reconstruction on the mask area;

[0018] The specular component and diffuse reflection component are separated by a polarization dual-channel separation; and

[0019] Generate unexposed texture maps by multi-exposure fusion.

[0020] Furthermore, in the above method, the interference fringe suppression process includes at least one of the following:

[0021] Suppressing stripe bands by estimating the main frequency band in the frequency domain and using an adaptive notch filter;

[0022] The image is decomposed into stripe components and defect components, and the defect components are preserved; and

[0023] When a texture exhibits a consistent direction and frequency across the entire frame and remains stable across frames, it is identified as a background texture and is weakened.

[0024] Furthermore, in the above method, the wrinkle parameterization features include at least two of the following:

[0025] Wrinkle direction , obtained through statistical analysis of the principal directions of the structure tensor or the tangents of the skeleton;

[0026] Wrinkle wavelength The frequency is obtained by Fourier transform or wavelet transform along the normal profile.

[0027] Wrinkle amplitude This is obtained through the peak-valley difference or gradient energy of the profile.

[0028] wrinkle density This is obtained by measuring the number of stripes per unit area or the number of energy peaks.

[0029] wrinkle area percentage ;

[0030] Persistence across frames ;

[0031] Edge offset ;as well as,

[0032] Complexity This includes the number of branches, the number of clusters, or the complexity of connected components.

[0033] Furthermore, in the above method, the regression model of the physical prior constraints satisfies at least one of the following constraints:

[0034] Monotonic constraint: When the wrinkle index When it increases, the equivalent imbalance quantity No decrease;

[0035] Boundary constraints: equivalent imbalance quantity The range of values ​​is limited by preset boundaries; and,

[0036] Consistency constraint: Under the same operating conditions, the changes in wrinkle direction and wavelength are equivalent to the amount of imbalance. The relationship remains stable; wherein, the estimation of the visual equivalent imbalance quantity adopts a combination of physical heuristic baseline and residual learning, expressed as:

[0037] ;

[0038] in For baseline estimation based on physical models, This is the residual learning function.

[0039] Furthermore, the above method also includes:

[0040] Candidate enhancement maps are constructed based on the preprocessed image, and these candidate enhancement maps are generated using at least one of directional filtering, structure tensor, or wavelet energy; and...

[0041] The candidate enhanced image and the preprocessed image are combined into a dual-stream input for wrinkle recognition, segmentation and classification.

[0042] Furthermore, the above method also includes:

[0043] The temporal consistency of the wrinkle identification, segmentation, and classification results is verified. When the wrinkle morphology is consistent across N consecutive frames and the confidence level exceeds a preset threshold, the wrinkle state is confirmed as a confirmed wrinkle state; and,

[0044] Output pixel-level uncertainty The overall uncertainty is calculated based on at least two of the following: image sharpness, exposure quality, reflectance intensity, fringe intensity, region of interest stability, and model output uncertainty. .

[0045] Furthermore, the above method also includes:

[0046] Construct a wrinkle index based on the parameterized vector. The wrinkle index By integrating the proportion of wrinkle area fold density fold amplitude and cross-frame persistence At least two of them were obtained; and,

[0047] Risk quantity is constructed by combining operating condition information and sensitive windows. The sensitive window includes a connector window or a highly reflective window.

[0048] Furthermore, in the above method, the multi-source adaptive fusion includes:

[0049] Computational visual uncertainty and external signal uncertainty ;

[0050] The fusion weight is calculated based on the visual uncertainty and the external signal uncertainty. ,in:

[0051] ;

[0052] Calculate the equivalent imbalance in fusion based on the fusion weights:

[0053] ;

[0054] In addition, an upper limit is set on the rate of change of the fusion weights. And set a hysteresis threshold to achieve weight stability.

[0055] Furthermore, in the above method, the conflict diagnosis and degradation state machine strategy includes:

[0056] when And if it continues for N frames, a conflict diagnosis is triggered;

[0057] Based on the conflict diagnosis results, the system will switch between the following states:

[0058] The NORMAL state indicates normal fusion output, the FREEZE state indicates frozen output as the final reliable value, the VIS_ONLY state indicates that only visual estimation is used, and the SEN_ONLY state indicates that only external signal estimation is used.

[0059] And, after the conflict disappears and the M-frame is maintained, it returns from the degraded state to the NORMAL state.

[0060] In summary, compared with the prior art, the present invention has at least one of the following beneficial technical effects:

[0061] The present invention provides a robust visual recognition and equivalent imbalance estimation method for continuous strip material wrinkles. Through reflection suppression and interference fringe suppression, it effectively eliminates the influence of optical interference such as highlights, specular reflection, and periodic background textures on wrinkle detection, significantly improving the robustness of the detection system in complex optical environments. By identifying, segmenting, and classifying wrinkles, it outputs wrinkle masks, wrinkle categories, and pseudo-wrinkle masks, effectively distinguishing between real and pseudo-wrinkle features. By extracting parameterized features from the wrinkle masks and estimating the visual equivalent imbalance using a regression model based on physical prior constraints, it transforms qualitative wrinkle detection results into quantifiable control variables, providing a direct and usable control basis for closed-loop adjustment of the production process. Through a multi-source adaptive fusion mechanism, it fuses visual estimation with external sensor signals, improving the accuracy and reliability of equivalent imbalance estimation. Through conflict diagnosis and degradation state machine strategies, it can automatically diagnose and execute corresponding degradation processing when visual signals conflict with external signals, ensuring stable operation of the system under abnormal conditions. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 The flowchart illustrates an embodiment of the present invention's method for robust visual recognition of wrinkles and reflections in continuous strip materials and estimation of equivalent imbalance quantities.

[0064] Figure 2 A schematic diagram of one embodiment of the three technical approaches for the reflection suppression treatment of the present invention is shown.

[0065] Figure 3 A schematic diagram of an embodiment of the interference fringe and periodic background texture suppression process of the present invention is shown.

[0066] Figure 4 A structural diagram of an embodiment of the wrinkle recognition processing flow of the present invention is shown.

[0067] Figure 5 A schematic diagram of an embodiment of the wrinkle parameterization characterization of the present invention is shown.

[0068] Figure 6 A schematic diagram of an embodiment of the learning framework and physical heuristic constraint mechanism for estimating equivalent imbalance quantities according to the present invention is shown.

[0069] Figure 7 A flowchart of an embodiment of the adaptive weighted fusion and weight stability mechanism of the present invention is shown.

[0070] Figure 8 A flowchart illustrating an embodiment of the conflict diagnosis and degradation state machine of the present invention is shown. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, it should be understood that the specific embodiments described herein are only for illustration and explanation of this application and are not intended to limit this application.

[0072] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments of this application. Furthermore, the descriptions of each embodiment in the following embodiments have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0073] The method steps described in this embodiment of the invention can be executed in the order described in the specific implementation, or the execution order of each step can be adjusted according to actual needs, provided that the technical problem can be solved. These are not listed one by one here.

[0074] The present invention will be further described in detail below with reference to the accompanying drawings.

[0075] This invention provides a method for robust visual recognition of reflectivity and estimation of equivalent imbalance in continuous strip-shaped material wrinkles. (Refer to...) Figure 1 The overall process of this method covers the complete processing chain from image acquisition, preprocessing, wrinkle recognition, parameterized characterization, equivalent imbalance estimation to multi-source fusion and conflict diagnosis and degradation.

[0076] A robust visual recognition method for identifying reflective wrinkles in continuous strip materials and estimating their equivalent imbalance involves acquiring image frames of the continuous strip material and simultaneously obtaining metadata. Continuous strip materials include films, paper, metal foils, and fabrics. During continuous processing, these strip materials are prone to surface wrinkling defects, including edge wrinkles, transverse wrinkles, periodic wrinkles, wavy edges, and wrinkle clusters. Figure 1 As shown, the camera acquisition synchronization step acquires image frames of the continuous strip material, and simultaneously acquires metadata. The metadata includes information such as exposure, light source intensity, velocity / displacement, and timestamp. This metadata is used for subsequent preprocessing and equivalent imbalance estimation.

[0077] Continue to refer to Figure 1 The acquired image frames then enter the reflection stripe suppression, motion compensation, and ROI alignment step. This step performs reflection suppression, interference stripe suppression, motion compensation, and adaptive ROI locking on the image frames. Motion compensation and adaptive ROI locking are then applied to the stripe-suppressed image, aligning it to the material coordinate system and dynamically updating the ROI to obtain the preprocessed image. During the adaptive ROI locking process, the ROI is divided into left edge, right edge, and center ROI regions, and the joint window and seam window are marked. This region segmentation method allows for differentiated processing strategies for different regions in subsequent wrinkle recognition.

[0078] The preprocessed image then proceeds to the wrinkle recognition and false wrinkle suppression step. This step identifies, segments, and classifies wrinkles, and suppresses false wrinkle features. The recognition results then proceed to the wrinkle parameter extraction step, from which parameters including orientation are extracted. ,wavelength Amplitude ,density Complexity Parametric features are then incorporated into the equivalent imbalance estimation step, which outputs the visual equivalent imbalance. .

[0079] Continue to refer to Figure 1 The uncertainty assessment step is used to calculate the overall uncertainty. and visual uncertainty It also provides data to the outside world through the output interface. Visual equivalent imbalance quantity Equivalent Imbalance to External Signal The fusion is performed in the multi-source adaptive fusion step to obtain the fusion equivalent imbalance quantity. The fusion results proceed to the conflict diagnosis and degradation step. This step outputs different system states based on the diagnosis results, including NORMAL (normal fusion output), FREEZE (frozen output), VIS-ONLY (visual estimation only), and SEN-ONLY (external signal estimation only). The conflict diagnosis and degradation step also outputs alarm, adjustment, and monitoring signals externally, forming a closed-loop feedback to the camera acquisition and synchronization step.

[0080] During image acquisition, the specular or specular reflection components on the surface of continuous strip-shaped materials can interfere with wrinkle recognition, leading to false detections or missed detections. This invention further performs reflection suppression processing on the image frames to suppress the specular or specular reflection components in the image, obtaining a reflection-suppressed image. (Refer to...) Figure 2 Reflection suppression processing includes three technical approaches, which can be used individually or in combination to effectively suppress highlight areas.

[0081] like Figure 2 As shown, the top of the image displays the original, continuous strip-shaped material surface image, which contains a significant area of ​​high reflectivity, manifested as a bright white spot in the center of the image. Three parallel processing branches extend downwards from the original image, corresponding to three different reflection suppression methods.

[0082] Continue to refer to Figure 2 The left branch is labeled as specular mask reconstruction. In this method, a specular mask is generated, and local texture reconstruction is performed on the mask area. The specular mask is generated based on pixel brightness threshold judgment or adaptive threshold segmentation, marking overexposed areas as mask areas. For the mask areas, texture inpainting techniques are used for local texture reconstruction. These techniques utilize effective texture information surrounding the mask area to fill and restore the overexposed areas. Figure 2From the processed image, the highlight areas are effectively suppressed, and the wrinkle texture features of the material surface are preserved.

[0083] like Figure 2 As shown, the middle branch is labeled as polarization imaging. In this method, the specular component and diffuse reflection component are separated by a dual polarization channel. Polarization imaging uses a polarization filter to acquire images of polarization channel 1 and polarization channel 2. Specular reflection light has strong polarization characteristics, while diffuse reflection light has weak polarization characteristics. By calculating the difference between the two polarization channels, the specular reflection component is effectively separated and removed, while the diffuse reflection component is retained. Figure 2 The polarization difference results show that specular reflection interference is effectively removed, and the true texture information of the material surface is preserved.

[0084] Continue to refer to Figure 2 The branch on the right is labeled as multi-exposure fusion. In this method, an unexposed texture image is generated through multi-exposure fusion. Multi-exposure fusion employs HDR fusion technology, acquiring multiple images under different exposure conditions. Short-exposure images retain texture details in highlight areas, while long-exposure images retain texture details in shadow areas. A fusion algorithm weights and fuses the images under different exposure conditions to generate an unexposed texture image with a wider dynamic range. Figure 2 The fusion results show that the surface texture details of the material are well preserved, and the problem of overexposure of highlights is effectively solved.

[0085] The three reflection suppression methods described above address the interference of specular or specular reflection components on wrinkle recognition from different technical perspectives. The specular mask reconstruction method is suitable for processing localized specular areas, the polarization imaging method is suitable for material surfaces with strong specular reflection, and the multi-exposure fusion method is suitable for imaging scenarios with a large dynamic range. In practical applications, depending on the surface characteristics of continuous strip-shaped materials and imaging conditions, one or more of these methods can be selected and combined to obtain a reflection-suppressed image, providing a higher-quality input image for subsequent wrinkle recognition.

[0086] After completing the reflection suppression processing, the image is then subjected to interference fringe suppression processing to suppress periodic background textures, resulting in a fringe-suppressed image. During the imaging process of continuous strip-shaped materials, interference fringes or periodic background textures may appear in the image due to the material's inherent periodic structure, the interaction between the light source and the material surface, or the optical characteristics of the imaging system. These periodic textures are similar to the texture characteristics of wrinkles and defects, easily leading to false positives or false negatives in the wrinkle identification process.

[0087] Reference Figure 3 The processing flow for suppressing interference fringes and periodic background textures includes three technical paths, which can be used individually or in combination to effectively suppress periodic background textures. Figure 3 The top image shows the original acquired image of a continuous strip of material with obvious periodic stripe texture, containing regular vertical stripe interference. The processing flow below is divided into three parallel technical paths, labeled as frequency domain notch filtering, stripe dictionary separation, and global consistency judgment.

[0088] like Figure 3 As shown, the left path illustrates the spectrum analysis process. In this method, the dominant frequency band is estimated in the frequency domain, and an adaptive notch filter is used to suppress the fringe band. A Fourier transform is performed on the image after reflection suppression to obtain its spectrum. In the spectrum, periodic fringes correspond to energy concentration regions with specific directions and frequencies. By analyzing the energy distribution in the spectrum, the direction and dominant frequency band of the fringes are estimated. Based on the estimated fringe direction and dominant frequency band parameters, an adaptive notch filter is constructed. The notch filter suppresses the dominant frequency band corresponding to the fringes in the frequency domain while preserving other frequency components. Figure 3 The bottom left side displays the frequency spectrum after frequency domain processing, showing that the energy of the frequency bands corresponding to the stripes has been effectively suppressed.

[0089] Continue to refer to Figure 3 The intermediate path illustrates the process of separating the stripe components. In this method, the image is decomposed into stripe components and defect components, with the defect components retained. Stripe component separation is based on sparse representation or dictionary learning methods to establish a dictionary model of the stripe texture. The image is decomposed using the stripe dictionary, representing it as a superposition of stripe and defect components. The stripe components correspond to periodic background textures, while the defect components correspond to non-periodic defect features such as wrinkles. By removing the stripe components and retaining the defect components, periodic background textures are effectively suppressed. Figure 3 The middle image shows the intermediate result after stripe stripping, and the stripe component image shows the separated periodic texture structure.

[0090] like Figure 3 As shown, the path on the right illustrates the orientation and frequency detection process. In this method, when a texture exhibits a consistent orientation and frequency across the entire image and remains stable across frames, it is identified as background texture and weakened. This method is based on the principle of global consistency discrimination, analyzing the orientation and frequency distribution of textures in the image. When a texture displays consistent orientation and frequency characteristics across the entire image, and these texture characteristics remain stable across multiple consecutive frames, the texture is identified as background texture rather than a wrinkle defect. For regions identified as background textures, spatial filtering or frequency domain filtering methods are used for weakening.

[0091] Continue to refer to Figure 3The rightmost image shows the final suppression result. After the above interference fringe suppression processing, the periodic fringe interference is effectively reduced, while the true wrinkle defect features are preserved. The image after fringe suppression provides a clearer input image for subsequent wrinkle recognition and segmentation, reducing the impact of periodic background texture on the accuracy of wrinkle recognition.

[0092] The three interference fringe suppression methods described above address the interference of periodic background textures on wrinkle recognition from different technical perspectives. The frequency domain dominant band estimation and adaptive notch filtering method is suitable for scenarios with relatively simple fringe directions and frequencies; the fringe component separation method is suitable for scenarios with more complex fringe textures; and the global consistency discrimination method is suitable for scenarios where the background texture is stable across frames. In practical applications, based on the surface characteristics and fringe texture features of continuous striped materials, one or more of the above methods can be selected and combined to obtain the image after fringe suppression.

[0093] After completing reflection suppression and interference fringe suppression processing, the preprocessed image is subjected to wrinkle recognition, segmentation, and classification, outputting wrinkle masks, wrinkle categories, pseudo-wrinkle masks, and confidence scores. (Refer to...) Figure 4 The overall architecture of the wrinkle recognition network includes four main output branches: segmentation head, classification head, pseudo-wrinkle head, and uncertainty head.

[0094] like Figure 4 As shown, the network's backbone receives the preprocessed input image and generates feature maps at different levels through multi-scale feature extraction. These feature maps are represented by 3D cubes of different colors, with their size gradually decreasing and the number of channels increasing from left to right. Multi-scale feature extraction enables the network to simultaneously capture both local detail features and global structural features of wrinkles, providing rich feature representations for subsequent multi-task outputs.

[0095] Continue to refer to Figure 4 Following the backbone, the network is divided into four parallel output heads. The H1 segmentation head is used to generate the wrinkled mask. The system outputs spatial location information for wrinkle detection. The wrinkle mask represents the spatial distribution of wrinkle regions in pixel-level binary or probabilistic form, providing a spatial localization basis for subsequent wrinkle parameterization. The H2 classification head is used to output the wrinkle category. This allows for the differentiation of different types of wrinkle defects, such as edge wrinkles, transverse wrinkles, and periodic wrinkles. Wrinkle category information is used for subsequent estimation of equivalent imbalance and quality monitoring.

[0096] The H3 pseudo-wrinkle head is used to identify and suppress pseudo-wrinkle regions. Pseudo-wrinkle categories include interfering factors such as joints, stains, scratches, reflective stripes, interference fringes, and regular textures. These pseudo-wrinkle features exhibit texture characteristics similar to real wrinkles in the image, but are not wrinkles or defects on the material surface. The pseudo-wrinkle head outputs a pseudo-wrinkle mask and fuses it with the segmentation result to obtain the final pseudo-wrinkle mask. By suppressing false wrinkles, real wrinkles and false wrinkle features can be effectively distinguished, reducing the false detection rate in the wrinkle recognition process.

[0097] The H4 uncertainty header is used to output the confidence level. and pixel-level uncertainty Pixel-level uncertainty The uncertainty of each pixel location is represented in the form of a heatmap. Areas with higher uncertainty correspond to areas with blurred wrinkle boundaries or indistinct features. Pixel-level uncertainty provides the basic data for subsequent calculation of overall uncertainty and determination of credibility.

[0098] The complete processing flow from the original image or candidate enhancement map to the final output shows examples of the input original image and candidate enhancement maps on the left, intermediate output results of each head in the middle, and the final wrinkle mask, wrinkle category, pseudo-wrinkle mask fusion result, and pixel-level and overall uncertainty on the right. Heatmap output.

[0099] Candidate enhancement maps are constructed based on the preprocessed image. These maps are generated using at least one of directional filtering, structure tensor, or wavelet energy. Directional filtering extracts the texture response in specific directions within the image using a directional filter bank, enhancing the directional features of the wrinkles. Structure tensor extracts the principal orientation and anisotropy of local regions by calculating the second-order moment matrix of the image gradient, enhancing the structural features of the wrinkles. Wavelet energy calculates the energy distribution at different scales and directions through multi-scale wavelet decomposition, enhancing the multi-scale features of the wrinkles.

[0100] The candidate augmented image and the preprocessed image are combined into a two-stream input for wrinkle recognition, segmentation, and classification. The two-stream input allows the network to simultaneously utilize the grayscale information of the original image and the structural information of the candidate augmented image, improving the accuracy and robustness of wrinkle recognition. In this two-stream input architecture, the two input streams are processed separately during the feature extraction stage and integrated during the feature fusion stage, enabling the network to comprehensively utilize different types of feature information.

[0101] The temporal consistency of wrinkle recognition, segmentation, and classification results is verified. When the wrinkle morphology is consistent across N consecutive frames and the confidence level exceeds a preset threshold, the wrinkle state is confirmed as a valid wrinkle state. This temporal consistency verification mechanism utilizes the temporal correlation between consecutive frames to verify and confirm the recognition results of a single frame. In the online detection of continuous strip materials, genuine wrinkle defects exhibit consistent morphological features and spatial locations across multiple consecutive frames, while noise or transient interference displays inconsistent characteristics between consecutive frames. Through temporal consistency verification, false detections caused by transient interference are effectively suppressed, and the stability of wrinkle recognition is improved.

[0102] Output pixel-level uncertainty The overall uncertainty is calculated based on at least two of the following: image sharpness, exposure quality, reflectance intensity, fringe intensity, region of interest stability, and model output uncertainty. Image sharpness is evaluated using metrics such as Laplacian variance, reflecting the degree of blur. Exposure quality is evaluated using saturation ratio and shadow ratio, reflecting the image's exposure state. Reflection intensity is evaluated using highlight mask area and intensity, reflecting the degree of highlight interference in the image. Stripe intensity is evaluated using frequency domain energy concentration, reflecting the degree of periodic stripe interference in the image. Region of interest stability is evaluated using edge drift and alignment error, reflecting the stability of region of interest locking. Model output uncertainty is evaluated using pixel-level uncertainty. The statistical evaluation reflects the confidence level of the network identification results.

[0103] Overall uncertainty The confidence level is obtained through the normalization and fusion of the above indicators and is used to output the confidence status. The confidence status output includes three levels: credible, doubtful, and unavailable. When the overall uncertainty... When the value is below the first threshold, the output confidence status is set to "confidential," indicating that the recognition result of the current frame has high reliability. When the overall uncertainty... When the value falls between the first and second thresholds, the output confidence level is "suspicious," indicating that the recognition result of the current frame has a certain degree of uncertainty, and the host system adopts a conservative strategy for processing. When the overall uncertainty... When the value exceeds the second threshold, the output confidence status is unavailable, indicating that the recognition result of the current frame is unreliable and the host system does not use the recognition result of this frame for decision-making.

[0104] After completing wrinkle recognition, segmentation, and classification, parametric features of the wrinkles are extracted from the wrinkle mask to obtain parametric vectors. (Refer to...) Figure 5 The schematic diagram of wrinkle parameterization illustrates the process of extracting multidimensional parameterized features from wrinkle images. Figure 5The upper part shows how to extract five key parameters, and the lower part shows the specific implementation of parameter extraction.

[0105] like Figure 5 As shown, the parametric features of wrinkles include wrinkle direction. , Wrinkle wavelength fold amplitude fold density , percentage of wrinkled area , continuity across frames Edge offset and complexity These parameterized features originate from wrinkled masks. Extracted from the data, and combined with the skeleton and boundary information for calculation.

[0106] Continue to refer to Figure 5 fold direction The principal directions are obtained through structural tensor methods or skeleton tangent statistics. The structural tensor method calculates the second-order moment matrix of the image gradient within the wrinkled region, and obtains the principal directions through eigenvalue decomposition. The skeleton tangent statistical method performs skeletonization on the wrinkled mask, statistically analyzes the tangent direction distribution at each point of the skeleton, and obtains the principal directions and directional dispersion of the wrinkles. Figure 5 The first column shows wrinkle images with directional arrows, illustrating the process of extracting directional features.

[0107] Wrinkle wavelength The dominant frequency is obtained by performing Fourier transform or wavelet transform along the normal profile. A grayscale profile curve is extracted along the normal direction of the folds, and Fourier transform or wavelet transform is performed on the profile curve to analyze the dominant frequency component in the spectrum. The period corresponding to the dominant frequency is the fold wavelength. , indicating the main periodic characteristics of the wrinkled stripes. Figure 5 The second column shows the periodic fluctuation characteristics of the wrinkles, and the wavelength parameters are obtained through spectral analysis below.

[0108] Wrinkle amplitude The height of the wrinkles is obtained through either peak-valley difference or gradient energy. The peak-valley difference method extracts a grayscale profile curve along the normal direction of the wrinkles and calculates the difference between the peak and valley values ​​in the profile curve; this difference reflects the height characteristics of the wrinkles. The gradient energy method calculates the energy statistics of the image gradient within the wrinkled region; a larger gradient energy indicates a larger wrinkle amplitude. (Wrinkle amplitude is then calculated.) The shadow span is obtained by dark field imaging. The width of the shadow produced by the wrinkle under dark field imaging conditions is related to the wrinkle height. The shadow span is converted into the actual wrinkle amplitude by combining the calibration coefficient. Figure 5 The third column shows how the amplitude is measured by overlaying a grid onto a wrinkled image, with the amplitude parameters obtained below by measuring the profile height.

[0109] wrinkle density The number of energy peaks is obtained through either the number of fringes per unit area or the number of energy peaks. The method of counting fringes per unit area counts the number of fringes within a unit area of ​​the wrinkled region; a higher number of fringes indicates a greater wrinkle density. The method of counting energy peaks performs frequency domain analysis on the wrinkled region, counting the number of energy peaks in the spectrum; the number of energy peaks reflects the density characteristics of the wrinkles. Figure 5 The fourth column shows the statistical process of the number of wrinkles and stripes per unit area.

[0110] wrinkle area percentage This represents the ratio of the area of ​​the wrinkled region to the total area of ​​the region of interest. (Wrinkled area percentage) It reflects the distribution range of wrinkle defects on the material surface; the larger the area ratio, the more severe the wrinkle defects.

[0111] Persistence across frames This indicates the temporal tracking persistence characteristics of wrinkles. (Cross-frame persistence) This is achieved by tracking and matching wrinkles in multiple consecutive frames of images, reflecting the consistency and stability of wrinkles over time. Figure 5 The fifth column shows the multi-frame tracking trajectory, illustrating the process of extracting continuous features across frames.

[0112] Edge offset This indicates the offset of the area of ​​concentrated wrinkles relative to the edge of the material. Edge offset Used to distinguish between edge wrinkles and center wrinkles. Edge wrinkles are concentrated in the edge area of ​​the material, while center wrinkles are distributed in the center area of ​​the material.

[0113] Complexity This includes the number of bifurcation points, the number of clusters, or the complexity of connected components. The number of bifurcation points counts the number of branching points in the wrinkled skeleton; more branching points indicate a more complex wrinkled structure. The number of clusters counts the number of independent clusters in the wrinkled region; more clusters indicate a more dispersed wrinkled distribution. Connected component complexity calculates the topological complexity of the wrinkled region through connected component analysis.

[0114] like Figure 5 As shown at the bottom, the parameterized vector The construction integrates the above-mentioned wrinkle parameterized features into a complete parameter vector, represented as:

[0115] ;

[0116] Parameterized vector The multidimensional geometric and statistical characteristics of wrinkles are integrated into a unified numerical representation for subsequent estimation of equivalent disequilibrium. The parameters in the parameterized vector describe the characteristics of the wrinkles from different dimensions, including orientation. With wavelength Describe the geometric shape and amplitude of the wrinkles. With density Describe the severity of the wrinkles and their area percentage. With edge offset Describe the spatial distribution of wrinkles and their persistence across frames. Describe the temporal stability and complexity of wrinkles. Describe the structural complexity of the wrinkles.

[0117] Constructing a wrinkle index based on parameterized vectors Wrinkle index By integrating the proportion of wrinkle area fold density fold amplitude and cross-frame persistence At least two of the following must be obtained: Wrinkle Index This is a comprehensive index of wrinkle severity, integrating multiple parametric features into a single value to facilitate subsequent estimation of equivalent imbalance and threshold determination. Wrinkle Index The calculation adopts a weighted summation or nonlinear fusion method, and the weight of each parameter is configured according to the material type and working conditions.

[0118] Risk quantity is constructed by combining operating condition information and sensitive windows. Sensitive windows include joint windows or highly reflective windows. Operating information includes parameters such as material speed, tension setting, and light source intensity. The joint window corresponds to the time window within which the material joint passes through the detection area; within this window, abrupt changes in material surface features increase the uncertainty of wrinkle recognition. The highly reflective window corresponds to areas or periods of high reflectivity on the material surface; within this window, image quality degrades, reducing the reliability of wrinkle recognition. Risk level. Taking into account the wrinkle index In relation to the sensitive window state, the risk amount is adjusted within the sensitive window to ensure that the risk amount... It can reflect the actual impact of wrinkle defects on product quality at the current moment.

[0119] After completing the parametric characterization of wrinkles and constructing the wrinkle index, based on the parametric vectors and metadata, a regression model with physical prior constraints is used to estimate the visual equivalent imbalance, and the estimated uncertainty is output. (Refer to...) Figure 6 Equivalent imbalance quantity The learning framework and physical heuristic constraint mechanism of the estimation step consists of two parts: the left side is the learning framework and the right side is a specific example of the physical heuristic constraint.

[0120] like Figure 6 As shown on the left, the top of the learning framework section displays a schematic diagram of the neural network structure with multi-layer feature representations. The estimation of the visual equivalent imbalance quantity adopts a combination of physically inspired baseline and residual learning, represented as follows:

[0121] ;

[0122] in, For baseline estimation based on physical models, This is the residual learning function. Baseline estimation based on the physical model. Establish parameterized vectors based on the physical mechanism of wrinkle formation An analytical relationship between the residual learning function and the equivalent imbalance quantity, which is physically interpretable. This method learns the residual between the physical baseline estimate and the actual equivalent imbalance, compensating for the deviation between the simplifying assumptions of the physical model and actual operating conditions. This combination of physical heuristic baseline and residual learning offers both interpretability and adaptability; the physical baseline provides a stable estimation basis, while residual learning provides adaptability to complex operating conditions.

[0123] Continue to refer to Figure 6 After the training data flows through the visual-physical baseline module, it enters the residual learning step. Physically inspired constraints and training data work together to influence the loss function. The loss function is composed of... Loss items and The loss consists of two parts. The loss term is used to constrain the deviation between the estimated value and the true value. The loss term is used to constrain the accuracy of the estimated uncertainty.

[0124] like Figure 6 As shown on the right, the regression model with physical prior constraints satisfies monotonicity constraints, boundary constraints, and consistency constraints. The monotonicity constraint requires that the wrinkle index... When it increases, the equivalent imbalance quantity The error does not decrease. This constraint is based on the physical mechanism of wrinkle formation; the more severe the wrinkles, the greater the degree of unevenness on the material surface, and the equivalent unevenness should increase accordingly. The monotonic constraint is implemented by adding a monotonicity penalty term to the loss function. When the model output violates monotonicity, the penalty term increases the loss value, guiding the model to learn a mapping relationship that satisfies monotonicity.

[0125] Continue to refer to Figure 6 Boundary constraints require equivalent imbalance quantities The value range is limited by preset boundaries. These preset boundaries are determined based on material type, process parameters, and equipment capacity. The upper limit of the equivalent imbalance corresponds to the maximum imbalance the material can withstand, while the lower limit corresponds to the baseline value in a wrinkle-free state. Boundary constraints are implemented by adding a boundary penalty term to the loss function. When the model output exceeds the preset boundaries, the penalty term increases the loss value, guiding the model output to an estimated value within a reasonable range. Figure 6 The right side shows the equivalent imbalance quantity. A diagram illustrating constraints within the preset upper and lower limits.

[0126] like Figure 6 As shown, the consistency constraint requires that, under the same operating conditions, the changes in wrinkle direction and wavelength are equivalent to the unbalanced amount. The relationship remains stable. Same working conditions refer to conditions where process parameters such as material type, operating speed, and tension settings are identical. Under the same working conditions, the wrinkle direction... With wavelength Changes and equivalent imbalance quantities There exists a stable correspondence between them, which is determined by the mechanical properties of the materials and the processing conditions. Consistency constraints are implemented by adding a consistency penalty term to the loss function. When the model produces inconsistent outputs to similar inputs under the same operating conditions, the penalty term increases the loss value, guiding the model to learn a stable mapping relationship.

[0127] Continue to refer to Figure 6 The figure shows the wrinkle variables. When it increases The correspondingly enlarged schematic diagram includes a graph showing the relationship between the actual wrinkle image and the corresponding parametric curve. Figure 6 The complete formula for the constraint loss function is given at the bottom. This function is... Loss term, boundary constraint term, summation of various constraint terms, and weighting coefficients Multiply The loss term is composed of several components. The constrained loss function is expressed in the following form:

[0128] ;

[0129] in, The main loss term for estimating the equivalent imbalance is... For boundary constraint penalty terms, Penalty terms for various physical constraints (including monotonic constraints and consistency constraints). The loss term for uncertainty estimation, This is the weighting coefficient for uncertainty loss.

[0130] Equivalent imbalance estimation employs piecewise or conditional models to improve generalization ability. Piecewise models select corresponding sub-models based on material type, operating speed, or lighting conditions. Different material types possess different mechanical properties, leading to variations in the mapping relationship between wrinkle formation mechanisms and equivalent imbalance; training dedicated sub-models for different material types improves estimation accuracy. Furthermore, the dynamic response characteristics of materials differ at different operating speeds, altering the relationship between wrinkle morphology and equivalent imbalance; training dedicated sub-models for different speed ranges adapts to speed variations. Finally, image quality and feature representation differ under different lighting conditions; training dedicated sub-models for different lighting conditions improves robustness to lighting changes.

[0131] Conditional models utilize load case embedding to improve generalization ability. Load case embedding encodes load case parameters such as material type, operating speed, and lighting conditions into vector representations, and links them with parameterized vectors. These parameters are input into the regression model. Operating condition embedding enables a single model to perceive current operating conditions and adjust its internal mapping relationships accordingly, achieving unified modeling of multiple operating conditions. Operating condition embedding methods include one-hot encoding, continuous value normalization, or learnable embedding vectors; the appropriate embedding method is selected based on the type and number of operating condition parameters.

[0132] Visual equivalent imbalance quantity The estimate simultaneously outputs the estimation uncertainty. Or confidence interval. Estimate uncertainty. The uncertainty reflects the reliability of the current estimate; a larger uncertainty indicates a lower level of confidence in the estimate. The calculation of the estimation uncertainty is based on the output distribution of the model or the prediction variance of the ensemble model, providing a basis for subsequent weight calculations in multi-source adaptive fusion.

[0133] In completing the visual equivalence imbalance quantity After estimation, when external sensor signals are present, multi-source adaptive fusion is performed on the visual equivalent imbalance and the external signal equivalent imbalance to obtain the fused equivalent imbalance. External sensor signals include signals acquired by force sensors, vibration sensors, displacement sensors, etc., which are processed and converted into the external signal equivalent imbalance. Multi-source adaptive fusion combines visual estimation with external signal estimation, leveraging the complementarity of the two signal sources to improve the accuracy and robustness of equivalent imbalance estimation.

[0134] Reference Figure 7 The detailed flowchart of the adaptive weighted fusion and weight stability mechanism shows the complete implementation of the fusion process. Figure 7 The left side shows the construction process of the candidate augmented response atlas, including multiple augmented response maps. These enhanced response maps, extracted from the preprocessed image, exhibit different texture features and wrinkle information. The candidate enhanced response map set is represented as follows: equal arrive A set of.

[0135] Multi-source adaptive fusion includes computational visual uncertainty. and external signal uncertainty Visual uncertainty As mentioned earlier, the calculation is based on the normalized fusion of indicators such as image sharpness, exposure quality, reflectivity, stripe intensity, region of interest stability, and model output uncertainty.

[0136] External signal uncertainty The parameters are calculated from drift detection, saturation detection, disconnection / packet loss detection, noise level, sampling integrity, and consistency with operating conditions. Drift detection detects slow baseline drift in the external sensor signal, which can lead to systematic biases in the estimation of equivalent imbalance quantities. Saturation detection detects whether the external sensor signal has reached its upper or lower limit; when the signal is saturated, the sensor cannot accurately reflect the actual changes in physical quantities. Disconnection / packet loss detection detects disconnections or data packet loss in the communication link of the external sensor; communication anomalies can lead to data loss or delays. Noise level assesses the noise intensity in the external sensor signal; higher noise levels indicate poorer signal quality. Sampling integrity assesses whether the external sensor signal is fully sampled; incomplete sampling leads to information loss. Consistency with operating conditions assesses the degree of matching between the external sensor signal and the current operating conditions; inconsistency between the signal and operating conditions indicates an abnormal sensor state.

[0137] Continue to refer to Figure 7 The fusion weights are calculated based on visual uncertainty and external signal uncertainty. Fusion weights The calculation formula is:

[0138] ;

[0139] The fusion weight calculation formula is based on the inverse variance weighting principle. Signal sources with lower uncertainty receive larger fusion weights, while signal sources with higher uncertainty receive smaller fusion weights. When the external signal uncertainty... Smaller visual uncertainty When the weight is large, the fusion weight As the value approaches 1, the fusion result relies more on external signal estimation. When visual uncertainty... Smaller external signal uncertainty When the weight is large, the fusion weight Approaching 0, the fusion result relies more on visual estimation.

[0140] like Figure 7 As shown, the equivalent imbalance in fusion is calculated based on the fusion weights:

[0141] ;

[0142] Fusion equivalent unbalanced quantity External signal equivalent imbalance quantity Visual equivalent imbalance quantity The weighted combination is determined adaptively by the uncertainties of the two signal sources. This adaptive weighted fusion method allows the fusion result to be dynamically adjusted according to the real-time quality status of the two signal sources. When the quality of the visual signal deteriorates, the weight of the external signal is automatically increased, and when the quality of the external signal deteriorates, the weight of the visual signal is automatically increased.

[0143] Continue to refer to Figure 7 , Figure 7 The right side details the implementation of the weight stability mechanism, including setting an upper limit on the rate of change for the fused weights. And set a hysteresis threshold to achieve weight stability. Upper limit of rate of change. Limit the variation of fusion weights between adjacent time steps to prevent drastic fluctuations due to transient disturbances. When the calculated change in fusion weights exceeds the upper limit of the rate of change, the actual fusion weights used are limited to the allowable variation range of the fusion weights from the previous time step.

[0144] The hysteresis threshold setting introduces a lag in the switching of fusion weights. Significant switching of fusion weights is only permitted when the uncertainty difference between the two signal sources consistently exceeds the hysteresis threshold T. This hysteresis threshold mechanism prevents frequent switching of fusion weights when the uncertainties of the two signal sources are close, thus improving the stability of the fusion output.

[0145] like Figure 7 As shown, when visual uncertainty or external signal uncertainty When the preset upper limit is exceeded, the system is forced to enter single-source mode and outputs a degradation flag. In single-source mode, only signal sources with uncertainties within the limit are used for equivalent imbalance estimation. The degradation flag is used to notify the host system that the system is currently in a degradation operation state.

[0146] Figure 7 The form of the adaptive loss function is shown in the figure, expressed as: This loss function is used for weight learning during the training process. Residual learning steps. The residuals are used to learn the physical heuristic baseline and the actual output. The residual consistency term is represented as follows: This is used to ensure output stability between adjacent frames.

[0147] The fusion interpretation output includes fusion weights. Visual uncertainty Sensor uncertainty Trustworthiness status and selection reasons. Fusion weights. Reflects the contribution ratio of the two signal sources to the fusion result at the current moment. Visual uncertainty. Reflects the reliability of visual estimation. Sensor uncertainty. It reflects the reliability of external signal estimation. The credibility status output indicates the credibility level of the current fusion result, including three levels: credible, doubtful, and unavailable. The selection reason output shows the basis for the current fusion weight configuration, including information such as which signal source has higher uncertainty, whether it is in degradation mode, and whether a hysteresis mechanism has been triggered. The fusion interpretation output facilitates diagnosis by the higher-level system and operations and maintenance personnel, making the system's operating status traceable and interpretable.

[0148] After completing multi-source adaptive fusion, conflict diagnosis is performed on the visual equivalent imbalance and the external signal equivalent imbalance, and a degraded state machine strategy is executed based on the diagnosis results. (Refer to...) Figure 8 The complete workflow diagram of the conflict diagnosis and degradation state machine shows the system's diagnostic logic and multi-level degradation protection mechanism when visual signals conflict with external signals.

[0149] like Figure 8 As shown, the conflict diagnosis and degradation state machine strategy includes conflict triggering condition judgment, conflict classification output, and degradation state switching. When And after N frames, a conflict diagnosis is triggered. Among the conflict triggering conditions, Where N is the preset conflict threshold, and N is the duration frame count threshold. When the visual equivalent imbalance... Equivalent Imbalance to External Signal The differences between them exceed the conflict threshold If this difference persists across N consecutive frames, the system determines that a conflict exists between the two signal sources and triggers the conflict diagnosis process. In the sliding window detection method, conflict diagnosis is also triggered when the proportion of conflicting frames within the sliding window exceeds a preset threshold.

[0150] Continue to refer to Figure 8 After a conflict diagnosis is triggered, the system categorizes and outputs the causes of the conflict. The conflict classification output includes three categories: visual distortion, abnormal external signals, and sudden changes in operating conditions.

[0151] Visual distortion types include overexposure, enhanced reflection, fringe recurrence, ROI shift, and blurring. Overexposure refers to the presence of large saturated areas in the image, resulting in the loss of wrinkle texture information. Enhanced reflection refers to the increased specular reflection component of the material surface, leading to increased highlight interference. Fringe recurrence refers to the reappearance of interference fringes after suppression, interfering with wrinkle recognition. ROI shift refers to the offset of the region of interest relative to the material coordinate system, resulting in inaccurate wrinkle localization. Blurring refers to a decrease in image sharpness, resulting in unclear wrinkle boundary features.

[0152] like Figure 8 As shown, external signal anomalies include drift, saturation, disconnection, and noise anomalies. Drift refers to a slow baseline drift in the external sensor signal, causing a systematic bias in the estimation of the equivalent imbalance quantity. Saturation refers to the external sensor signal reaching its upper or lower limit of range, failing to accurately reflect the actual physical quantity changes. Disconnection refers to a break in the communication link of the external sensor or data packet loss, resulting in missing or delayed data. Noise anomalies refer to an abnormally high noise level in the external sensor signal, leading to a deterioration in signal quality.

[0153] Abrupt changes in operating conditions include joint passage, velocity abrupt change, and contamination coverage. Joint passage refers to a sudden change in the material surface characteristics as the material joint passes through the detection area. Velocity abrupt change refers to a sudden change in the material's operating speed, leading to a mismatch between image acquisition and processing parameters. Contamination coverage refers to the presence of contaminants on the material surface, interfering with wrinkle recognition.

[0154] The degradation state machine starts running from the top normal state. Based on the conflict diagnosis results, it switches between the following states: NORMAL state indicates normal fusion output, FREEZE state indicates frozen output as the last reliable value, VIS_ONLY state indicates visual estimation only, and SEN_ONLY state indicates external signal estimation only.

[0155] The NORMAL state represents the system's normal operating state. In this state, the system performs normal multi-source adaptive fusion and outputs the fused equivalent imbalance. When conflict diagnosis is triggered and the diagnosis result indicates that both signal sources are unreliable, the system switches from the NORMAL state to the FREEZE state. In the FREEZE state, the system freezes the output to the last reliable value, that is, it retains the last reliable estimate of the equivalent imbalance before the conflict occurred, and at the same time sends an alarm signal to the upper-level system.

[0156] When the conflict diagnosis indicates that the external signal is abnormal but the visual signal is reliable, the system switches from the NORMAL state to the VIS_ONLY state. In the VIS_ONLY state, the system only uses visual estimation to output the equivalent imbalance. The system does not use external signal estimation. When the conflict diagnosis indicates visual distortion and the external signal is reliable, the system switches from the NORMAL state to the SEN_ONLY state. In the SEN_ONLY state, the system only uses the external signal to estimate the equivalent output imbalance. Visual estimation is not used.

[0157] Continue to refer to Figure 8 The degradation state machine also includes a RAISE_THRESHOLD state, in which the output is marked as suspicious, and the host system adopts a conservative strategy. When the conflict diagnosis results indicate that the reliability of both signal sources has decreased to some extent, but has not yet reached the point of being completely unusable, the system enters the RAISE_THRESHOLD state. In this state, the system continues to output the fused equivalent imbalance quantity, but simultaneously outputs a suspicious flag, notifying the host system that the confidence level of the current estimation result has decreased. Upon receiving the suspicious flag, the host system adopts a conservative strategy to reduce its dependence on the estimation result of the equivalent imbalance quantity and increase the safety margin.

[0158] After the collision disappears and remains for M frames, it recovers from the degraded state to the NORMAL state. Collision disappearance refers to the visual equivalent imbalance. Equivalent Imbalance to External Signal The difference between them fell back to the conflict threshold. Below, M represents the recovery frame threshold. When the conflict-free state persists for M frames, the system determines that the conflict has been stably eliminated and recovers from the current degraded state to the NORMAL state, restoring normal multi-source adaptive fusion output. Setting the recovery frame threshold M prevents the system from frequently switching states due to transient fluctuations, improving the stability of the state machine operation.

[0159] Continue to refer to Figure 8 The diagram shows multiple state transition paths. From the normal state, the path proceeds down to the decision node, where a decision is made. Is there any abnormal error? If the result is no, the process moves to the right to the conflict diagnosis step. If the result is yes, it continues to the node determining whether the diagnosis was successful. If the diagnosis is successful, it returns to the normal state via the recovery path. If the diagnosis is unsuccessful, it enters the degraded state. From the degraded state, it continues to the node determining the recovery conditions. If the recovery conditions are met, it returns to the diagnosis node for re-evaluation; if the recovery conditions are not met, it enters the minimized state.

[0160] Each state transition records the cause code and duration in the degraded state machine. The cause code identifies the specific reason for the state transition, including the conflict type (visual distortion, abnormal external signal, sudden change in operating conditions) and the specific subtype of the anomaly. The duration records the length of time the system remains in each state. The recording of cause codes and durations provides data support for system operation and maintenance and fault analysis, making the system's operational history traceable. By analyzing the distribution of cause codes and the statistics of durations during state transitions, operation and maintenance personnel can identify system weaknesses and optimize system configuration and maintenance strategies.

[0161] The software implementation and hardware deployment scheme of the method for reflective robust visual recognition and equivalent imbalance estimation of continuous strip material wrinkles described in this invention are deployed on an edge industrial control computer or embedded GPU platform for real-time processing. The edge industrial control computer possesses industrial-grade reliability and environmental adaptability, suitable for industrial environments such as factory workshops. The embedded GPU platform provides parallel computing capabilities, accelerating the inference process of neural network models. In the edge deployment architecture, image acquisition, preprocessing, wrinkle recognition, parameterization, equivalent imbalance estimation, and multi-source fusion are all completed locally, reducing data transmission latency and meeting the real-time requirements of online detection.

[0162] Model lightweighting techniques are applied to optimize neural network models, including pruning, quantization, and knowledge distillation. Pruning reduces the number of parameters and computational cost by removing redundant connections or channels in the neural network. Structured pruning removes entire convolutional channels or layers, while unstructured pruning removes individual weight connections. Pruned models maintain recognition accuracy while improving inference speed and reducing memory usage. Quantization converts floating-point weights and activation values ​​in the neural network into low-bit fixed-point representations. INT8 quantization converts 32-bit floating-point numbers to 8-bit integers, and INT4 quantization converts them to 4-bit integers. Quantized models achieve significant inference acceleration on hardware platforms supporting fixed-point arithmetic while reducing memory bandwidth requirements. Knowledge distillation transfers knowledge from a large teacher model to a small student model. The teacher model is the original model with a large number of parameters and high accuracy, while the student model is a lightweight model with fewer parameters and faster inference speed. Through distillation training, the student model learns the output distribution and intermediate feature representations of the teacher model, achieving model lightweighting while maintaining high recognition accuracy.

[0163] Resolution adaptation and ROI inference are used to ensure frame rate requirements are met. Resolution adaptation dynamically adjusts the input image resolution based on the current computational load and frame rate requirements. When computational resources are sufficient, a higher input resolution is used to obtain more refined wrinkle recognition results. When computational resources are limited or the frame rate decreases, the input resolution is reduced to decrease computational load, ensuring that the processing frame rate meets the real-time requirements of online detection. ROI inference only performs neural network inference on the region of interest, skipping the processing of background areas. In the detection scenario of continuous strip materials, wrinkle defects are concentrated on the material surface area, and the background area outside the material boundary does not need to be wrinkle recognized. Through ROI inference, computational resources are concentrated on the effective area, and inference efficiency is improved. The combination of resolution adaptation and ROI inference achieves a balance between frame rate and accuracy under different hardware platforms and operating conditions.

[0164] In low-computing-power scenarios, a combination of traditional candidate augmentation and a lightweight classifier is used as a degradation implementation. Traditional candidate augmentation methods include image processing techniques such as directional filtering, structure tensor, and wavelet energy, which have low computational complexity on CPU platforms. Lightweight classifiers include traditional machine learning classifiers such as support vector machines, random forests, and gradient boosting trees, which have faster inference speeds than deep neural networks. In the degradation implementation, traditional candidate augmentation methods extract candidate regions and feature descriptors for wrinkles, and the lightweight classifier classifies the candidate regions. The degradation implementation has lower recognition accuracy than deep learning methods, but it can meet the basic wrinkle detection requirements on low-computing-power platforms, ensuring that the system still has detection capabilities when hardware resources are limited.

[0165] Cross-material and cross-lighting adaptation utilizes unlabeled data for self-supervised domain adaptation. In practical applications, continuous strip materials are diverse, including films, paper, metal foils, and fabrics, with varying surface properties and wrinkle morphologies. Lighting conditions vary across different production lines and time periods, causing changes in image brightness, contrast, and color distribution. Self-supervised domain adaptation leverages unlabeled data from the target domain, adjusting the model's feature representation through self-supervised learning tasks to adapt the model to new material types or lighting conditions. Self-supervised learning tasks include image reconstruction, contrastive learning, and rotation prediction. These tasks do not rely on manually labeled wrinkle tags and can utilize large amounts of unlabeled production data for model adaptation.

[0166] Online calibration performs small-step updates only on high-confidence samples to maintain model accuracy. High-confidence samples are those whose model output confidence exceeds a preset threshold; these samples have high reliability in their identification results. During online calibration, only high-confidence samples are used to update model parameters, avoiding the negative impact of noisy labels from low-confidence samples on the model. Small-step updates refer to using a small learning rate to update parameters, limiting the magnitude of changes to model parameters in a single update. This small-step update strategy prevents drastic changes in the model due to online calibration, maintaining the model's performance on the original data distribution.

[0167] Any process or method described in the flowcharts of the embodiments of the present invention or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the present invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0168] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for robust visual recognition and equivalent imbalance estimation of reflective folds in continuous strip materials, characterized in that, include: Acquire image frames of continuous strip-shaped materials and simultaneously obtain metadata; The image frame is subjected to reflection suppression processing to suppress the highlight or specular reflection components in the image, resulting in a reflection-suppressed image; The image after reflection suppression is subjected to interference fringe suppression processing to suppress periodic background texture, resulting in a fringe-suppressed image. Motion compensation and region of interest adaptive locking are performed on the image after stripe suppression. The image is aligned to the material coordinate system and the region of interest is dynamically updated to obtain the preprocessed image. The preprocessed image is used to identify, segment and classify wrinkles using a wrinkle recognition network, and outputs wrinkle mask, wrinkle category, pseudo-wrinkle mask and confidence score. The wrinkle recognition network includes four output branches: segmentation head, classification head, pseudo-wrinkle sub-head and uncertainty head. Extract the wrinkle parameterization features from the wrinkle mask to obtain the parameterization vector; Based on the parameterized vector and the metadata, the visual equivalent imbalance is estimated using a regression model with physical prior constraints, and the estimation uncertainty is output. The estimation of the visual equivalent imbalance employs a combination of physically heuristic baseline and residual learning, expressed as follows: ； Among them, the For visually equivalent imbalance quantity, For baseline estimation based on physical models, For residual learning function; When external sensor signals are present, the visual equivalent imbalance and the external signal equivalent imbalance are subjected to multi-source adaptive fusion to obtain a fused equivalent imbalance. The multi-source adaptive fusion includes: Computational visual uncertainty and external signal uncertainty ; calculate the fusion weights based on the visual uncertainty and the external signal uncertainty. ,in: ； Calculate the equivalent imbalance in fusion based on the fusion weights: ； Among them, the To integrate equivalent unbalanced quantities, For visually equivalent imbalance quantity, For external signals, the equivalent imbalance quantity; and, For the aforementioned visual equivalent imbalance Equivalent imbalance quantity to the external signal Conflict diagnosis is performed, and a degradation state machine strategy is executed based on the diagnosis results. The conflict diagnosis and degradation state machine strategy include: when And if it continues for N frames, a conflict diagnosis is triggered. Where N is the preset conflict threshold, and N is the duration frame threshold. Based on the conflict diagnosis results, the system will switch between the following states: The NORMAL state indicates normal fusion output. In this state, normal multi-source adaptive fusion is performed, and the output is the equivalent imbalance of the fusion. ; When a collision diagnosis is triggered and the diagnosis result indicates that both signal sources are unreliable, the system switches from the NORMAL state to the FREEZE state. The FREEZE state means that the output is frozen as the last reliable value, that is, the last reliable fused equivalent imbalance before the collision occurred is preserved. ; When the conflict diagnosis result indicates that the external signal is abnormal but the visual signal is reliable, the system switches from the NORMAL state to the VIS_ONLY state. The VIS_ONLY state indicates that only visual estimation is used, that is, only the visual equivalent imbalance is output. ; When the conflict diagnosis result indicates visual distortion but the external signal is reliable, the system switches from the NORMAL state to the SEN_ONLY state. The SEN_ONLY state indicates that only the external signal estimation is used, that is, only the equivalent imbalance of the external signal is output. ; And, after the conflict disappears and the M-frame is maintained, it returns from the degraded state to the NORMAL state.

2. The method according to claim 1, characterized in that, The reflection suppression treatment includes at least one of the following: Generate a specular mask and perform local texture reconstruction on the mask area; The specular component and diffuse reflection component are separated by a polarization dual-channel separation; and Generate unexposed texture maps by multi-exposure fusion.

3. The method according to claim 1, characterized in that, The interference fringe suppression process includes at least one of the following: Suppressing stripe bands by estimating the main frequency band in the frequency domain and using an adaptive notch filter; The image is decomposed into stripe components and defect components, and the defect components are preserved. as well as When a texture exhibits a consistent direction and frequency across the entire frame and remains stable across frames, it is identified as a background texture and is weakened.

4. The method according to claim 1, characterized in that, The wrinkle parameterization features include at least two of the following: Wrinkle direction , obtained through statistical analysis of the principal directions of the structure tensor or the tangents of the skeleton; Wrinkle wavelength The frequency is obtained by Fourier transform or wavelet transform along the normal profile. Wrinkle amplitude This is obtained through the peak-valley difference or gradient energy of the profile. wrinkle density This is obtained by measuring the number of stripes per unit area or the number of energy peaks. wrinkle area percentage ; Persistence across frames ; Edge offset ; as well as, Complexity This includes the number of branches, the number of clusters, or the complexity of connected components.

5. The method according to claim 1, characterized in that, The regression model with the physical prior constraints satisfies at least one of the following constraints: Monotonic constraint: When the wrinkle index When it increases, the equivalent imbalance quantity No decrease; Boundary constraints: equivalent imbalance quantity The range of values ​​is limited by preset boundaries; and, Consistency constraint: Under the same operating conditions, the changes in wrinkle direction and wavelength are equivalent to the amount of imbalance. The relationship remains stable.

6. The method according to claim 1, characterized in that, Also includes: Based on the preprocessed image, a candidate enhancement map is constructed, which is generated by at least one of directional filtering, structure tensor, or wavelet energy. as well as, The candidate enhanced image and the preprocessed image are combined into a dual-stream input for wrinkle recognition, segmentation and classification.

7. The method according to claim 1, characterized in that, Also includes: The results of wrinkle recognition, segmentation and classification are confirmed to be consistent in time. When the wrinkle morphology is consistent in N consecutive frames and the confidence level exceeds a preset threshold, the wrinkle state is confirmed as a confirmed wrinkle state. as well as, Output pixel-level uncertainty Based on image sharpness, exposure quality, reflectance intensity, stripe intensity, region of interest stability, and model output uncertainty, the overall uncertainty is calculated. .

8. The method according to claim 1, characterized in that, Also includes: Construct a wrinkle index based on the parameterized vector. The wrinkle index By integrating the proportion of wrinkle area fold density fold amplitude and cross-frame persistence At least two of them were obtained; and, Risk quantity is constructed by combining operating condition information and sensitive windows. The sensitive window includes a connector window or a highly reflective window.

9. The method according to claim 1, characterized in that, Set an upper limit on the rate of change for the fusion weights. ,satisfy And set a hysteresis threshold to achieve weight stability.

Images