A few-shot industrial visual anomaly detection method based on dynamic weight multi-feature reconstruction

Abstract

The application discloses a few-sample industrial visual anomaly detection method based on dynamic weight multi-feature reconstruction, and belongs to the technical fields of industrial machine vision, image processing and intelligent anomaly detection. The application can reduce the false detection caused by normal state changes, reflection, blur, contour loss and transient jitter under the conditions of limited number of abnormal reference samples, variable normal target state and complex on-site interference by using the normal sample training state coding constraint texture, edge and structure multi-branch normal state reconstruction model, combining the cross-influence dynamic weight of controlled quality disturbance verification set calibration, cross-frame time sequence consistency analysis and the determination of the certainty of the original feature matching of the limited abnormal standard sample. Meanwhile, the application can avoid the overfitting caused by training a complex abnormal model with a small number of abnormal samples, and the training distribution bias and reconstruction error propagation possibly caused by directly reusing the normal encoder features, so that the accuracy, stability and interpretability of the industrial visual anomaly detection are improved.

Description

Technical Field

[0001] This invention relates to the fields of industrial machine vision, image processing, and intelligent anomaly detection. Specifically, it relates to a method for industrial visual anomaly detection that utilizes normal samples to train a multi-branch normal state reconstruction model with state coding constraints on texture, edge, and structure, under the condition that the number of abnormal reference samples is limited and they do not participate in model training. This method combines image quality-driven dynamic weights, cross-frame temporal consistency analysis, and feature matching of limited anomaly standard samples. Background Technology

[0002] Industrial production sites typically require visual inspection to identify anomalies such as contamination, cracks, gaps, missing parts, misalignment, deformation, or foreign objects on target surfaces. Existing supervised defect detection methods usually rely on a sufficient number of anomaly samples with complete category labeling. However, industrial anomalies are characterized by low occurrence probability, diverse types, high collection costs, and the difficulty in pre-enumerating new anomaly morphologies, making it difficult to obtain a large-scale training set covering all anomaly states. The term "few samples" in this specification does not refer to the few-sample learning in machine learning literature that relies on a small number of labeled samples for meta-learning or rapid adaptation for each category. Rather, it refers to a limited number of available anomaly reference samples, which are not used in the normal state reconstruction model training but only serve as feature matching benchmarks in the secondary anomaly determination stage.

[0003] Reconstruction-based anomaly detection methods based on normal samples learn the representational patterns of normal images and judge anomalies based on the differences between the image to be detected and the normal reconstruction results, thus reducing the dependence on training data with anomaly samples. However, industrial targets may present multiple normal appearances under different camera positions, poses, operating stages, and working conditions; if a single normal reference is used for reconstruction, it is easy to misjudge normal state changes as anomalies.

[0004] In addition, industrial environments also present interference such as glare, motion blur, contour occlusion, and equipment vibration. The reliability of texture, edge, and structural information within the same image frame is not constant, and fixed-weight fusion can easily lead to the interference of feature dimensions dominating the anomaly score. Judging solely based on the reconstruction error of a single frame may also misclassify instantaneous highlights, noise, or jitter as persistent anomalies.

[0005] On the other hand, while a small number of confirmed anomalous samples are insufficient to train a stable anomaly classification model, they can serve as a secondary matching benchmark. Existing solutions often directly use a limited number of anomalous samples to train new anomaly models, which is prone to overfitting and weakens the technical approach of training primarily with normal samples in scenarios where anomalous samples are scarce. Therefore, there is a need for an industrial visual anomaly detection method that can complete the main model training with normal samples while making reasonable use of a limited number of standard anomalous samples to suppress false detections. Summary of the Invention

[0006] To address the problems of high false detection rate and insufficient reliability of anomaly confirmation in existing industrial visual anomaly detection methods under conditions of limited number of abnormal samples, changes in the state of normal targets, and interference from complex industrial environments, this invention provides a small-sample industrial visual anomaly detection method based on dynamic weighted multi-feature reconstruction.

[0007] The technical solution of this invention is as follows: A method for detecting industrial visual anomalies using a small number of samples based on dynamic weighted multi-feature reconstruction, comprising the following steps: S1: Initialize and build the normal sample library and the abnormal database.

[0008] The normal sample library is used to store normal industrial images that have been manually confirmed to be free of target anomalies, and is used to train a multi-branch normal state reconstruction model and establish a normal state representation.

[0009] The anomaly database is used to uniformly manage anomaly-related data generated during the detection process, including an anomaly standard library, a confirmed anomaly image set, and an anomaly suspected library.

[0010] Furthermore, the anomaly standard library is used to store the standard anomaly regions and their characteristic information corresponding to the representative standard anomaly samples that have been manually confirmed, serving as the feature matching benchmark for the subsequent secondary anomaly judgment stage; the confirmed anomaly image set is used to store the anomaly samples that have been confirmed after the secondary anomaly judgment; and the suspected anomaly library is used to store the suspected anomaly samples that have not yet been confirmed as anomalies after the secondary anomaly judgment and require manual review.

[0011] Furthermore, before the standard abnormal samples are stored in the abnormal standard library, the abnormal regions in the standard abnormal samples are manually labeled, their size is normalized, and their orientation is corrected. The texture features, edge features, and structural features of the abnormal regions are extracted and stored in the abnormal standard library in association with the sample index of the corresponding samples, for use in step S7 when performing secondary abnormality determination.

[0012] Furthermore, the texture features adopt local binary mode features, the edge features adopt directional gradient histogram features, and the structural features adopt Hu invariant moments and contour geometric features.

[0013] Furthermore, the aforementioned anomaly standard library is only initialized and periodically reviewed and updated manually, and does not participate in the training of the multi-branch normal state reconstruction model.

[0014] S2: Establish a multi-branch normal state reconstruction model based on the normal sample library.

[0015] Step S2.1: Extract and preprocess the target region from the normal samples in the normal sample library. The target region extraction method is to use a preset region of interest for cropping under a fixed position; the preprocessing includes background suppression, position registration, size normalization, and pixel normalization. Step S2.2: Based on the camera acquisition points, equipment operation stage and industrial operating parameters, the normal samples are initially grouped, and K-means clustering is performed using target contour, target pose and overall appearance features. A state number is assigned to each normal state category, and the state number is mapped to the corresponding state code. Step S2.3: Construct and train a multi-branch normal state reconstruction model.

[0016] The preprocessed normal samples obtained in step S2.1 and the corresponding state codes determined in step S2.2 are used together as model training data to construct a multi-branch normal state reconstruction model. This multi-branch normal state reconstruction model is used to learn the texture, edge, and structural representations of the target object under different normal states. In subsequent step S4, based on the target region in the current frame and its corresponding normal state code, it generates the texture, edge, and structural reconstruction results that the target region in the current frame should have under the corresponding normal state.

[0017] The multi-branch normal state reconstruction model includes a shared image encoder, a state coding fusion module, and parallel texture reconstruction branches, edge reconstruction branches, and structure reconstruction branches. The shared image encoder and state coding fusion module are used by the three reconstruction branches, each with independent network parameters.

[0018] The shared image encoder includes multiple levels of convolutional coding units connected in sequence. Each level of convolutional coding unit includes a convolutional layer, a batch normalization layer, and an activation layer. Adjacent convolutional coding units reduce the spatial size of the feature map and increase the number of feature channels through downsampling operations, thereby extracting multi-scale image features of preprocessed normal samples and obtaining the encoder bottleneck features.

[0019] The state encoding fusion module is used to fuse the state encoding obtained in step S2.2 with the encoder bottleneck features. Specifically, the state encoding is mapped to a state condition vector through a fully connected layer, and the state condition vector is expanded in the spatial dimension to form a state condition feature map with the same spatial size as the encoder bottleneck features; the state condition feature map and the encoder bottleneck features are concatenated along the channel dimension to obtain the state fusion feature.

[0020] The state fusion feature simultaneously includes image information of the input normal sample and condition information of the corresponding normal state, and is respectively input to the texture reconstruction branch, edge reconstruction branch and structure reconstruction branch.

[0021] Furthermore, the texture reconstruction branch, edge reconstruction branch, and structure reconstruction branch each include a multi-level upsampling convolutional unit and an output layer connected in sequence; the multi-level upsampling convolutional unit is used to recover the spatial dimensions of the state fusion features step by step, and the output layer is used to convert the recovered features into the reconstruction results of the corresponding branch. The three reconstruction branches share the shared image encoder and state encoding fusion module, and each has independent network parameters.

[0022] The texture reconstruction branch takes state fusion features as input and outputs the texture reconstruction result after multi-level upsampling convolution processing. The texture reconstruction result is a normalized color image with the same spatial size as the preprocessed normal sample, which is used to represent the color distribution and local surface texture that the target to be detected should have in the corresponding normal state.

[0023] The edge reconstruction branch takes state fusion features as input and outputs the edge reconstruction result after multi-level upsampling convolution processing. The edge reconstruction result is a single-channel edge map with the same spatial size as the preprocessed normal sample, which is used to represent the contour boundary, local edge and edge continuity that the target to be detected should have in the corresponding normal state.

[0024] The structural reconstruction branch takes state fusion features as input and outputs structural reconstruction results after multi-level upsampling convolution processing. The structural reconstruction results are single-channel distance transformation maps with the same spatial size as the preprocessed normal samples, which are used to represent the overall contour, region shape, and spatial relationship of the target's internal pixels relative to the contour boundary that the target should have in the corresponding normal state.

[0025] During training, a random rectangular mask and Gaussian noise are applied to the preprocessed normal samples obtained in step S2.1 to obtain perturbed normal samples. The perturbed normal samples and their corresponding state codes are input into the multi-branch normal state reconstruction model. The unperturbed normal samples, the edge map generated from the unperturbed normal samples, and the distance transformation map generated from the target contour mask are used as the training targets of the three branches, respectively. This suppresses the direct copying of the input image by the multi-branch normal state reconstruction model and enhances the learning of normal structural patterns.

[0026] Loss from texture reconstruction Edge reconstruction loss and structural reconstruction loss Jointly train the three reconstruction branches and calculate the total training loss. The network parameters of the shared image encoder, the state encoding fusion module, and the three reconstruction branches are updated to obtain the trained multi-branch normal state reconstruction model.

[0027] The total training loss For texture reconstruction loss Edge reconstruction loss Structural remodeling loss Weighted sum: Among them, the texture reconstruction loss is composed of the absolute error loss and structural similarity loss used in the texture reconstruction branch, the edge reconstruction loss is composed of the binary cross-entropy loss and Dice loss used in the edge reconstruction branch, and the structural reconstruction loss is composed of the absolute error loss and Dice loss used in the structural reconstruction branch. , and These are the fixed loss coefficients used during the training phase. The fixed loss coefficients during the training phase are independent of the dynamic weights determined based on image quality during the online detection phase.

[0028] The trained multi-branch normal state reconstruction model is used as the output of step S2, which is then used by step S4 to generate texture reconstruction results, edge reconstruction results, and structural reconstruction results based on the detection region of the current frame and its corresponding state encoding.

[0029] S3: Acquire images of the industrial site and calculate image quality parameters.

[0030] Image frames of the target to be inspected in the industrial field are continuously acquired using an RGB camera. The region to be inspected is extracted from the current image frame, and background suppression, position registration, size normalization, and pixel normalization are performed to obtain a standardized region to be inspected.

[0031] Image quality parameters are calculated based on the standardized region to be detected, including the degree of reflectivity. degree of ambiguity and structural visibility .

[0032] Among them, reflectivity Used to indicate the degree to which an image region is affected by highlight reflections, calculated based on the proportion of pixels in the region to be detected whose normalized brightness is greater than a highlight threshold; blur level. The degree to which image edge information is affected by blur is used to represent the degree of blurring, calculated based on the normalized result of the Laplacian response variance relative to the calibration interval of the sharp image; structural visibility. It is used to represent the completeness of the target structure contour, and is calculated based on the ratio of the current contour length to the corresponding normal state reference contour length. and The larger the value, the more severe the reflection and blurring, respectively. The larger the value, the more complete the structure.

[0033] S4: Perform multi-feature reconstruction based on state coding and calculate anomaly error.

[0034] Specifically, the state code of the current region to be detected is first extracted according to the method in step S2.2. The current region to be detected and its corresponding state code are then input into the multi-branch normal state reconstruction model established in step S2 to obtain the texture reconstruction result, edge reconstruction result, and structure reconstruction result, respectively.

[0035] The actual texture representation, actual edge representation, and actual structure representation are compared position-by-position with the corresponding reconstruction results to obtain the texture error map. Edge error map and structural error diagram And further calculate the texture error Edge error and structural errors .

[0036] S5: Dynamically adjust the feature error weights based on image quality parameters to obtain the weighted feature anomaly score and candidate anomaly regions for the current frame.

[0037] Step S5.1: Based on the reflectivity calculated in step S3 degree of ambiguity and structural visibility Construct the quality degradation vector for the current frame. : , in, Indicates the degree of reflectivity , Indicates the degree of ambiguity , Indicates the degree of invisibility of the structure ; Step S5.2: Divide normal verification samples from the normal sample library, and construct a quality perturbation verification set based on the normal verification samples. Specifically, apply multiple levels of reflection perturbation, blur perturbation, and contour occlusion perturbation to the normal verification samples to simulate reflection, blur, and structural invisibility in industrial settings, respectively.

[0038] Input the normal verification samples without quality perturbation and the normal verification samples with different levels of quality perturbation into the multi-branch normal state reconstruction model established in step S2, respectively, calculate the texture error, edge error and structural error according to the method described in step S4, and count the average relative error increment of each quality degradation factor on each feature branch.

[0039] Normalizing the three average relative error increments corresponding to the same feature branch yields the quality-branch influence coefficient: in For feature branch index, These represent texture, edge, and structural branches, respectively. Index of factors contributing to quality degradation These represent reflection, blur, and invisible structure, respectively. Indicates the first The first quality degradation factor on the second The average relative error increment caused by each feature branch This represents a preset positive number used to prevent the denominator from being zero. Indicates the first The first quality degradation factor on the second The influence coefficient of the reliability of each feature branch.

[0040] The mass-branch influence coefficient matrix is ​​composed of the mass-branch influence coefficients of each mass: Step S5.3: Based on the quality degradation vector of the current frame obtained in step S5.1 The quality-branch influence coefficient matrix obtained in step S5.2 The dynamic weights of the three branches—texture, edge, and structure—are calculated separately.

[0041] Specifically, first calculate according to the following formula: in, Indicates the first The reliability weights of each feature branch, This represents the preset minimum reliability weight lower limit, and This is used to prevent the weight of any feature branch from decreasing to zero; The three reliability weights are normalized to obtain the dynamic weights. 、 and : Step S5.4: Based on the texture error obtained in step S4 Edge error Structural errors and the dynamic weights obtained in step S5.3 The weighted feature anomaly score of the current frame is calculated according to the following formula. : .

[0042] Step S5.5: Based on the texture error map obtained in step S4 Edge error map and structural error diagram The comprehensive anomaly heatmap is obtained by performing position-by-position weighted fusion according to the following formula. : Candidate abnormal regions are obtained through threshold segmentation and connected component processing.

[0043] S6: Perform temporal consistency analysis on candidate abnormal regions and make preliminary anomaly determination by combining weighted feature anomaly scores.

[0044] The temporal consistency analysis involves mapping the candidate anomaly region of the current frame and the candidate anomaly region of the historical frames within a preset time window to a unified coordinate system, and calculating the intersection-union ratio between the current candidate anomaly region and the historical candidate anomaly region. When the weighted feature anomaly score of the current frame is greater than the anomaly score threshold, and the candidate anomaly region of the current frame satisfies the intersection-union ratio constraint condition within the preset time window, the candidate anomaly region of the current frame that satisfies the intersection-union ratio constraint condition is determined as a preliminary anomaly region; otherwise, the corresponding candidate anomaly region is determined as a non-continuous anomaly response and is not entered into the anomaly standard library matching process.

[0045] S7: Perform secondary anomaly determination based on the anomaly standard library in the anomaly database.

[0046] The initial abnormal region obtained in step S6 is normalized in size and corrected in orientation, and texture features, edge features and structural features are extracted using the same feature extraction method as the standard abnormal sample in step S1.

[0047] The texture, edge, and structural features obtained from the initial anomaly region are matched with standard anomaly regions in the anomaly standard library to obtain texture matching distance, edge matching distance, and structural matching distance. The texture matching distance is calculated using chi-square distance, the edge matching distance using 1 minus cosine similarity, and the structural matching distance using Euclidean distance normalized to the variance of each dimension. The three types of matching distances are then normalized to obtain texture matching difference, edge matching difference, and structural matching difference. Using the dynamic weights determined in step S5, the weighted feature difference score corresponding to the i-th standard anomaly region is calculated. in, Indicates the first i Weighted feature difference scores corresponding to each standard abnormal region; Indicates texture matching differences; Indicates edge matching differences; Indicates poor structural matching; 、 and The dynamic weights are obtained in step S5.

[0048] Take the minimum value among the weighted feature difference scores corresponding to all standard outlier regions in the outlier standard library: in, N This indicates the number of standard anomaly regions in the anomaly standard library.

[0049] when When the difference is less than or equal to a preset difference threshold, the current preliminary anomaly is determined as a confirmed anomaly and stored in the confirmed anomaly image set in the anomaly database; when... When the difference exceeds the preset threshold, the current preliminary anomaly will be stored in the suspected anomaly database for manual review.

[0050] In this invention, "few samples" refers to a limited number of abnormal samples available for anomaly reference and verification, not to using a small number of labeled samples for each anomaly category for meta-learning or rapid model adaptation. The multi-branch normal state reconstruction model is trained using only normal samples; abnormal standard samples do not participate in the model training, but are only used as a limited number of feature matching benchmarks with controllable sources in the secondary anomaly determination stage.

[0051] This invention employs a two-stage approach combining learning-based normal reconstruction and deterministic original feature matching. The first stage aims to locate regions deviating from the normal state using a model trained solely on normal samples. The second stage aims to verify whether these regions possess sufficient feature similarity to a limited set of anomaly standard samples, rather than retraining the anomaly classification model. Since the shared encoder optimizes only for normal state reconstruction, its bottleneck features are not trained for similarity between anomaly samples. Directly using encoder features or reconstruction residuals for secondary matching may compound state selection and reconstruction errors. Furthermore, a limited set of anomaly standard samples is insufficient to support stable network fine-tuning. Therefore, after size normalization and principal direction correction, this invention uses original texture, edge, and structural features—which require no anomaly sample training, have low dimensionality, and possess clear physical meaning—for secondary matching. These features correspond to the three reconstruction branches of the first stage, reducing the risk of overfitting and error propagation under conditions of limited anomaly sample numbers.

[0052] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention utilizes normal samples to train a multi-branch normal state reconstruction model with state encoding constraints, incorporating texture, edge, and structural elements. Combined with dynamic weights of cross-influence calibrated using a controlled quality perturbation validation set, cross-frame temporal consistency analysis, and deterministic original feature matching of limited abnormal standard samples, it can reduce false detections caused by changes in normal state, reflections, blurring, missing contours, and transient jitter, even under conditions of limited abnormal reference samples, variable normal target states, and complex on-site interference. Simultaneously, it avoids overfitting caused by training complex abnormal models with a small number of abnormal samples, and the training distribution bias and reconstruction error propagation that may result from directly reusing normal encoder features. This improves the accuracy, stability, and interpretability of industrial visual anomaly detection. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall process of the industrial visual anomaly detection method of the present invention.

[0054] Figure 2 This is a schematic diagram of the logical structure of the normal sample database and the abnormal database.

[0055] Figure 3 A schematic diagram illustrating the process of classifying and coding normal states.

[0056] Figure 4 A schematic diagram of the structure of the multi-branch normal state reconstruction model.

[0057] Figure 5 A schematic diagram of the training process for reconstructing a multi-branch normal state model.

[0058] Figure 6 This is a schematic diagram of the online reconstruction, dynamic weight fusion, and candidate abnormal region generation process.

[0059] Figure 7 This is a schematic diagram of the cross-frame timing consistency analysis process.

[0060] Figure 8 This is a schematic diagram illustrating the feature matching process of the anomaly standard library and the workflow of the anomaly database sub-library. Detailed Implementation

[0061] The present invention will be further described below with reference to specific embodiments. This embodiment describes the sample library construction, normal state division, multi-branch normal state reconstruction model training, online image processing, dynamic weight calculation, temporal consistency analysis, and secondary judgment of the anomaly standard library in the order of steps S1 to S7 in the invention description. The following scenario settings and parameters are used to illustrate a preferred method that can be repeatedly implemented, and do not constitute a limitation on the scope of protection of the present invention.

[0062] Example 1: Anomaly Detection of Industrial Equipment under Fixed Industrial Camera Conditions This embodiment uses online visual inspection of industrial equipment components at a fixed workstation as an application scenario. The target to be inspected exists in multiple legitimate and normal states during the production process, and may exhibit anomalies such as surface contamination, cracks, gaps, missing parts, misalignment, structural deformation, or foreign object adhesion. Simultaneously, non-abnormal interferences such as localized reflections, motion blur, contour occlusion, and equipment vibration also exist in the industrial environment.

[0063] This embodiment uses a fixed-mount RGB industrial camera to acquire images of the target to be detected, with the video acquisition frame rate set to 15 frames per second. Normal samples are divided into normal training samples, normal validation samples, and normal test samples in an 8:1:1 ratio. Abnormal reference samples are only used to construct an abnormal standard library and perform secondary abnormality determination; they do not participate in the training of the multi-branch normal state reconstruction model. The test data includes both persistent real abnormal images and non-abnormal interference images formed only by reflections, blurring, occlusion, and slight jitter.

[0064] Step S1: Initialize and build the normal sample library and the abnormal database.

[0065] Industrial images that have been manually confirmed to be free of target anomalies are stored in the normal sample library, and the image number, camera acquisition point, equipment operation stage, industrial operating parameters, acquisition time, and review status are recorded. The normal sample library is used for normal state classification and multi-branch normal state reconstruction model training in step S2.

[0066] The anomaly database includes an anomaly standard library, a confirmed anomaly image set, and a suspected anomaly library. The anomaly standard library is manually constructed before online detection, selecting representative standard anomaly samples with clear images and complete anomaly regions from historical anomaly records, and manually labeling the standard anomaly regions. The confirmed anomaly image set stores the anomaly samples confirmed in step S7, while the suspected anomaly library stores suspected anomaly samples that have not yet been confirmed as anomalies after a second anomaly assessment and require manual review.

[0067] Before storing standard anomaly samples in the anomaly standard library, the bounding box of the standard anomaly region is expanded by 10% of its original size, maintaining the aspect ratio while scaling and padding to 128 pixels × 128 pixels; orientation correction is performed based on the major axis direction of the smallest bounding rectangle. Texture features use a unified local binary pattern feature with a radius of 1 and 8 neighboring points, and a 59-dimensional normalized histogram is calculated; edge features use directional gradient histogram features, with a pixel unit size of 8 pixels × 8 pixels, a feature block size of 2 × 2 pixel units, and 9 orientation intervals; structural features consist of the signed logarithmic values ​​of seven Hu invariant moments, as well as area ratio, perimeter ratio, aspect ratio, roundness, and convexity. The three types of features are associated with the sample indexes of the corresponding standard anomaly samples and stored in the anomaly standard library for direct access in step S7.

[0068] The anomaly standard library is initialized and updated manually only, and is periodically reviewed and updated. Samples in the confirmed anomaly image set are only added to the anomaly standard library or have existing standard anomaly samples replaced after their image clarity, anomaly region integrity, and morphological representativeness have been manually reviewed, thus preventing false positives from contaminating the anomaly standard library.

[0069] Step S2: Establish a multi-branch normal state reconstruction model based on the normal sample library.

[0070] In step S2.1, both normal training samples and normal validation samples adopt the same target region extraction and preprocessing process as in the online detection stage. At a fixed station, the target region is cropped according to a preset region of interest. A binary target mask is generated based on the target's outer contour. The background pixels outside the target mask are set to preset background values ​​to achieve background suppression. The target centroid and principal direction are calculated using the target's outer contour, and the target contour is rigidly registered with a preset reference contour. Subsequently, the aspect ratio is maintained, and the dimensions are scaled and padded to 256 pixels × 256 pixels. The RGB pixel values ​​are divided by 255 and normalized to [0,1].

[0071] In step S2.2, normal samples are initially grouped according to camera acquisition points, equipment operating stages, and industrial operating parameters. A global state feature vector is extracted from the normal samples in each initial group. This global state feature vector includes the logarithm of the seven Hu moments of the target contour, the aspect ratio of the circumscribed rectangle, the principal direction of the target, the normalized centroid coordinates, and the global grayscale mean. After standardizing the global state feature vector, K-means clustering is performed. Contour coefficients are calculated within the range of K from 2 to 8, and the K with the largest contour coefficient is determined as the number of normal states in the corresponding initial group.

[0072] Each normal state is assigned a state number k, and the state number is mapped to a state code through a 16-dimensional learnable embedding layer. During online detection, the region to be detected in the current frame is first grouped according to the camera acquisition point, equipment operation stage, and industrial operating condition parameters. Then, the Euclidean distance between its global state feature vector and the cluster centers of each normal state in the group is calculated, and the state code corresponding to the normal state cluster center with the smallest distance is selected.

[0073] In step S2.3, the multi-branch normal state reconstruction model includes a shared image encoder, a state coding fusion module, and parallel texture reconstruction branches, edge reconstruction branches, and structure reconstruction branches. The shared image encoder includes four levels of convolutional coding units, each using 3×3 convolution, batch normalization, and ReLU activation, and downsampling through convolution with a stride of 2. The number of feature channels is 32, 64, 128, and 256, respectively.

[0074] The state encoding is mapped to a state condition vector through a fully connected layer and copied in the spatial dimension to form a state condition feature map with the same spatial size as the encoder bottleneck feature. The state condition feature map and the encoder bottleneck feature are concatenated along the channel dimension to obtain the state fusion feature. The texture reconstruction branch, edge reconstruction branch, and structure reconstruction branch each adopt a four-level upsampling convolution structure to restore the state fusion feature to 256 pixels × 256 pixels, and output the normalized color texture reconstruction result, single-channel edge reconstruction result, and single-channel distance transform map form of the structure reconstruction result, respectively.

[0075] Training samples are drawn exclusively from a normal sample library. During each training iteration, a rectangular mask occupying 5% to 20% of the target region is randomly generated and superimposed with Gaussian noise of standard deviation 0.02 to obtain perturbed normal samples. The training objective for the texture reconstruction branch is an unperturbed, normalized RGB image; the training objective for the edge reconstruction branch is a binary edge map generated using the Canny operator after converting the unperturbed normal image to grayscale; and the training objective for the structure reconstruction branch is a distance transform map generated and normalized based on the target contour mask.

[0076] Texture reconstruction loss Composed of L1 loss and structural similarity loss, edge reconstruction loss The structural reconstruction loss consists of binary cross-entropy loss and Dice loss. It consists of L1 loss and Dice loss. The total training loss in this embodiment is... Set to: in, This represents the total training loss of the multi-branch normal state reconstruction model; , and represents the texture reconstruction loss, edge reconstruction loss, and structure reconstruction loss, respectively; 0.4, 0.3, and 0.3 represent the fixed loss coefficients used for the three reconstruction losses during the training phase.

[0077] The multi-branch normal state reconstruction model is trained using the Adam optimizer, with an initial learning rate set to... The batch size is set to 32, and the training epochs are set to 100. After each training epoch, the total training loss on the normal validation samples is calculated, and the model parameters at which the total training loss on the normal validation samples is minimized are saved to obtain the trained multi-branch normal state reconstruction model.

[0078] Step S3: Acquire images of the industrial site and calculate image quality parameters.

[0079] During online inspection, the RGB industrial camera continuously acquires image frames from the industrial site. The current frame undergoes the same processing as step S2.1: region of interest cropping, background suppression, position registration, size normalization, and pixel normalization, to obtain a standardized region to be inspected in the current frame.

[0080] reflectivity This is set to the ratio of the number of pixels with a normalized brightness greater than 0.95 in the current frame's detection region to the total number of pixels in that region. Let the Laplacian response variance of the current frame's detection region be... The lower and upper limits for clear image calibration are respectively and The degree of ambiguity Calculate according to the following formula: in, This means that the calculation result is restricted to the interval [0,1]. and The variance distribution of the Laplace response was determined offline from clear, normal validation samples.

[0081] Structural visibility Set as the ratio of the current effective contour length to the corresponding normal state reference contour length, and limit the calculation result to the interval [0,1]; the corresponding normal state reference contour length is the average of the effective contour lengths of the normal verification samples in the normal state. and The larger the value, the more severe the reflection and blurring, respectively. A larger value indicates a more complete target structure, and defines the degree to which the structure is invisible. .

[0082] Step S4: Reconstruct multiple features based on state coding and calculate the anomaly error.

[0083] Extract the global state feature vector of the current frame to be detected, and determine the corresponding state code according to step S2.2; input the standardized current frame to be detected region and its corresponding state code into the trained multi-branch normal state reconstruction model to obtain the texture reconstruction result, edge reconstruction result and structure reconstruction result.

[0084] The actual texture representation is a normalized RGB image of the region to be detected in the current frame; the actual edge representation uses the same Canny binary edge map as the model training target; and the actual structure representation uses the same normalized distance transform map as the model training target. The pixel-wise absolute difference between the three types of actual representations and their corresponding reconstruction results is calculated to obtain the texture error map. Edge error map and structural error diagram .

[0085] To improve sensitivity to local anomalies, the average of the top 10% of pixels with the highest error values ​​in each error map is used to obtain the texture error. Edge error and structural errors .

[0086] Step S5: Dynamically adjust the feature error weights based on image quality parameters.

[0087] During the system deployment phase, a quality perturbation verification set is constructed from normal verification samples. Multiple levels of local brightness saturation and highlight region overlay, Gaussian blur, and contour occlusion are applied to the normal verification samples to simulate three quality degradation factors: reflection, blurring, and structural invisibility. The unperturbed normal verification samples and the perturbed normal verification samples are then input into the multi-branch normal state reconstruction model, and the scalar errors of the three feature branches are calculated according to step S4.

[0088] For the The first feature branch and the first A quality degradation factor is defined as the average relative error increment relative to the unperturbed reference at each disturbance level. The quality-branch influence coefficient is calculated according to the following formula. : in, These represent texture, edge, and structural branches, respectively. These represent quality degradation factors such as reflection, blurring, and invisible structure, respectively. Set as This is used to prevent the denominator from being zero. (From all) Constructing the quality-branch influence coefficient matrix and the matrix It is now configured for online detection.

[0089] During online detection, based on the results obtained in step S3 , and Construct the current frame quality degradation vector: in, , and These represent the degree of reflectivity, the degree of blurriness, and the degree of structural invisibility, respectively. Let the minimum reliability weight lower bound be... , No. The unnormalized reliability weights of each feature branch are calculated according to the following formula: Normalize the three unnormalized reliability weights to obtain the texture dynamic weights. dynamic weights at the edge and structural dynamic weights : The three normalized dynamic weights satisfy Based on the results obtained in step S4 , and The weighted feature anomaly score of the current frame is calculated according to the following formula. : At the same time, according to the following formula , and A pixel-by-pixel position-weighted fusion is performed to obtain a comprehensive anomaly heatmap. : Abnormal score threshold Take normal verification samples 99th percentile of the distribution; pixel segmentation threshold of the comprehensive anomaly heatmap Take the 99.5 percentile of the pixel value distribution of the heatmap for normal validation samples. After threshold segmentation, morphological closing operations are performed using 3×3 structuring elements. Connected regions are extracted from the closing operation results, and adjacent connected regions with a minimum bounding box distance of less than 8 pixels are merged into the same candidate anomaly region.

[0090] Step S6: Perform timing consistency analysis and preliminary anomaly detection.

[0091] This embodiment employs a five-frame sliding window, including the current frame. The candidate anomaly regions from the previous four frames are mapped to the current frame coordinate system using a target contour registration matrix. The intersection-union ratio (IUR) between the candidate anomaly regions in the current frame and those from the previous four frames is then calculated. in, Indicates the candidate anomaly region in the current frame. This represents a candidate anomaly region in a historical frame mapped to the current frame coordinate system. This represents the area of ​​the intersection of two regions. This represents the area of ​​the union of two regions.

[0092] Timing consistency threshold Offline calibration is performed using normally functioning videos and videos confirmed to be persistently abnormal. When the current frame... And at least three of the first four frames of the candidate anomaly region correspond to Greater than If the candidate abnormal region is identified as an initial abnormal region, it is identified as a normal region and the detection of the current frame ends. This process is used to filter out specular highlights, noise, transient blur, and device jitter responses that only last for a single frame or a few frames.

[0093] Step S7: Perform secondary anomaly determination based on the anomaly standard library in the anomaly database.

[0094] The preliminary abnormal region obtained in step S6 is processed in the same way as the standard abnormal region in step S1. The bounding box is expanded by 10% in all directions, the aspect ratio is maintained, and the edges are padded to 128 pixels × 128 pixels. The orientation is corrected according to the major axis of the smallest bounding rectangle.

[0095] Using the same feature extraction parameters as the anomaly standard library, 59-dimensional unified pattern local binary pattern texture features, directional gradient histogram edge features, and structural features consisting of the signed logarithmic values ​​of seven Hu invariant moments and five contour geometric parameters were extracted from the initial anomaly region. These features were compared with those in the anomaly standard library. The three types of features of the standard anomaly regions are matched to calculate the texture matching distance, edge matching distance, and structure matching distance. The texture matching distance is calculated using chi-square distance, the edge matching distance is calculated using 1 minus cosine similarity, and the structure matching distance is calculated using Euclidean distance after variance normalization of each dimension.

[0096] Based on the offline calibration set, the minimum and maximum values ​​of texture matching distance, edge matching distance, and structure matching distance are determined respectively. Max-min normalization is then used to obtain the texture matching differences. Edge matching differences Differences in structural matching Reuse the dynamic weights of the current frame and calculate the weights according to the following formula. Weighted feature difference score corresponding to each standard abnormal region : Take the minimum weighted feature difference score from all N standard anomaly regions in the anomaly standard library. : Where N represents the number of standard anomaly regions stored in the anomaly standard library.

[0097] Difference score threshold The calibration is achieved through the matching distance distribution between standard abnormal regions and the matching distance distribution from non-abnormal interference regions to standard abnormal regions. When the current preliminary anomaly is identified as a confirmed anomaly, it is written into the confirmed anomaly image set; when At this time, the initial anomaly is added to the suspected anomaly database for manual review. This step only confirms whether the current area is similar to known anomaly characteristics, and does not rely on the generated anomaly. The standard anomaly sample output determines the anomaly type.

[0098] The system saves the original image frame, coordinates of the candidate anomaly region, and area of ​​the candidate anomaly region for each detection result. , It can display dynamic weights, timestamps, device locations, and audit status, and output anomaly confirmation results, comprehensive anomaly heatmaps, and alarm information.

Claims

1. A few-sample industrial visual anomaly detection method based on dynamic weighted multi-feature reconstruction, characterized in that, Includes the following steps: S1: Initialize and build the normal sample library and the abnormal database; S2: Establish a multi-branch normal state reconstruction model based on a normal sample library; S3: Acquire images of the industrial site and calculate image quality parameters; S4: Perform multi-feature reconstruction based on state coding and calculate anomaly error; S5: Dynamically adjust the feature error weights based on image quality parameters to obtain the weighted feature anomaly score and candidate anomaly regions for the current frame; S6: Perform temporal consistency analysis on candidate abnormal regions and make preliminary anomaly determination by combining weighted feature anomaly scores; S7: Perform secondary anomaly determination based on the anomaly standard library in the anomaly database.

2. The method for detecting few-sample industrial visual anomalies based on dynamic weighted multi-feature reconstruction according to claim 1, characterized in that, In step S1: The normal sample library is used to store normal industrial images that have been manually confirmed to be free of target anomalies, and is used to train a multi-branch normal state reconstruction model and establish a normal state representation. The anomaly database is used to uniformly manage anomaly-related data generated during the detection process, including an anomaly standard library, a set of confirmed anomaly images, and a suspected anomaly library; The anomaly standard library stores the standard anomaly regions and their characteristic information corresponding to representative standard anomaly samples that have been manually confirmed, serving as the feature matching benchmark for the subsequent secondary anomaly determination stage; the confirmed anomaly image set stores the anomaly samples that have been confirmed after secondary anomaly determination; and the suspected anomaly library stores suspected anomaly samples that have not yet been confirmed as anomalies after secondary anomaly determination and require manual review. Before the standard abnormal samples are stored in the abnormal standard library, the abnormal regions in the standard abnormal samples are manually labeled, their size is normalized, and their orientation is corrected. The texture features, edge features, and structural features of the abnormal regions are extracted and stored in the abnormal standard library in association with the sample index of the corresponding samples, for use in step S7 when performing secondary abnormality determination. The texture features adopt local binary mode features, the edge features adopt directional gradient histogram features, and the structural features adopt Hu invariant moments and contour geometric features; The aforementioned anomaly standard library is only initialized and periodically reviewed and updated manually, and does not participate in the training of the multi-branch normal state reconstruction model.

3. The method for detecting few-sample industrial visual anomalies based on dynamic weighted multi-feature reconstruction according to claim 2, characterized in that, In step S2: Step S2.1: Extract and preprocess the target region from the normal samples in the normal sample library; The target region extraction method involves cropping a preset region of interest at a fixed workstation; preprocessing includes background suppression, position registration, size normalization, and pixel normalization. Step S2.2: Based on the camera acquisition points, equipment operation stage and industrial operating parameters, the normal samples are initially grouped, and K-means clustering is performed using target contour, target pose and overall appearance features. A state number is assigned to each normal state category, and the state number is mapped to the corresponding state code. Step S2.3: Construct and train a multi-branch normal state reconstruction model; The preprocessed normal samples obtained in step S2.1 and the corresponding state codes determined in step S2.2 are used together as model training data to construct a multi-branch normal state reconstruction model. The multi-branch normal state reconstruction model is used to learn the texture representation, edge representation and structural representation of the target to be detected under different normal states. In the subsequent step S4, based on the target region to be detected in the current frame and its corresponding normal state code, the texture reconstruction result, edge reconstruction result and structural reconstruction result that the target region to be detected in the current frame should have under the corresponding normal state are generated. The multi-branch normal state reconstruction model includes a shared image encoder, a state coding fusion module, and parallel texture reconstruction branches, edge reconstruction branches, and structure reconstruction branches; wherein the shared image encoder and the state coding fusion module are used by the three reconstruction branches, and the three reconstruction branches have independent network parameters. The shared image encoder includes a series of convolutional coding units connected in sequence. Each convolutional coding unit includes a convolutional layer, a batch normalization layer and an activation layer. Adjacent convolutional coding units reduce the spatial size of the feature map and increase the number of feature channels through downsampling operations, thereby extracting multi-scale image features of preprocessed normal samples and obtaining encoder bottleneck features. The state encoding fusion module is used to fuse the state encoding obtained in step S2.2 with the encoder bottleneck features; specifically, the state encoding is mapped to a state condition vector through a fully connected layer, and the state condition vector is expanded in the spatial dimension to form a state condition feature map with the same spatial size as the encoder bottleneck features; the state condition feature map and the encoder bottleneck features are concatenated along the channel dimension to obtain the state fusion features; The state fusion feature simultaneously includes image information of the input normal sample and conditional information of the corresponding normal state, and is respectively input to the texture reconstruction branch, edge reconstruction branch and structure reconstruction branch; Furthermore, the texture reconstruction branch, edge reconstruction branch, and structure reconstruction branch each include a multi-level upsampling convolutional unit and an output layer connected in sequence; the multi-level upsampling convolutional unit is used to recover the spatial size of the state fusion features step by step, and the output layer is used to convert the recovered features into the reconstruction results of the corresponding branch; the three reconstruction branches share the shared image encoder and state coding fusion module, and each has independent network parameters; The texture reconstruction branch takes the state fusion feature as input, and outputs the texture reconstruction result after multi-level upsampling convolution processing. The texture reconstruction result is a normalized color image with the same spatial size as the preprocessed normal sample, which is used to represent the color distribution and local surface texture that the target to be detected should have in the corresponding normal state. The edge reconstruction branch takes state fusion features as input, and outputs the edge reconstruction result after multi-level upsampling convolution processing. The edge reconstruction result is a single-channel edge map with the same spatial size as the preprocessed normal sample, which is used to represent the contour boundary, local edge and edge continuity that the target to be detected should have in the corresponding normal state. The structural reconstruction branch takes the state fusion features as input, and outputs the structural reconstruction result after multi-level upsampling convolution processing. The structural reconstruction result is a single-channel distance transformation map with the same spatial size as the preprocessed normal sample, which is used to represent the overall contour, region shape and spatial relationship of the target's internal pixels relative to the contour boundary that the target should have in the corresponding normal state. During training, a random rectangular mask and Gaussian noise are applied to the preprocessed normal samples obtained in step S2.1 to obtain perturbed normal samples. The perturbed normal samples and their corresponding state codes are input into the multi-branch normal state reconstruction model. The unperturbed normal samples, the edge map generated from the unperturbed normal samples, and the distance transformation map generated from the target contour mask are used as the training targets of the three branches, respectively. This suppresses the direct copying of the input image by the multi-branch normal state reconstruction model and enhances the learning of normal structural patterns. Loss from texture reconstruction Edge reconstruction loss and structural reconstruction loss Jointly train the three reconstruction branches and calculate the total training loss. Update the network parameters of the shared image encoder, the state encoding fusion module, and the three reconstruction branches to obtain the trained multi-branch normal state reconstruction model. The total training loss For texture reconstruction loss Edge reconstruction loss Structural remodeling loss Weighted sum: Among them, the texture reconstruction loss is composed of the absolute error loss and structural similarity loss used in the texture reconstruction branch, the edge reconstruction loss is composed of the binary cross-entropy loss and Dice loss used in the edge reconstruction branch, and the structural reconstruction loss is composed of the absolute error loss and Dice loss used in the structural reconstruction branch. , and These are the fixed loss coefficients used during the training phase; the fixed loss coefficients during the training phase are independent of the dynamic weights determined based on image quality during the online detection phase. The trained multi-branch normal state reconstruction model is used as the output of step S2, which is then used by step S4 to generate texture reconstruction results, edge reconstruction results, and structural reconstruction results based on the detection region of the current frame and its corresponding state encoding.

4. The method for detecting few-sample industrial visual anomalies based on dynamic weighted multi-feature reconstruction according to claim 3, characterized in that, In step S3: Image frames of the target to be inspected in the industrial field are continuously acquired by an RGB camera; the area to be inspected is extracted from the current image frame, and background suppression, position registration, size normalization and pixel normalization are performed to obtain a standardized area to be inspected; Image quality parameters are calculated based on the standardized region to be detected, including the degree of reflectivity. ambiguity and structural visibility ; Among them, reflectivity Used to indicate the degree to which an image region is affected by highlight reflections, calculated based on the proportion of pixels in the region to be detected whose normalized brightness is greater than a highlight threshold; blur level. The degree to which image edge information is affected by blur is used to represent the degree of blurring, calculated based on the normalized result of the Laplacian response variance relative to the calibration interval of the sharp image; structural visibility. Used to represent the completeness of the target structure contour, calculated based on the ratio of the current contour length to the corresponding normal state reference contour length; and The larger the value, the more severe the reflection and blurring, respectively. The larger the value, the more complete the structure.

5. The method for detecting few-sample industrial visual anomalies based on dynamic weighted multi-feature reconstruction according to claim 4, characterized in that, In step S4: Specifically, firstly, the state code of the current region to be detected is extracted according to the method in step S2.2; then, the current region to be detected and its corresponding state code are input into the multi-branch normal state reconstruction model established in step S2 to obtain the texture reconstruction result, edge reconstruction result and structure reconstruction result respectively. The actual texture representation, actual edge representation, and actual structure representation are compared position-by-position with the corresponding reconstruction results to obtain the texture error map. Edge error map and structural error diagram And further calculate the texture error Edge error and structural errors .

6. The method for detecting few-sample industrial visual anomalies based on dynamic weighted multi-feature reconstruction according to claim 5, characterized in that, In step S5: Step S5.1: Based on the reflectivity calculated in step S3 ambiguity and structural visibility Construct the quality degradation vector for the current frame. : ， in, Indicates the degree of reflectivity , Indicates the degree of ambiguity , Indicates the degree of invisibility of the structure ; Step S5.2: Divide normal verification samples from the normal sample library and construct a quality perturbation verification set based on the normal verification samples; specifically, apply multiple levels of reflection perturbation, blur perturbation and contour occlusion perturbation to the normal verification samples to simulate reflection, blur and invisible structure in industrial field respectively. Input the normal verification samples without quality perturbation and the normal verification samples with different levels of quality perturbation into the multi-branch normal state reconstruction model established in step S2, calculate the texture error, edge error and structural error according to the method described in step S4, and calculate the average relative error increment of each quality degradation factor on each feature branch. Normalizing the three average relative error increments corresponding to the same feature branch yields the quality-branch influence coefficient: in For feature branch index, These represent texture, edge, and structural branches, respectively. Index of factors contributing to quality degradation These represent reflection, blur, and invisible structure, respectively. Indicates the first The first quality degradation factor on the second The average relative error increment caused by each feature branch This represents a preset positive number used to prevent the denominator from being zero. Indicates the first The first quality degradation factor on the second The influence coefficient of the reliability of each characteristic branch; The mass-branch influence coefficient matrix is ​​composed of the mass-branch influence coefficients of each mass: Step S5.3: Based on the quality degradation vector of the current frame obtained in step S5.1 The quality-branch influence coefficient matrix obtained in step S5.2 The dynamic weights of the three branches—texture, edge, and structure—are calculated separately. Specifically, first calculate according to the following formula: in, Indicates the first The reliability weights of each feature branch, This represents the preset minimum reliability weight lower limit, and This is used to prevent the weight of any feature branch from decreasing to zero; The three reliability weights are normalized to obtain the dynamic weights. 、 and : Step S5.4: Based on the texture error obtained in step S4 Edge error Structural errors and the dynamic weights obtained in step S5.3 The weighted feature anomaly score of the current frame is calculated according to the following formula. : ; Step S5.5: Based on the texture error map obtained in step S4 Edge error map and structural error diagram The comprehensive anomaly heatmap is obtained by performing position-by-position weighted fusion according to the following formula. : Candidate abnormal regions are obtained through threshold segmentation and connected component processing.

7. The method for detecting few-sample industrial visual anomalies based on dynamic weighted multi-feature reconstruction according to claim 6, characterized in that, In step S6: The temporal consistency analysis involves mapping the candidate anomaly region of the current frame and the candidate anomaly region of the historical frames within a preset time window to a unified coordinate system, and calculating the intersection-union ratio between the current candidate anomaly region and the historical candidate anomaly region. When the weighted feature anomaly score of the current frame is greater than the anomaly score threshold, and the candidate anomaly region of the current frame satisfies the cross-union ratio constraint condition within the preset time window, the candidate anomaly region of the current frame that satisfies the cross-union ratio constraint condition is determined as a preliminary anomaly region. Otherwise, the corresponding candidate anomaly region will be judged as a non-continuous anomaly response and will not enter the anomaly standard library matching process.

8. The method for detecting few-sample industrial visual anomalies based on dynamic weighted multi-feature reconstruction according to claim 7, characterized in that, In step S7: The size of the preliminary abnormal region obtained in step S6 is normalized and the orientation is corrected. Texture features, edge features and structural features are extracted using the same feature extraction method as the standard abnormal sample in step S1. The texture, edge, and structural features obtained from the initial anomaly region are matched with standard anomaly regions in the anomaly standard library to obtain texture matching distance, edge matching distance, and structural matching distance. The texture matching distance is calculated using chi-square distance, the edge matching distance is calculated using 1 minus cosine similarity, and the structural matching distance is calculated using Euclidean distance after variance normalization of each dimension. The three types of matching distances are then normalized to obtain texture matching difference, edge matching difference, and structural matching difference. Using the dynamic weights determined in step S5, the weighted feature difference score corresponding to the i-th standard anomaly region is calculated. in, Indicates the first i Weighted feature difference scores corresponding to each standard abnormal region; Indicates texture matching differences; Indicates edge matching differences; Indicates poor structural matching; 、 and The dynamic weights obtained in step S5; Take the minimum value among the weighted feature difference scores corresponding to all standard outlier regions in the outlier standard library: in, N Indicates the number of standard anomaly regions in the anomaly standard library; when When the difference is less than or equal to a preset difference threshold, the current preliminary anomaly is determined as a confirmed anomaly and stored in the confirmed anomaly image set in the anomaly database; when... When the difference exceeds the preset threshold, the current preliminary anomaly will be stored in the suspected anomaly database for manual review.

Images