A method, apparatus, device and medium for industrial image logical anomaly detection

Abstract

The present application relates to the technical field of computer vision, and aims at the problem of insufficient part-level logical anomaly detection capability and high missing detection rate of existing industrial image anomaly detection methods, and provides a kind of industrial image logical anomaly detection method, device, equipment and medium.The method comprises: obtaining an industrial image to be detected, and inputting first and second processing branches respectively;The first processing branch performs component segmentation, extracts component area, color and spatial layout features, and calculates a first anomaly score based on the distance between these features and the corresponding features in the memory library;The second processing branch performs knowledge distillation-based anomaly detection, and obtains a second anomaly score using the feature differences of the teacher network, student network and autoencoder;The first and second anomaly scores are updated based on cross guidance, and the logical anomaly of the industrial image to be detected is judged after fusion.The complex logical anomaly is accurately identified through double-branch parallel fusion, and the detection accuracy is high and the explanation is strong, which is widely used in industrial automation quality inspection.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and specifically relates to a method, apparatus, equipment and medium for detecting logical anomalies in industrial images. Background Technology

[0002] Industrial anomaly detection is a crucial component of intelligent manufacturing and product quality control. With the development of computer vision and deep learning technologies, anomaly detection methods based on unsupervised or self-supervised learning have received widespread attention and research in the industry due to their advantage of not requiring a large number of labeled anomaly samples.

[0003] Currently, mainstream industrial image anomaly detection methods can be mainly divided into two categories: Knowledge distillation-based methods typically employ a teacher-student network architecture. The teacher network is pre-trained on a large-scale dataset with its parameters frozen; the student network is trained to mimic the teacher network's output on a training set containing only normal samples. During inference, because the student network has only seen normal patterns, the outputs of the teacher and student networks in anomalous regions will differ significantly, thus locating anomalies. However, existing methods (such as Uninformed Students, AST, RD4AD, etc.) all use a single-pair teacher-student distillation architecture, which is sensitive to pixel-level texture and shape anomalies, but often fails to produce sufficient anomaly responses for high-level anomalies that require understanding global semantics and logical relationships between components (such as mismatched component models, incorrect wire color order, and reversed item arrangement), leading to missed detections.

[0004] Methods based on feature embedding and distance metrics. These methods utilize pre-trained networks to extract features from normal images, constructing a memory database to store these normal features. During detection, features of the image to be tested are extracted and compared with the normal features in the memory database to generate anomaly scores. Essentially, these methods perform statistics in the feature space of pixels or patches, failing to explicitly understand the semantic concept of "parts," making it difficult to determine whether a specific part's area is too small, its color abnormal, or to accurately count sets of multiple identical parts.

[0005] However, the aforementioned existing technologies still have significant technical shortcomings when dealing with complex industrial scenarios, especially when dealing with deep logical anomalies: insufficient ability to detect anomalies in spatial layout and structural relationships; difficulty in capturing component-level attributes based on pixel-level features; and insensitivity to individual attribute differences of component instances.

[0006] Therefore, how to provide an industrial image anomaly detection method that can simultaneously and effectively detect pixel-level structural anomalies and complex component-level logical anomalies requiring semantic understanding (including component categories, quantities, individual attributes, and spatial layout relationships, etc.), and achieve high-precision and high-robust detection, is an urgent problem to be solved in this field. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a method, apparatus, device, and medium for detecting logical anomalies in industrial images. The method acquires an industrial image to be detected, performs logical anomaly detection based on component segmentation using a first processing branch, extracts component area, color, and spatial layout features, and calculates the distance to the memory database to obtain a first anomaly score; performs anomaly detection based on three pairs of knowledge distillation relationships using a second processing branch, calculates the feature differences between the teacher network and the student network, as well as between the autoencoder and the student network, to obtain a second anomaly score; cross-guides the component segmentation image and the feature differences, updating the first and / or second anomaly scores based on the cross-guided results; merges the updated first and second anomaly scores to determine whether logical anomalies such as missing, misaligned, incorrect quantity, or incorrect color exist; and finally outputs the logical anomaly detection result and the anomaly location.

[0008] This invention provides a method for detecting logical anomalies in industrial images, comprising: Acquire industrial images to be inspected; The industrial images to be detected are input into the first processing branch and the second processing branch respectively; The first processing branch is used to perform a logical anomaly detection process based on component segmentation, including: segmenting the industrial image to be detected into components to generate a component segmentation map and a binary mask map; extracting component-level features from the binary mask map, wherein the component-level features include at least component area features, component color features and component spatial layout features; and calculating a first anomaly score based on the distance metric between the component-level features and a preset memory library. The second processing branch is used to perform an anomaly detection process based on knowledge distillation, including: inputting the industrial image to be detected into a teacher network, a student network, and an autoencoder network, respectively, wherein the teacher network is a pre-trained feature extraction network with frozen parameters; and calculating a second anomaly score based on the differences between the feature maps of the teacher network and the student network and the differences between the feature maps of the autoencoder network and the student network. The differences between the component segmentation map and the feature maps of the teacher network and the student network are cross-guided; the first anomaly score and / or the second anomaly score are updated based on the cross-guided result. The first anomaly score and the second anomaly score, which have been updated by the cross-guided method, are fused to obtain a fused anomaly score. If the fused anomaly score is greater than or equal to a preset anomaly threshold, it is determined that the industrial image to be detected has a logical anomaly; otherwise, it is determined that there is no logical anomaly.

[0009] Preferably, the component area feature is determined by the number of pixels with a pixel value of 1 after binarization in each component image of the binary mask image; The color characteristics of the component are obtained by calculating the ratio of the a-channel value to the b-channel value in the CIELAB color space. The spatial layout characteristics of the components include component instance counting characteristics and small block area distribution characteristics; The component instance counting feature is used to describe the quantity distribution of component instances; The small patch area distribution feature is used to describe the distribution of component area features in the industrial image space.

[0010] Preferably, the calculation of the first anomaly score based on the distance metric between the component-level features and the preset memory library includes: Based on the component feature vector formed by the component area features and component color features of the industrial image to be detected, a first distance is calculated between the component feature vector and the first reference feature in the memory database. The first reference feature in the memory library is the set of component feature vectors of all normal images in the training set. Calculate the second distance between the component instance count feature of the industrial image to be detected and the second reference feature in the memory library. The second reference feature in the memory library is a set of component instance count features for all normal images in the training set. Calculate the third distance between the small patch area distribution features of the industrial image to be detected and the third reference feature in the memory library. The third reference feature in the memory library is a statistical feature vector of small blocks obtained by concatenating the mean and variance of the area distribution features of all small image blocks. The first anomaly score is obtained by weighted summation of the first, second, and third distances: ; in, and These are the first balancing weight and the second balancing weight, respectively.

[0011] Preferably, in the second processing branch, the knowledge distillation process includes three pairs of distillation relationships: The first distillation relation constrains the student network to learn the feature map output of the teacher network; The second distillation relation constrains the autoencoder network to learn and reconstruct the feature map output of the teacher network; The third distillation relation constrains the student network to learn the reconstruction error of the autoencoder network; The student network and the autoencoder network are trained in the following manner: Train using a training set that contains only normal images; The bottleneck layer dimension of the autoencoder network is limited to one-fifth of the number of feature map channels output by the teacher network, forcing the autoencoder network to learn the global feature distribution of a normal image and thus preventing it from accurately reconstructing local details. Parameter optimization is performed using a difficult case mining strategy: Calculate the pixel-by-pixel loss between the student network and the teacher network, and the pixel-by-pixel loss between the student network and the autoencoder network; Backward gradient propagation is performed on the pixel locations where the pixel-by-pixel loss value is greater than a preset loss threshold, so that the student network focuses on learning the regions where the teacher network and the autoencoder network respond strongly.

[0012] Preferably, the step of cross-guiding the differences between the component segmentation map and the feature maps of the teacher network and the student network includes: When the first processing branch is used to determine the first... When a component malfunctions, the first component will be... The abnormal marking information of each component is input into the second processing branch; ; Total number of component categories; In the first feature extraction block of the teacher network, the anomaly marker information is used to locate the component segmentation map. The pixel positions corresponding to each component image; The first Each component image is spatially aligned with the feature image of the first feature extraction block to obtain an aligned feature image; For the aligned feature map corresponding to the first The pixel positions of each component region are weighted and enhanced, so that the teacher network focuses on the first component region in subsequent feature extraction. The component area; the first The first component area refers to the first... The spatial region in which the probability distribution values ​​of the corresponding components in the component diagram satisfy the preset confidence conditions; The enhanced feature map of the teacher network weights is passed to the student network through the first distillation relation, and then to the autoencoder network through the second distillation relation, so that the student network and the autoencoder network synchronously enhance the feature map of the teacher network weights. Feature learning for each component.

[0013] On the other hand, the present invention also provides an apparatus for detecting logical anomalies in industrial images, comprising: Image acquisition module, used to acquire industrial images to be inspected; The branch input module is used to input the industrial image to be detected into the first processing branch and the second processing branch respectively; The first processing module is used to perform a logical anomaly detection process based on component segmentation using the first processing branch, including: segmenting the industrial image to be detected into components to generate a component segmentation map and a binary mask map; extracting component-level features from the binary mask map, wherein the component-level features include at least component area features, component color features and component spatial layout features; and calculating a first anomaly score based on the distance metric between the component-level features and a preset memory library. The second processing module is used to perform an anomaly detection process based on knowledge distillation using the second processing branch, including: inputting the industrial image to be detected into a teacher network, a student network, and an autoencoder network respectively, wherein the teacher network is a pre-trained feature extraction network with frozen parameters; and calculating a second anomaly score based on the differences between the feature maps of the teacher network and the student network and the differences between the feature maps of the autoencoder network and the student network. A cross-guiding module is used to cross-guide the differences between the component segmentation map and the feature maps of the teacher network and the student network; and to update the first anomaly score and / or the second anomaly score based on the result of the cross-guiding. The fusion judgment module is used to fuse the first abnormal score and the second abnormal score after the cross-guided update to obtain a fused abnormal score; if the fused abnormal score is greater than or equal to a preset abnormal threshold, it is determined that the industrial image to be detected has a logical abnormality; otherwise, it is determined that there is no logical abnormality.

[0014] On the other hand, the present invention protects a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned method for detecting logical anomalies in industrial images.

[0015] Furthermore, the present invention also protects a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for detecting logical anomalies in industrial images.

[0016] Compared with the prior art, the beneficial effects of the present invention include: (1) The method of this invention has accurate component-level logical anomaly detection capability. In the first processing branch, by introducing multi-dimensional features such as area and color based on component instances, it can accurately identify complex logical anomalies such as incorrect component quantity, incorrect type, and color deviation. Furthermore, by using component instance counting features (component histogram vector) and selection distance metric, it can effectively capture logical anomalies of increasing or decreasing component instance quantity; by using small block area distribution features (small block histogram vector) and selection distance metric, it can detect spatial layout anomalies of component area distribution in local areas. The two features complement each other from different dimensions, together constituting a comprehensive evaluation of spatial layout anomalies.

[0017] (2) The knowledge distillation branch, which focuses on structural anomalies, and the component segmentation branch, which focuses on logical anomalies, are organically integrated to achieve synergistic effect between the two branches. The resulting fusion anomaly score is more robust and accurate than that of any single branch.

[0018] (3) This invention achieves deep fusion and bidirectional verification of component semantic information and pixel-level feature differences by establishing a cross-guidance mechanism between the component segmentation map and the feature differences between the teacher network and the student network. On the one hand, the component segmentation map provides component-level semantic annotation for feature differences, which improves the anomaly detection results from location-level anomaly localization to component-level anomaly tracing, greatly enhancing the industrial interpretability of the detection results. On the other hand, feature differences provide edge-assisted adjustment for the component segmentation map through their edge response information, making up for the shortcomings of unsupervised component segmentation in edge details. The two verify each other and complement each other, maintaining stable detection performance under complex conditions such as illumination changes, occlusion, and segmentation noise, significantly enhancing the robustness of the model. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the steps of a method for detecting logical anomalies in industrial images according to one embodiment of the present invention. Figure 2 This is a schematic diagram of the spatial layout features of the first processing branch generation component in one embodiment of the present invention, wherein, Figure 2 (a) is a map showing the area distribution of small parts in a normal image. Figure 2(b) is a diagram showing the area distribution of small components in a logical anomaly image; Figure 3 This is a schematic diagram of the three-pair knowledge distillation relation network structure of the second processing branch in one embodiment of the present invention; Figure 4 This is a schematic diagram of the structural framework of an industrial image logic anomaly detection device in one embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] In one embodiment, such as Figure 1 As shown, the present invention provides a method for detecting logical anomalies in industrial images, comprising: Step S110: Obtain the industrial image to be inspected; Step S120: Input the industrial image to be detected into the first processing branch and the second processing branch respectively; Step S130: Execute a logical anomaly detection process based on component segmentation using the first processing branch, including: segmenting the industrial image to be detected into components to generate a component segmentation map and a binary mask map; extracting component-level features from the binary mask map, wherein the component-level features include at least component area features, component color features, and component spatial layout features; and calculating a first anomaly score based on the distance metric between the component-level features and a preset memory library. Step S140, using the second processing branch to perform an anomaly detection process based on knowledge distillation, includes: inputting the industrial image to be detected into a teacher network, a student network and an autoencoder network respectively, wherein the teacher network is a pre-trained feature extraction network with frozen parameters; and calculating a second anomaly score based on the difference between the feature maps of the teacher network and the student network and the difference between the feature maps of the autoencoder network and the student network. Step S150: Cross-guide the difference between the component segmentation map and the feature maps of the teacher network and the student network; update the first anomaly score and / or the second anomaly score based on the result of the cross-guide; Step S160: The first abnormal score and the second abnormal score after the cross-guided update are fused to obtain a fused abnormal score; if the fused abnormal score is greater than or equal to a preset abnormal threshold, it is determined that the industrial image to be detected has a logical abnormality; otherwise, it is determined that there is no logical abnormality.

[0023] It should be noted that this invention uses a training set containing only normal images for training. All statistics in all training stages (such as mean part area, mean color, histogram statistical features, etc.) are calculated based on normal images.

[0024] Specifically, in step S110, acquiring the industrial image to be detected includes: Industrial cameras are used to capture product images on the production line. During the acquisition process, the position of the camera and the product is kept fixed to ensure image consistency. To reduce computational complexity, the image is preprocessed by normalizing its size to a uniform pixel size while preserving the RGB three channels, resulting in the preprocessed image. This refers to the industrial image to be inspected; Preprocessed image Gaussian filtering and histogram equalization are performed to eliminate the effects of uneven illumination. It is an image Resolution (pixels) It is an image The channel dimension.

[0025] Further, in step S120, the industrial image to be detected is input into the first processing branch and the second processing branch respectively. The first processing branch is used to perform logical anomaly detection based on component segmentation, and the second processing branch is used to perform anomaly detection based on knowledge distillation.

[0026] In one embodiment, a dual-branch processing architecture is adopted, in which the two branches share the same input image but perform initial processing independently. Subsequently, information interaction and collaborative judgment between the branches are achieved through cross-guidance and score fusion.

[0027] This dual-branch processing architecture enables simultaneous analysis of images from two dimensions: "component semantics" (i.e., component-level features) and "pixel features" (i.e., differences between feature maps), providing complementary information for subsequent cross-guidance and fusion judgment. In the field of complex behavior pattern discrimination, rehabilitation training interaction intent recognition technology based on multimodal data fusion also faces the problem of distinguishing between normal and abnormal patterns. Its multi-source information fusion concept provides technical inspiration for the dual-branch collaborative detection architecture of this invention.

[0028] Next, in step S130, the logic anomaly detection process based on component segmentation in the first processing branch is executed, including: Step S131: Perform component segmentation on the industrial image to be detected to generate a component segmentation map and a binary mask map.

[0029] In one embodiment, a weakly supervised part segmentation method is used for part segmentation, requiring no pixel-level annotation and training only using image-level labels. Specifically: (1) Use a pre-trained visual Transformer model to extract features from the industrial image to be detected to obtain multi-scale feature maps.

[0030] In one embodiment, a DINO-ViT-S / 8 model pre-trained on the ImageNet dataset is used as a feature extractor. The input is a pre-processed image. After passing through the Visual Transformer (ViT) model, the output of the first Transformer block is taken as the feature map. .in, It is a feature map resolution, It is a feature map The channel dimension.

[0031] Using a pre-trained ViT model can fully utilize the general visual features learned from large-scale natural images, thereby improving the generalization ability of part segmentation.

[0032] (2) Construct component feature codebooks by Coreset sampling and K-means clustering.

[0033] To reduce computation, Coreset sampling is used on normal images in all training sets. Perform feature sampling to obtain the sampled feature vector.

[0034] For the images in the training set The corresponding feature map after feature extraction ,in, ; This represents the number of normal images in the training set. The original feature set is obtained by pooling the feature vectors from all pixel locations across all images. The total number of features it contains is .

[0035] Using Coreset sampling from Select one with a size much smaller than subset of (called the Coreset set), so that in Solving the nearest neighbor problem and finding the solution in The solution obtained above is approximate.

[0036] In one embodiment, the Coreset sampling employs a minimum-maximum facility location algorithm to construct an optimal Coreset set, the core formula of which is: ; in, The optimal Coreset set; It is taken from The feature vector in It is taken from The feature vectors in; This represents the L2 distance / norm.

[0037] Set the sampling rate for Coreset sampling ( ), target Coreset size (i.e., the target size of the optimal Coreset set) That is, the number of feature points retained after Coreset sampling.

[0038] The minimum-maximum facility location algorithm specifically includes: 1. Initialize the Coreset set ; 2. Regarding to Execute in a loop: From the original feature set Select a feature vector from To make it to the present Maximize the minimum distance between all feature vectors in the set, that is: ; in, It is a random linear projection used to accelerate the calculation of approximate nearest neighbors; Will join in ; 3. Return As the optimal Coreset set .

[0039] After Coreset sampling, a pixel feature sampling set is obtained. .

[0040] Then, to K-means clustering is performed on the feature vectors in the dataset, and the number of clusters is . ,get Cluster centers serve as component feature codebooks Each cluster center of the component feature codebook corresponds to a typical feature of a type of component (e.g., "nut", "washer", "bolt head"), etc. The component feature codebook is stored in a memory database. Each cluster center corresponds to Class of components, For component category indexing, , This represents the total number of component categories.

[0041] By using Coreset sampling and K-means clustering, the most representative component prototypes can be extracted from massive features efficiently, while reducing storage and computational overhead.

[0042] (3) Generate an initial component segmentation map based on the distance from the feature vector of each pixel position in the feature map to the component feature codebook.

[0043] Specifically, for each pixel location in the feature map, the distance between the feature vector of each pixel location and the cluster centers in the component feature codebook is obtained by calculating the minimum distance between the feature vector of each pixel location and the component feature codebook. Based on the component category corresponding to the cluster center with the minimum distance in the component feature codebook, a component category probability distribution (soft label) is generated for that pixel, thus generating an initial component segmentation map. The resolution of the initial component segmentation map is [resolution missing]. The number of channels is The size is Each channel corresponds to a probability map of a component, and each pixel position is in... The values ​​on each channel constitute the probability distribution of the component category at that location.

[0044] (4) The initial component segmentation map is optimized using a conditional random field to obtain a component segmentation map.

[0045] A Conditional Random Field (CRF) model is used to correct local prediction errors in the initial segmentation image by utilizing color and spatial differences between pixels, resulting in smoother and more regular part boundaries. The resulting part segmentation image is obtained after CRF optimization. .in Each channel (total) The value of (number) represents the spatial distribution probability of the corresponding component category (i.e., pixel value). Through CRF optimization, the boundary accuracy of component segmentation is significantly improved, providing more accurate component regions for subsequent feature extraction.

[0046] In one embodiment, a fully connected conditional random field (DenseCRF) is used to optimize the initial segmentation graph. The DenseCRF is a CRF with a complete graph structure.

[0047] In the above component segmentation and conditional random field optimization process, feature extraction, initial segmentation, and CRF optimization are all performed at low resolution. The above steps are completed to improve computational efficiency and maintain spatial alignment with the feature difference map of the second processing branch. This stage produces a component segmentation map in the form of a probabilistic map. Used for subsequent cross-booting.

[0048] (5) Upsample the component segmentation image to the original resolution of the industrial image to be detected to obtain the upsampled component segmentation image.

[0049] In one embodiment, bilinear interpolation is used to... Upsampled to the same resolution as the original image The upsampled component segmentation image is obtained. .at this time, Each channel corresponds to a component map, representing the probability map of the corresponding component, with pixel values ​​ranging from [value range missing]. .

[0050] In another embodiment, the upsampling can also employ bicubic interpolation, which provides better smoothing quality but increases computational cost.

[0051] (6) Segmentation diagram of the upsampled components Component images are filtered and binarized to obtain a binary mask image. .

[0052] Due to the variability of industrial scenarios, the number of preset component categories... This may not perfectly fit all images. Therefore, it is necessary to filter the component images in the upsampled component segmentation image, retaining only valid component images. Specifically, the following filtering operation should be performed: Each component diagram is smoothed and filtered. If the maximum response value after filtering is less than the preset noise threshold, the component diagram is determined to be noise and deleted.

[0053] The component images retained after smoothing filtering are binarized. The probability map of the corresponding component for each channel (component image) is converted into a binary mask, so that each component image becomes a binary image with a pixel value of 0 or 1, which respectively indicates that the pixel does not belong to the component or belongs to the component, thus obtaining a binary region image.

[0054] If the binary region map covers at least two corners of the image (e.g., any two of the four corners: top left, top right, bottom left, and bottom right), then the component map is determined to be the background and is deleted.

[0055] After filtering and binarizing the component images, the upsampled component segmentation images are obtained. Transform into a binary mask image ,in Number of remaining parts ( ),and Each channel is already a part map in the form of a binary mask. This step effectively removes invalid parts and background interference, and directly provides the accurate binary regions needed for subsequent feature extraction.

[0056] In one embodiment, the smoothing filter employs mean filtering.

[0057] In another embodiment, the smoothing filter may also employ any one of median filtering, Gaussian filtering, bilateral filtering, or wavelet denoising.

[0058] In one embodiment, the OTSU algorithm is used for binarization. The OTSU algorithm adaptively calculates the binarization threshold, converting continuous probability values ​​into a binary mask.

[0059] The OTSU algorithm, also known as the Otsu algorithm or the maximum inter-class variance method, is a classic algorithm for automatic image binarization. It calculates the inter-class variance of the image into foreground and background classes at each possible grayscale threshold by traversing all possible grayscale thresholds, and selects the threshold that maximizes the inter-class variance as the optimal binarization threshold.

[0060] because At original resolution The OTSU algorithm provides a continuous probability distribution and can adaptively optimize within the original image space to determine the optimal binarization threshold, achieving pixel-level segmentation accuracy. Therefore, the OTSU algorithm can adaptively adapt to changes in illumination and contrast in different images, improving the robustness of segmentation.

[0061] In another embodiment, the binarization process may also employ any one of the following: fixed threshold method, adaptive threshold method, iterative threshold method, or histogram-based bimodal method.

[0062] Step S132, from the binary mask image Extracting component-level features, which include at least component area features, component color features, and component spatial layout features. The process of extracting component-level features includes: (1) Extracting component area features: for Middle Part diagram ( The area feature of a component is determined by the number of pixels with a value of 1 after binarization in the component image. The number of pixels with a value of 1 after binarization is counted and denoted as . , representing the current image Middle The area of ​​each component (pixels).

[0063] During the training phase, the average part area is calculated for parts of the same category in all normal images in the training set: .

[0064] Using the average component area, the current image of the first... Area of ​​each component Normalization is performed to obtain the component area characteristics: .

[0065] The (normalized) component area feature can reflect whether the component size is abnormal (such as too small or too large). By comparing the normalized component area features, logical anomalies such as missing components (area close to 0) or deformed / misassembled components (area deviating from the normal range) can be accurately detected.

[0066] (2) Extracting component color features: The component's color characteristics are obtained by calculating the ratio of the a-channel value to the b-channel value in the CIELAB color space. Specifically: Input image Convert from RGB color space to CIELAB color space. The CIELAB color space (which includes three channels: L, a, and b) has a and b channels that can better quantify color differences.

[0067] for Middle For each component image, find the corresponding position of the pixel with a value of 1 after binarization in the original image, and extract the a-channel and b-channel values ​​at these positions.

[0068] Calculate the ratio of the a-channel value to the b-channel value, and then average the ratio across all pixels belonging to that component to obtain the current image. Middle Color of each component .

[0069] During the training phase, the average part color is calculated for parts of the same category in all normal images in the training set: ; Using the average component color, the current image's first... Color of each component Normalization is performed to obtain the component color characteristics: .

[0070] This (normalized) component color feature can reflect whether the component's color deviates from the normal range (e.g., abnormal color of metal plating). Using the ratio of the a-channel value to the b-channel value in the CIELAB space as the component color (feature), it has high sensitivity to subtle color deviations such as abnormal color of metal component plating and color difference of plastic components, and can effectively detect color logic anomalies that are difficult to distinguish with traditional RGB features.

[0071] Utilizing the area features of all components and component color characteristics , Build the current image Component feature vectors: .

[0072] (3) Extract the spatial layout features of the components, including component instance count features and small block area distribution features.

[0073] The process of extracting component instance count features includes: First, the binary mask image For each component image (in binary mask form), a density-based clustering algorithm is used to cluster component instances, resulting in multiple clusters. Specifically: For each component image in the binary mask image, connected components are extracted through connected component analysis, and each connected component is a component instance; small noise regions with an area smaller than a preset noise reduction threshold are filtered out (considered as invalid instances). The connected components are clustered using a density-based spatial clustering algorithm (DBSCAN): the area of ​​each connected component is used as a feature, and the cluster radius is set accordingly. The minimum number of points, MinPts, is used to group connected components with similar areas into the same cluster. Each cluster represents a class of component instances with similar areas.

[0074] Then, the number of component instances in each cluster is counted, and component histogram vectors are generated. .

[0075] Finally, the histogram vector is normalized by dividing it by its dimension (number of clusters) to obtain the normalized component histogram vector. As a feature for counting component instances, where, It represents the number of clusters.

[0076] The process of extracting the area distribution features of small patches includes: (i) Divide the industrial image to be detected into uniform portions. A number of image patches (or simply patches), among which , It is a positive integer; (ii) For each image patch, calculate the component area feature of each component and generate a patch histogram vector as the patch area distribution feature.

[0077] During the training phase, for the input images in the training set The generated small patch histogram vector ( , Its dimensions are , No. ( Each component represents a part. The component area within this small block. This feature is used to detect spatial layout anomalies in the component area distribution within a local area.

[0078] For training images (i.e., input images in the training set) The same small block position in ) Calculate the patch histogram vectors of all training images. mean ,variance and covariance matrix When the components are independent, It is a diagonal matrix, with the diagonal elements representing all variances. The mean and variance are concatenated to form the statistical characteristic of the small patch's location. The histogram vector of the small patch... The statistical features (mean and variance) are concatenated and stored in the memory library to form a small statistical feature vector: .

[0079] The aforementioned extracted spatial layout features are used to capture the spatial arrangement relationship of components, such as the count and positional distribution of multiple identical components.

[0080] During the training phase, a memory library (denoted as MemComp) for component-level features is pre-built. This memory library stores the following three types of data: Part feature vectors of all training images (Composed of normalized component area features and component color features of each component), stored as a set of component feature vectors in the memory library. ; Normalized part histogram vectors of all training images Stored as a set of component histogram vectors (i.e., component instance count features) in the memory library. ; Histogram vectors of small patches from all training images and small block statistical feature vectors .

[0081] Step S133: Calculate the first anomaly score based on the distance metric between the component-level features and the preset memory library.

[0082] During the detection phase, for the input industrial image to be detected... Perform the following calculations: (1) First distance based on component area features and component color features

[0083] Based on the component feature vector formed by the component area features and component color features of the industrial image to be detected, a first distance is calculated between the component feature vector and the first reference feature in the memory database. The first reference feature in the memory library is the set of component feature vectors of all normal images in the training set. .

[0084] For images The corresponding extracted component area features and component color characteristics , , build Component feature vector : .

[0085] In one embodiment, the K-Nearest Neighbors (KNN) algorithm is used to calculate... Zhongyu The closest Each component feature vector, using this The feature vector of each component and The mean or minimum L2 distance is used as the first distance. . This represents the number of clusters in the KNN algorithm.

[0086] In one embodiment, the KNN algorithm with a clustering number of 1 is used to calculate the feature vector of the component. The first distance obtained from the KNN distance (or nearest neighbor L2 distance) to the memory database : ; in, It means In the component feature vector set The nearest neighbor feature vector in ( Zhongyu (nearest component feature vector) This represents the L2 distance / norm.

[0087] (2) Second distance based on component instance count feature

[0088] Calculate the second distance between the component instance count feature of the industrial image to be detected and the second reference feature in the memory library. The second reference feature in the memory library is the set of part instance count features of all normal images in the training set. .

[0089] Specifically for the industrial images to be inspected The For each component diagram, extract its normalized component histogram vector. Calculate its relationship with the memory bank stored in the memory. Find the nearest neighbor L2 distance of the set and sum it over all components: .

[0090] (3) Calculate the third distance based on the distribution characteristics of small patch areas.

[0091] Calculate the third distance between the small patch area distribution features of the industrial image to be detected and the third reference feature in the memory library. The third reference feature in the memory library is a statistical feature vector of small patches obtained by concatenating the mean and variance of the patch area distribution features of all image patches. .

[0092] For images Divide it evenly into The image patch. For the first a small piece ( Extracting small patch histogram vectors (dimension) ).

[0093] In one embodiment, calculation Each small block of histogram vector (i.e. The Mahalanobis distance between the small patch area distribution characteristics and the statistical feature vector of the small patch: ; in, Represents the transpose of a vector / matrix.

[0094] Then, the largest Mahalanobis distance among all the small block histogram vectors is taken as the third distance. : .

[0095] The Mahalanobis distance used in this invention can automatically take into account the variance differences between different small blocks, making it more sensitive to abnormal fluctuations.

[0096] In one embodiment, it can also be calculated The normalized Euclidean distance or Manhattan distance between each small patch histogram vector and the small patch statistical feature vector is calculated, and then the largest normalized Euclidean distance or Manhattan distance among all small patch histogram vectors is taken as the third distance. .

[0097] (4) The weighted sum is used to obtain the first abnormal score. The first, second, and third distances are weighted and summed to obtain the first anomaly score: ; in, and These are the first balancing weight and the second balancing weight, respectively. A higher value indicates a greater likelihood of logical anomalies in the image.

[0098] Furthermore, in one embodiment, the second processing branch of the anomaly detection process based on knowledge distillation in step S140 includes: Step S141: Construct and train three pairs of knowledge distillation relation networks: Input the industrial image to be detected into the teacher network, student network and autoencoder network respectively, wherein the teacher network is a pre-trained feature extraction network with frozen parameters.

[0099] Teacher network (T): Employs a lightweight feature extraction network. Its input is a local image patch, and its output is a feature map. Let the number of channels in the teacher network's output feature map be denoted as . Spatial resolution is The teacher network is pre-trained on a large dataset from a large pre-trained network through knowledge distillation, and its parameters are frozen during subsequent training.

[0100] In one embodiment, the lightweight feature extraction network employs a lightweight PDN (Patch Description Network) pre-trained on ImageNet from WideResNet-101 via knowledge distillation.

[0101] Student Network (S): The structure of the student network is exactly the same as that of the teacher network, but with the addition of one output branch. That is, the student network includes two output branches: the first output branch... Second output branch The trainable parameters of the student network are updated during normal sample training.

[0102] Autoencoder Network (AE): Employs a standard convolutional autoencoder structure. The encoder part consists of multiple convolutional layers, compressing the input into low-dimensional bottleneck features; the decoder part consists of multiple deconvolutional layers, used to reconstruct the image and output the reconstructed image.

[0103] Limiting the bottleneck layer dimension of autoencoder networks Less than the number of channels in the teacher network output feature map One-fifth, that is This constraint forces the autoencoder to discard local detail information and focus on learning the global feature distribution of normal images. This makes the third distillation relation more sensitive to logical anomalies that require global understanding (such as missing parts or spatial layout errors), while avoiding over-responding to local structural anomalies such as texture, thus achieving functional decoupling from the first distillation relation.

[0104] In this embodiment, three pairs of knowledge distillation relations are constructed using the teacher network, student network, and autoencoder network: First distillation relation: Constraining student network S (specifically to the first output branch) Learn the feature map output of the teacher network T.

[0105] Second distillation relation: Constrained autoencoder network (AE) learns to reconstruct the feature map output of teacher network T.

[0106] Third distillation relation: Constraining the student network (specifically, to the newly added second output branch) Learn the reconstruction error of the autoencoder network.

[0107] Step S142: Based on the differences between the feature maps of the teacher network and the student network, and the differences between the feature maps of the autoencoder network and the student network, a second anomaly score is calculated.

[0108] During the training phase, the student network (S) and the autoencoder network (AE) are jointly trained using a training set containing only normal samples. The specific loss function is designed as follows: First distillation relationship (corresponding loss ST): Loss function Where MSE is the mean squared error; This represents the feature map output value of the teacher network. The feature map output value of the first output branch of the student network at the same position.

[0109] Second distillation relationship (corresponding loss AE): Loss function ,in, This represents the feature map output value of the autoencoder network at the same location.

[0110] Third distillation relationship (corresponding loss STAE): Loss function ,in, The feature map output value of the second output branch of the student network at the same position.

[0111] Construct the total loss function for training: .

[0112] A difficult example discovery strategy is employed during training: Calculate the pixel-by-pixel loss between the student network and the teacher network, and the pixel-by-pixel loss between the student network and the autoencoder network.

[0113] For each pixel location, only if the pixel-wise loss value at that pixel location is greater than a preset loss threshold... Only when the pixel location is inverted gradient propagation is performed, so that the student network focuses on learning the regions where the teacher network and autoencoder network respond strongly (i.e., regions with large changes or complex structures in normal images), avoiding the dominance of flat background regions, thereby improving the detection accuracy of small anomalies and boundary regions.

[0114] Through the third distillation relation, the student network learns the systematic reconstruction error pattern of the autoencoder for normal samples. When an anomalous image is input, the autoencoder produces errors exceeding the normal range, which the student network cannot mimic, thus generating a strong response in the loss STAE and achieving effective detection of logical anomalies. Compared to directly using the reconstruction error of the autoencoder ( This distillation method can eliminate systematic error interference from normal samples and improve the accuracy of detection.

[0115] During the inference phase (i.e., the detection phase), for the input image Calculate the following two feature difference maps: Structural anomaly difference diagram : Calculate the first output branch of the teacher network T and the student network Output differences, pixel by pixel Calculate the L2 norm and then average the channels: ; in, This indicates that the teacher network responds to the input image. The first feature map in the output Each channel, located at pixel position The value at that location (scalar); This represents the first output branch of the student network. For the input image The first feature map in the output Each channel, located at pixel position The value (scalar) at that location.

[0116] Logical anomaly difference diagram Computation of the autoencoder network AE and the second output branch of the student network Output differences: ; in, This indicates that the autoencoder network (AE) is used to process the input image. The first feature map in the output Each channel, located at pixel position The value at that location (scalar); This represents the second output branch of the student network. For the input image The first feature map in the output Each channel, located at pixel position The value (scalar) at that location.

[0117] The two difference maps are then merged pixel by pixel to obtain the second anomaly score map: .

[0118] In one embodiment, take from all pixels The maximum value is used as the second anomaly score at the image level: .

[0119] In another embodiment, the mean, median, or a high quantile (e.g., the 95th percentile) of all pixels can also be taken as the second anomaly score. .

[0120] In practical applications, the most appropriate statistical measure can be selected based on the specific scenario to construct the second anomaly score. . The larger the value, the higher the probability of structural or logical anomalies.

[0121] Then, in step S150, the differences between the component segmentation map and the feature maps of the teacher network and the student network are cross-guided; the first anomaly score and / or the second anomaly score are updated based on the result of the cross-guided analysis.

[0122] In one embodiment, specifically targeting the CRF-optimized part segmentation map in the first processing branch of step S130. And the differences between the feature maps of the teacher network and the student network in the second processing branch (referred to as feature differences). Cross-guidance is then implemented.

[0123] Let the feature map output by the teacher network be... Student network (first output branch) The output feature map is . For feature map and The resolution, and with resolution The equality allows for direct pixel-level cross-guiding between the two without the need for additional spatial alignment operations.

[0124] The differences between the feature graphs of the teacher network and the student network (referred to as feature differences). Calculated pixel by pixel and The Euclidean distance between them is obtained as follows: ,in This indicates the pixel position in the feature map. and The teacher network and student network are located at pixel positions respectively. The feature vector at that location. Because the teacher network parameters are frozen, it can still extract features in abnormal regions; however, the student network is only trained on normal images and struggles to accurately extract features from abnormal regions. Therefore, when the location belongs to a normal region, The value is small (e.g., less than the preset feature threshold); when the location belongs to an abnormal region. The value is relatively large (e.g., greater than the preset feature threshold).

[0125] The cross-guidance process specifically includes guidance in two directions: Direction 1 (component semantics → pixel features), which guides feature differences in the component segmentation map to improve the source of anomalies from location-level anomaly response to component-level anomaly tracing; Direction 2 (pixel features → component semantics), which uses feature differences to help adjust the edges of the component segmentation map to compensate for the shortcomings of unsupervised component segmentation in edge details.

[0126] (1) Using component partition diagrams By identifying the differences in feature maps between the teacher network and the student network, component-level anomaly tracing can be performed.

[0127] Feature differences The data includes the abnormal response intensity at each pixel location, but the abnormal response intensity itself cannot indicate the component to which the abnormality belongs. To solve this problem, the present invention utilizes a component segmentation map. The differences in characteristics between teacher networks and student networks (differences between feature maps). Label the components.

[0128] Because the component segmentation diagram is The component map is a probabilistic map. The argmax function is used to determine the most likely component category of each pixel location, thus locating the "component region" as a semantic label for anomaly tracing. Here, the first... Each component area refers to The The spatial region in the component diagram where the probability distribution values ​​of the corresponding components satisfy the preset confidence conditions.

[0129] For feature differences Each pixel position The pixel position The corresponding component category label in the component segmentation diagram is determined by the following formula: ,in Indicates the pixel position Belongs to the first The probability of class components, Label the component category to which this location belongs. This leads to the characteristic differences. Each abnormal response can be associated with the component category to which the abnormal response belongs, i.e. The pixel location that originally only had a scalar anomaly score has been expanded into a tuple carrying part semantic information (anomaly score, part category).

[0130] (2) Use the differences between the feature maps of the teacher network and the student network to help adjust the edges of the component segmentation map.

[0131] Due to factors such as lighting variations and complex textures, unsupervised component segmentation in industrial images often suffers from blurry or noisy edge regions. This invention utilizes feature differences to assist in adjusting the edges of the segmented component image.

[0132] The principle is that even in normal images, the texture, gradient, and other features change significantly at the edge regions of components. Student networks have relatively limited learning ability in these edge regions, leading to more pronounced feature differences between teacher and student networks in these areas. Therefore, the feature difference map naturally contains spatial location information about the component edges.

[0133] Specifically, in one embodiment, from the feature difference map Extracting edge response intensity: ,in The spatial gradient magnitude of the feature differences. Indicates pixel position The gradient strength at a given point, the sigmoid function maps the edge response to... Intervals are used to form edge weight masks. .

[0134] Component segmentation map using the edge weight mask The edge areas are adjusted as needed. First, the component segmentation diagram is checked. Define the edge position of each component channel and define the edge indicator function. : ,like Located at the edge region of any component channel; Otherwise. In one embodiment, the edge region is extracted from the component segmentation map using the Canny edge detection operator. Subsequently, the component segmentation map is adjusted as follows: ; in, This is the adjusted component segmentation diagram. This is the preset enhancement coefficient.

[0135] In one embodiment, the preset enhancement coefficient is taken as... .

[0136] (3) Synergistic effect of bidirectional cross-guidance Through the aforementioned bidirectional cross-guidance mechanism, a two-way information flow is established between the component segmentation map and the feature differences: Component semantics → pixel feature direction: The component segmentation map provides component semantic annotation for feature differences, enabling pixel-level abnormal responses to be traced back to specific component categories, achieving an improvement from "location-level abnormal location" to "component-level abnormal source tracing".

[0137] Pixel features → Part semantic direction: Feature differences provide edge-assisted adjustments to the part segmentation map through their edge response information, making up for the lack of edge details in unsupervised part segmentation and making the part boundaries clearer and more accurate.

[0138] The two methods mutually validate and complement each other. When factors such as changes in lighting or occlusion render feature differences unreliable, the structured component information in the component segmentation map can maintain anomaly detection; when noise exists in component segmentation, feature differences can provide auxiliary correction information. This mutual validation significantly improves the model's robustness in complex industrial scenarios.

[0139] Based on the results of the cross-guidance, update the first anomaly score and / or the second anomaly score, specifically: After the component segmentation map completes component annotation for feature differences, the feature difference map of the second processing branch carries component semantic information, and its corresponding second anomaly score is also updated accordingly. After the feature differences complete the auxiliary adjustment of the edge of the component segmentation map, the feature statistics such as component area and shape provided by the binary mask map obtained by preprocessing (upsampling, component map filtering and binarization) are more accurate, and its corresponding first anomaly score is also updated accordingly.

[0140] Through the aforementioned cross-guidance mechanism, the differences between the feature maps of the teacher network and the student network are used to trigger weight enhancement for anomalous component regions.

[0141] The weighted feature map is passed to the student network through the first distillation relation and to the autoencoder network through the second distillation relation, causing the feature map output of the autoencoder network to change accordingly, thereby updating the difference between the feature maps of the autoencoder network and the student network.

[0142] In a preferred embodiment, the step of cross-guiding the differences between the component segmentation map and the feature maps of the teacher network and the student network includes: When the first processing branch is used to determine the first... When a component malfunctions, the first component will be... The abnormal marking information of each component is input into the second processing branch; ; This represents the total number of component categories.

[0143] In the first feature extraction block of the teacher network, the anomaly marker information is used to locate the component segmentation map. The pixel positions corresponding to each component image.

[0144] The first Each component image is spatially aligned with the feature image of the first feature extraction block to obtain an aligned feature image.

[0145] The first The first component area refers to the first... The spatial region in which the probability distribution values ​​of the corresponding components in the component diagram satisfy the preset confidence conditions, that is, by comparing the first component... The probability distribution values ​​of each channel in the component image are used to determine the component category for that pixel location. The channel with the highest probability is taken as the component category. This is determined by all components labeled as the component category. The pixel positions of a component constitute the probability region of that component.

[0146] For the aligned feature map corresponding to the first The pixel positions of each component region are weighted and enhanced, so that the teacher network focuses on the first component region in subsequent feature extraction. Each component area.

[0147] The enhanced feature map of the teacher network weights is passed to the student network through the first distillation relation, and then to the autoencoder network through the second distillation relation, so that the student network and the autoencoder network synchronously enhance the feature map of the teacher network weights. Feature learning for each component.

[0148] After completing the above cross-guidance, update the first anomaly score based on the cross-guidance results. and / or second abnormal score For example, cross-guidance enhances the second branch's response to the abnormal component region, resulting in a second anomaly score. Updated to higher outliers; simultaneously, cross-guided low-resolution probability graphs are used. Edge adjustment results can be transferred back to high-resolution space to fine-tune the first anomaly score. .

[0149] Finally, in step S160, the first and second anomaly scores, updated via cross-guided analysis, are fused to obtain a fused anomaly score. The fused anomaly score is then used to determine whether the industrial image to be detected contains logical anomalies.

[0150] In one embodiment, a weighted average is used to fuse the first anomaly score and the second anomaly score: ; in, These are weighting coefficients, determined based on experience or optimization using a validation set.

[0151] Will With preset threshold Compare. If If the logic is not correct, the (industrial) image to be detected is determined to have a logical anomaly; otherwise, it is determined that there is no logical anomaly, that is, it is determined to be normal.

[0152] If an anomaly is identified, anomaly location information is further output. In this embodiment, the anomaly location is provided by the component segmentation map of the first processing branch: the difference between each component map and the normal template is calculated, and the component region with the largest difference is output as the anomaly region and marked on the original image in a visual manner.

[0153] This invention employs a dual-branch fusion approach, retaining the high sensitivity of the knowledge distillation branch to structural anomalies while enhancing the component segmentation branch's accurate detection of logical anomalies. The two branches complement and synergize, significantly improving the overall robustness and accuracy of the detection. Furthermore, the anomaly localization results can be directly traced back to specific components, exhibiting high interpretability.

[0154] In a specific embodiment, a more specific network structure and parameter settings for the above method are given: (1) Specific parameter settings for the first processing branch Image acquisition and preprocessing: The industrial camera is a CMOS camera, and the original image size is 1920×1200 pixels. The image size is normalized to a uniform pixel size of 224×224 pixels (width×height). The preprocessed image... The kernel size (window size) of the Gaussian filtering denoising is 5×5, and the standard deviation is... .

[0155] Parameter settings for the component segmentation stage: Using the DINO-ViT-S / 8 model, if the input image... The feature map size is 28×28×384 ( (28×28 is the spatial resolution, and 384 is the channel dimension).

[0156] Coreset sampling: The training set contains Each image contains a normal image, with a feature map size of 28×28 pixels, resulting in a total of 5000×784=3920000 features. The sampling rate is... Target Coreset size The coreset is constructed using the min-max facility location algorithm to obtain the pixel feature sampling set. .right Perform K-means clustering, number of clusters Five cluster centers were obtained as component feature codebooks. .

[0157] CRF optimization: Using a fully connected conditional random field (DenseCRF), iterating twice. Common parameter settings in DenseCRF: .in, To control the weights of the position kernel, To control the weights of the color kernels; It is the scale parameter (Gaussian kernel variance) of the position kernel, used to control the influence range of positional differences between pixels; It is the scale parameter of the color kernel, which controls the sensitivity to color differences between pixels; It is a smoothing term parameter used to control the smoothness of the final segmentation map (usually the weight of an additional smoothing kernel).

[0158] Cross-branching phase: CRF optimization results of the first branch Feature difference diagram with the second branch Natural alignment at spatial resolution allows for direct pixel-level cross-guiding operations. After cross-guiding is complete, the optimized cross-guiding result is obtained. .

[0159] Upsampling, component image filtering, and binarization: After cross-guided operation is completed, the component partition diagram optimized by cross-guided operation will be generated. After upsampling to the original resolution of 224×224 using bilinear interpolation, the upsampled component segmentation image is obtained. .

[0160] Then on Component image filtering and binarization are performed as follows: mean filtering is used, template size is 5×5, and noise threshold is 0.5; OTSU algorithm is used for binarization; background judgment: the binary mask covers at least two corners of the image. After filtering and binarization, The actual remaining components in the preset type Part diagram ( ),get The diagrams of each component are now in binary mask form.

[0161] Component-level feature extraction: from binary mask images Features such as component area, color, and spatial layout are extracted.

[0162] DBSCAN clustering: radius Set the average area of ​​the connected components to 10%, the minimum number of points MinPts = 10, and the connectivity rule for connected component analysis to be 8-connectivity.

[0163] Image patch division: , Each small block is 112×112 pixels in size.

[0164] Calculate the first and second balancing weights for the first outlier score: , .

[0165] Figure 2 This example demonstrates the effectiveness of small-block histograms in handling location-related logical anomalies. Specifically, Figure 2 (a) is an area distribution map of small parts in a normal image. The image is divided into 2×2 small blocks: block 1, block 2, block 3, and block 4. Figure 2(b) is a distribution map of the area of ​​small components in the logical anomaly image. The distribution of component area changes in the histogram of small components. This change in distribution reflects that the overall spatial position or arrangement order of the components (the lines connecting the small components) has become abnormal.

[0166] For each small block, a density-based noisy spatial (DBSCAN) clustering algorithm (radius) is used. , the minimum number of points For each non-zero pixel, perform connected component clustering and count the number of clusters (i.e., the number of instances of that component in that small block). If there are no non-zero pixels in a small block, the count is 0.

[0167] Therefore, for the first For each component diagram, generate a 4D component histogram vector. Each component represents the number of component instances corresponding to a small image patch.

[0168] During the training phase, each (the first) is calculated separately. The mean of the histogram vectors of the (number) small blocks ,variance and covariance matrix , .

[0169] The statistical features (mean and variance) of the four small blocks are concatenated to form a small block statistical feature vector: .

[0170] By using small-block histograms, this invention can quantitatively describe the spatial distribution patterns of component instances in an image. When components are misaligned, arranged in the wrong order, or have an abnormal number of components in a local area, the small-block histogram will show significant changes, thus accurately detecting spatial layout-related logical anomalies that are difficult to identify using traditional methods, such as… Figure 2 As shown in (b).

[0171] (2) Specific network structure settings for the second processing branch Teacher Network (T): Employs the lightweight PDN network proposed in EfficientAD, such as... Figure 3 As shown in the upper left branch, this network consists of 4 convolutional layers and 2 max-pooling layers. The specific structure is as follows: Input: Local image patch size 33×33×3, that is ; First convolutional layer (conv4x4): 4 convolutional kernels, c=128 output channels; First pooling layer (pool2x2): Window size is 2, which is 2×2 max pooling; Second convolutional layer (conv4x4): 4 convolutional kernels, c=256 output channels; Second pooling layer (pool2x2): Window size is 2; The third convolutional layer (conv3x3): 3 convolutional kernels, output channels c=256; Fourth convolutional layer (conv4x4): 4 convolutional kernels, output channels c=256; Output: Final number of output channels 384 (out=384), output feature map spatial resolution .

[0172] The teacher network was pre-trained from WideResNet-101 on the ImageNet dataset through knowledge distillation, and its parameters were frozen during subsequent training.

[0173] The network structure of the second processing branch in this embodiment is as follows: Figure 3 As shown, Figure 3 The paper clearly demonstrates the structure of the teacher network, student network, and autoencoder network, as well as the connection relationships of the three pairs of distillation losses (ST, AE, STAE).

[0174] Student network (S): such as Figure 3 As shown, the structure is exactly the same as the teacher network, but with the addition of one output branch, resulting in two output branches: the first output branch... (Imitating the teacher), second output branch (Imitation of an autoencoder). The trainable parameters of the student network are updated during normal sample training.

[0175] Autoencoder Network (AE): Employs a standard convolutional autoencoder structure. The encoder part contains multiple EncConv4x4 layers with channel numbers of 32, 32, 75, 64, 64, and 64 respectively, ultimately outputting the bottleneck layer dimension. Bottleneck characteristics ( Figure 3 The decoder consists of a DecConv4x4 layer (with channels 64, 64, 64, 64 respectively) and a final DecConv4x1 layer (c=64), outputting a 384-dimensional feature vector (out=384). Clearly, the bottleneck layer dimension satisfies... .

[0176] In the second processing branch, the hard example mining strategy during the training phase presets a loss threshold. Set the value to 0.1. The optimizer uses AdamW, and the learning rate is set to... The weight decay is set to Iterate 1000 times, with an early stop tolerance of 50 times.

[0177] During inference (detection phase): Input the industrial image to be detected. Spatial dimensions of the output feature maps of the teacher network, student network, and autoencoder network Number of channels Calculate pixel position Structural anomaly difference diagram Difference diagram of logical anomalies , fusion The image-level second anomaly score was obtained by taking the maximum value. .

[0178] (3) When performing dual-branch fusion: the weight coefficients for weighted fusion are taken as follows: (That is, it focuses more on the result of the first processing branch), because this branch is directly designed to address logical anomalies. The preset threshold for determining logical anomalies... Take 0.7, if the fusion is abnormal, the score is 0.7. If the condition is met, the image to be detected is determined to have a logical anomaly; otherwise, it is determined to be normal.

[0179] In one embodiment, such as Figure 4 As shown, the present invention also provides an apparatus for detecting logical anomalies in industrial images, comprising: Image acquisition module, used to acquire industrial images to be inspected; The branch input module is used to input the industrial image to be detected into the first processing branch and the second processing branch respectively; The first processing module is used to perform a logical anomaly detection process based on component segmentation using the first processing branch, including: segmenting the industrial image to be detected into components to generate a component segmentation map and a binary mask map; extracting component-level features from the binary mask map, wherein the component-level features include at least component area features, component color features and component spatial layout features; and calculating a first anomaly score based on the distance metric between the component-level features and a preset memory library. The second processing module is used to perform an anomaly detection process based on knowledge distillation using the second processing branch, including: inputting the industrial image to be detected into a teacher network, a student network, and an autoencoder network respectively, wherein the teacher network is a pre-trained feature extraction network with frozen parameters; and calculating a second anomaly score based on the differences between the feature maps of the teacher network and the student network and the differences between the feature maps of the autoencoder network and the student network. A cross-guiding module is used to cross-guide the differences between the component segmentation map and the feature maps of the teacher network and the student network; and to update the first anomaly score and / or the second anomaly score based on the result of the cross-guiding. The fusion judgment module is used to fuse the first abnormal score and the second abnormal score after the cross-guided update to obtain a fused abnormal score; if the fused abnormal score is greater than or equal to a preset abnormal threshold, it is determined that the industrial image to be detected has a logical abnormality; otherwise, it is determined that there is no logical abnormality.

[0180] The first processing module includes at least: a component segmentation unit for generating a component segmentation map; a component feature extraction unit for extracting component-level features from the component segmentation map, including at least component area and component color; and a first score calculation unit for calculating a first anomaly score based on the distance metric between the component-level features and a preset memory library.

[0181] The second processing module includes at least: a pre-trained teacher network; a student network; and a second score calculation unit, used to calculate a second anomaly score based on the differences between the feature maps of the teacher network and the student network and the differences between the feature maps of the autoencoder network and the student network.

[0182] In one embodiment, the knowledge distillation process includes three pairs of distillation relations: a first distillation relation constrains the student network to learn the feature map output of the teacher network; a second distillation relation constrains the autoencoder network to learn and reconstruct the feature map output of the teacher network; and a third distillation relation constrains the student network to learn the reconstruction error of the autoencoder network.

[0183] In the cross-guidance module, the component segmentation map is used to identify the components to which each region belongs in the difference between the feature maps of the teacher network and the student network, and the difference between the feature maps of the teacher network and the student network is used to assist in adjusting the edges of the component segmentation map.

[0184] To verify the effectiveness of the method provided in this invention, extensive comparative experiments were conducted on the MVTec LOCO anomaly detection dataset. The experimental program was implemented based on the PyTorch framework and ran on a single Nvidia RTX4090 graphics card. The experiments compared existing methods based on feature embedding (SimpleNet, PatchCore), knowledge distillation (AST, EfficientAD), supervised segmentation (PSAD), and unsupervised segmentation (ComAD). SimpleNet, PatchCore, AST, and EfficientAD are typical non-segmentation algorithms.

[0185] All comparative experiments were based on the official implementation and default parameters of the PyTorch framework. The MVTec LOCO dataset contains five subclasses: breakfast boxes, juice bottles, pins, screw bags, and connectors. Logical anomaly detection (LA) and structural anomaly detection (SA) were performed on each subclass. Table 1 shows the AUC (Area Under the Curve) performance comparison results of different methods on the MVTec LOCO dataset. AUC specifically refers to the area under the ROC curve (Receiver Operating Characteristic Curve). The AUC value measures the probability that the model ranks the predicted score of "abnormal samples" higher than that of "normal samples," reflecting the model's ability to rank positive and negative samples.

[0186] Table 1. Comparison of AUC performance of different methods on the MVTec LOCO dataset.

[0187] As can be seen from the data in Table 1: (1) In terms of logical anomaly detection, the average AUC of this invention reaches 91.4%, which is significantly better than all unsupervised comparison methods (EfficientAD 87.5%, ComAD 89.8%), and achieves 100% detection accuracy in the three subclasses of juice bottles, pins, and connectors. This fully verifies the effectiveness of the component-level feature extraction and spatial layout analysis mechanism proposed in this invention for logical anomaly detection.

[0188] (2) In terms of structural anomaly detection, the average AUC of the present invention reaches 93.9%, which is comparable to the current state-of-the-art EfficientAD (93.6%) and is superior to other comparative methods, indicating that the present invention improves the ability to detect logical anomalies without sacrificing the accuracy of structural anomaly detection.

[0189] (3) In terms of overall performance, the overall average AUC (92.7%) of the present invention is better than all unsupervised comparison methods, which proves the effectiveness and superiority of the dual-branch collaborative detection architecture and cross-guided mechanism.

[0190] On the other hand, in one embodiment, the present invention provides a computer device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the industrial image logic anomaly detection method provided in any of the above embodiments. The computer device may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores sample data. The network interface of the computer device is used for communication with external terminals via a network connection.

[0191] On the other hand, in one embodiment of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for detecting industrial image logic anomalies provided in any of the above embodiments.

[0192] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0193] Matters not covered in this invention are common knowledge.

[0194] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0195] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

[0196] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting logical anomalies in industrial images, characterized in that, include: Acquire industrial images to be inspected; The industrial images to be detected are input into the first processing branch and the second processing branch respectively; The first processing branch is used to perform a logical anomaly detection process based on component segmentation, including: segmenting the industrial image to be detected into components to generate a component segmentation map and a binary mask map; extracting component-level features from the binary mask map, wherein the component-level features include at least component area features, component color features and component spatial layout features; and calculating a first anomaly score based on the distance metric between the component-level features and a preset memory library. The second processing branch is used to perform an anomaly detection process based on knowledge distillation, including: inputting the industrial image to be detected into a teacher network, a student network, and an autoencoder network, respectively, wherein the teacher network is a pre-trained feature extraction network with frozen parameters; and calculating a second anomaly score based on the differences between the feature maps of the teacher network and the student network and the differences between the feature maps of the autoencoder network and the student network. The component segmentation map is cross-guided with the differences between the feature maps of the teacher network and the student network; the first anomaly score and / or the second anomaly score are updated based on the result of the cross-guided analysis. The first anomaly score and the second anomaly score, which have been updated by the cross-guided method, are fused to obtain a fused anomaly score. If the fused anomaly score is greater than or equal to a preset anomaly threshold, it is determined that the industrial image to be detected has a logical anomaly; otherwise, it is determined that there is no logical anomaly.

2. The method for detecting logical anomalies in industrial images according to claim 1, characterized in that, The generation of the component segmentation map and the binary mask map includes: The industrial image to be detected is used to extract features using a pre-trained visual Transformer model to obtain multi-scale feature maps. A component feature codebook was constructed using Coreset sampling and K-means clustering. An initial component segmentation map is generated based on the distance from the feature vector of each pixel position in the feature map to the component feature codebook. The initial component segmentation map is optimized using a conditional random field to obtain a new component segmentation map. The component segmentation image is upsampled to the original resolution of the industrial image to be detected to obtain the upsampled component segmentation image; The upsampled component segmentation image is filtered and binarized to obtain a binary mask image.

3. The method for detecting logical anomalies in industrial images according to claim 1, characterized in that: The component area feature is determined by the number of pixels with a pixel value of 1 after binarization in each component image of the binary mask image; The color characteristics of the component are obtained by calculating the ratio of the a-channel value to the b-channel value in the CIELAB color space. The spatial layout characteristics of the components include component instance counting characteristics and small block area distribution characteristics; The component instance counting feature is used to describe the quantity distribution of component instances; The small patch area distribution feature is used to describe the distribution of component area features in the industrial image space.

4. The method for detecting logical anomalies in industrial images according to claim 3, characterized in that, The component instance count feature is extracted in the following way: For each component image of the binary mask image, a density-based clustering algorithm is used to cluster the component instances to obtain multiple clusters; Count the number of component instances in each cluster and generate a component histogram vector; The component histogram vector is normalized, and the normalized component histogram vector is used as the component instance counting feature; The small patch area distribution features are extracted in the following way: The industrial image to be detected is uniformly divided into... A few small image patches, in which , It is a positive integer; For each image patch, calculate the component area feature of each part and generate a patch histogram vector as the patch area distribution feature.

5. The method for detecting logical anomalies in industrial images according to claim 4, characterized in that, The first anomaly score is calculated based on the distance metric between the component-level features and the preset memory library, including: Based on the component feature vector formed by the component area features and component color features of the industrial image to be detected, a first distance is calculated between the component feature vector and the first reference feature in the memory database. The first reference feature in the memory library is the set of component feature vectors of all normal images in the training set. Calculate the second distance between the component instance count feature of the industrial image to be detected and the second reference feature in the memory library. The second reference feature in the memory library is a set of component instance count features for all normal images in the training set. Calculate the third distance between the small patch area distribution features of the industrial image to be detected and the third reference feature in the memory library. The third reference feature in the memory library is a statistical feature vector of small blocks obtained by concatenating the mean and variance of the area distribution features of all small image blocks. The first anomaly score is obtained by weighted summation of the first, second, and third distances: ； in, and These are the first balancing weight and the second balancing weight, respectively.

6. The method for detecting logical anomalies in industrial images according to claim 1, characterized in that, In the second processing branch, the knowledge distillation process includes three pairs of distillation relationships: The first distillation relation constrains the student network to learn the feature map output of the teacher network; The second distillation relation constrains the autoencoder network to learn and reconstruct the feature map output of the teacher network; The third distillation relation constrains the student network to learn the reconstruction error of the autoencoder network; The student network and the autoencoder network are trained in the following manner: Train using a training set that contains only normal images; The bottleneck layer dimension of the autoencoder network is limited to one-fifth of the number of feature map channels output by the teacher network, forcing the autoencoder network to learn the global feature distribution of a normal image and thus preventing it from accurately reconstructing local details. Parameter optimization is performed using a difficult case mining strategy: Calculate the pixel-by-pixel loss between the student network and the teacher network, and the pixel-by-pixel loss between the student network and the autoencoder network; Backward gradient propagation is performed on the pixel locations where the pixel-by-pixel loss value is greater than a preset loss threshold, so that the student network focuses on learning the regions where the teacher network and the autoencoder network respond strongly.

7. The method for detecting logical anomalies in industrial images according to claim 6, characterized in that, The step of cross-guiding the differences between the component segmentation map and the feature maps of the teacher network and the student network includes: When the first processing branch is used to determine the first... When a component malfunctions, the first component will be... The abnormal marking information of each component is input into the second processing branch; ; Total number of component categories; In the first feature extraction block of the teacher network, the anomaly marker information is used to locate the component segmentation map. The pixel positions corresponding to each component image; The first Each component image is spatially aligned with the feature image of the first feature extraction block to obtain an aligned feature image; For the aligned feature map corresponding to the first The pixel positions of each component region are weighted and enhanced, so that the teacher network focuses on the first component region in subsequent feature extraction. The component area; the first The first component area refers to the first... The spatial region in which the probability distribution values ​​of the corresponding components in the component diagram satisfy the preset confidence conditions; The enhanced feature map of the teacher network weights is passed to the student network through the first distillation relation, and then to the autoencoder network through the second distillation relation, so that the student network and the autoencoder network synchronously enhance the feature map of the teacher network weights. Feature learning for each component.

8. A device for detecting logical anomalies in industrial images, characterized in that, include: Image acquisition module, used to acquire industrial images to be inspected; The branch input module is used to input the industrial image to be detected into the first processing branch and the second processing branch respectively; The first processing module is used to perform a logical anomaly detection process based on component segmentation using the first processing branch, including: segmenting the industrial image to be detected into components to generate a component segmentation map and a binary mask map; extracting component-level features from the binary mask map, wherein the component-level features include at least component area features, component color features and component spatial layout features; and calculating a first anomaly score based on the distance metric between the component-level features and a preset memory library. The second processing module is used to perform an anomaly detection process based on knowledge distillation using the second processing branch, including: inputting the industrial image to be detected into a teacher network, a student network, and an autoencoder network respectively, wherein the teacher network is a pre-trained feature extraction network with frozen parameters; and calculating a second anomaly score based on the differences between the feature maps of the teacher network and the student network and the differences between the feature maps of the autoencoder network and the student network. A cross-guiding module is used to cross-guide the differences between the component segmentation map and the feature maps of the teacher network and the student network; and to update the first anomaly score and / or the second anomaly score based on the result of the cross-guiding. The fusion judgment module is used to fuse the first abnormal score and the second abnormal score after the cross-guided update to obtain a fused abnormal score; if the fused abnormal score is greater than or equal to a preset abnormal threshold, it is determined that the industrial image to be detected has a logical abnormality; otherwise, it is determined that there is no logical abnormality.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for detecting logical anomalies in industrial images as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for detecting logical anomalies in industrial images as described in any one of claims 1 to 7.

Images