An open-world oriented background-defect joint modeling defect detection method and system

Abstract

The application discloses an open-world-oriented background-defect joint modeling defect detection method and system, first acquires an industrial dataset and pre-processes; extracts a to-be-detected feature and a normal background reference feature respectively, constructs a background feature robust manifold, and performs feature alignment and fusion through a defect-background interactive learning network to sharpen a discrimination boundary; a global-local collaborative anomaly measurement strategy is executed to generate a collaborative anomaly measurement score; a hybrid proposal box generation mechanism based on semantic and anomaly double driving is designed to generate a joint proposal box as a location prior, which is sent into a decoding network to output a detection result; in the training stage, a multi-task joint loss optimization target is constructed to force the model to establish high sensitivity to abnormal background features; in the test, an open set classification discrimination logic and a post-processing algorithm are used to eliminate redundant prediction, and finally the defect category and the boundary box coordinates are output. The application effectively breaks through the closed set detection limitation and significantly improves the discovery ability and detection robustness of unknown defects outside the distribution.

Description

Technical Field

[0001] This invention belongs to the field of industrial defect detection and computer vision technology, and in particular relates to a defect detection method and system for background-defect joint modeling in an open world. Background Technology

[0002] Industrial defect detection is a core component of intelligent manufacturing and product quality control, widely applied in key sectors such as precision electronics, automotive parts, and semiconductor manufacturing. In modern assembly line production, real-time, high-precision defect detection of product surfaces effectively eliminates substandard products, optimizes processes, and reduces production costs. Therefore, building efficient and reliable defect detection models is of significant economic and strategic importance for ensuring the final delivery quality of industrial products and eliminating potential safety hazards.

[0003] Currently, deep learning-based object detection technology has become the mainstream solution. These methods mainly adopt a fully supervised learning paradigm, where specific types of defect samples are manually collected and labeled during the training phase. The model learns the feature distribution of these samples to establish its ability to distinguish these defects. Therefore, the detection capability of existing models is strictly limited to the predefined set of categories defined in the training set. However, actual industrial production is a dynamic and open scenario. Due to fine-tuning of production processes, changes in raw material batches, or sudden equipment failures, unknown defect classes not included in the training set often appear during the testing phase. The morphological characteristics of these defects are completely different from known classes and cannot be predefined. Constrained by the traditional closed-set assumption, existing models tend to force all input features to be mapped to a predefined category space or background space. This leads to serious missed detection problems when encountering unseen defects, as the model often cannot classify them as known defects and incorrectly classifies them as normal background.

[0004] Defects and shortcomings of existing technology: 1. Identifying Closed Logic: Existing methods rely on the closed-set assumption that training and testing are identically distributed. During the inference phase, when the model receives features of unknown defect classes, due to the lack of an independent anomaly representation space, the network forces these out-of-distribution features to be mapped to a predefined known category space or normal background space, leading to the misclassification or missed detection of unknown defects.

[0005] 2. Lack of anomaly detection: Existing models generally employ discriminative binary classification logic, treating "background" merely as the remaining region that does not contain known targets, without actively modeling the feature distribution of normal background. When faced with anomalous samples that are neither known defects nor conform to the normal background, the model lacks an effective distance metric and cannot calculate feature deviation, leading to a failure in defect detection capabilities.

[0006] 3. One-sided detection criteria: Current mainstream detection frameworks rely primarily on target-oriented priors learned from known defects when generating candidate boxes. This screening criterion leads to unknown morphological anomalies being treated as low-confidence background noise during the initial feature extraction screening stage, fundamentally blocking the possibility of unknown defects entering subsequent detection processes.

[0007] In summary, existing industrial defect detection methods, limited by the closed-set assumption and static background modeling, struggle to handle dynamically emerging unknown defects in open worlds. The fundamental problem lies in the lack of explicit modeling of normal background features and the ability to proactively perceive anomalous distributions, causing the model to fail when encountering out-of-distribution samples. Therefore, a novel method is urgently needed that can break down closed-set boundaries and jointly model background and defect distributions to improve the ability to discover and detect unknown defects in open environments. Summary of the Invention

[0008] To address the above technical problems, this invention provides a method and system for defect detection using background-defect co-modeling in an open world.

[0009] The technical solution adopted by this invention to solve its technical problem is: A defect detection method for background-defect joint modeling in an open world includes the following steps: S100: Obtain an industrial defect dataset containing the image to be detected and defect-free normal sample images, and preprocess it to construct a standardized feature tensor; S200: The feature extraction network extracts the features to be detected and the normal background reference features respectively. Based on the normal background reference features, a robust manifold representing the defect-free state of the background features is constructed. The feature to be detected and the robust manifold of the background features are aligned and fused through the defect-background interaction learning network to sharpen the discrimination boundary between the background and the defect. S300: Based on the feature interaction reconstruction results, a global-local collaborative anomaly measurement strategy is executed to generate a collaborative anomaly measurement score for measuring the degree of anomalies in the overall image and local regions. S400: Design a hybrid proposal box generation mechanism based on semantics and anomalies. It calculates the target score of the known category and the score of the unknown anomaly respectively, generates a joint proposal box through a two-branch hybrid sorting, and feeds the joint proposal box as a position prior into the decoding network to output the detection result. S500: Construct a multi-task joint loss optimization objective based on known class supervision and abnormal prior guidance. During the training phase, supervised learning is performed using samples with known class labels, forcing the model to establish a high sensitivity to abnormal background features, so as to achieve the ability to capture unknown defects outside the distribution during the testing phase. S600: Input the industrial image to be detected into the trained model, obtain the original prediction box containing the known category probability and anomaly score, remove redundant predictions through open set classification and discrimination logic and post-processing algorithm, and output the final defect category and bounding box coordinates.

[0010] In one embodiment, S100 includes: S110: Acquire real-world images of defective industrial components. and its corresponding defect categories and true bounding box annotation information. To construct the sample set to be tested ,in This indicates the total number of samples to be tested; S120: Obtain images of defect-free, normal components. This is used as a benchmark reference set to provide prior information on the distribution of normal background features. ,in This represents the total number of normal samples, which is used to build the normal feature library later. S130: Sample set to be tested With reference set The images in the dataset undergo uniform spatial scale alignment and normalization, outputting a normalized tensor of the features to be detected that satisfies the input specifications of the detection network. (1) (2) in, The image to be detected after preprocessing. This is the normal image after preprocessing. This represents a spatial geometric transformation function that includes multi-scale random scaling and horizontal flipping. This represents the image tensor quantization and pixel normalization processing functions.

[0011] In one embodiment, S200 includes: S210: The preprocessed image to be detected... Compared with normal images A convolutional backbone network with shared input parameters extracts features, and channel dimensionality reduction is performed through a projection layer to obtain multi-scale features for testing. Compared with normal reference features : (3) (4) in, This indicates a joint feature extraction operation involving convolutional layers and channel-wise dimension reduction projection; S220: In order to establish an absolute metric for open set detection within the feature space, multiple batches of extracted normal reference features will be used. Flattening the matrix in the spatial dimension and stitching it across batches, a robust manifold matrix representing the distribution of pure background features is constructed. : (5) in, This represents a flattening operation that converts the spatial dimension of the feature map to the sequence dimension. This indicates an operation that stitches together the flattened features of multiple normal samples along the sample dimensions; S230: Introduces a defect-background feature interactive learning mechanism to learn the features to be tested. Mapped to query vector The robust manifold matrix of background features Mapped to a key-value pair matrix and The cross-attention mechanism is used to extract the most relevant normal visual semantics from the background manifold to the features to be detected, and then feature alignment and fusion are performed. (6) in, This indicates the cross-attention fusion operation. This is the feature representation after interactive reconstruction.

[0012] In one embodiment, S300 includes: S310: Perform global anomaly measurement: for features to be detected Global average pooling is performed to obtain the global query vector, and the robust manifold matrix of background features is calculated. The mean feature vector along the sample dimension is used to obtain the global feature residual, which is then input into a scoring network to generate a global anomaly score that measures the degree of global anomaly. : (7) (8) (9) in, This indicates a global average pooling operation. This indicates the calculation of the mean. This represents a multilayer perceptron network used to calculate feature deviation. S320: Perform local anomaly measurement: Calculate the feature to be detected at each spatial location of the feature map. Interactive reconstruction features output from S200 Pixel-by-pixel comparison is performed, the maximum similarity at each location is taken, and its residual is calculated to generate a pixel-level anomaly feature map reflecting the distribution of local anomalies. : (10) in, This function represents the cosine similarity between query features and manifold features. S330: Constructing a collaborative verification and fusion mechanism: Extracting local anomaly feature maps The global extreme response component and its correlation with the global anomaly score. Weighted fusion is performed to generate the final collaborative anomaly measurement score. : (11) in, This indicates the extraction of the maximum value from the local anomaly feature map. and These are the preset fusion weight coefficients.

[0013] In one embodiment, S400 includes: S410: Obtain the interactive reconstruction features output by S200 The classification prediction head is used to calculate the target score of each spatial pixel belonging to a known defect category. Simultaneously, the robust manifold matrix of background features constructed in S200 is invoked. , computational interaction reconstructed features With manifold matrix The similarity residuals in the feature space are used to generate pixel-level anomaly scores that reflect the degree of out-of-distribution deviation. : (12) (13) in, It is the Sigmoid activation function. For a linear classification mapping network with known category objectives, This is an operation for calculating similarity residuals; S420: Establish a dual-branch hybrid proposal box allocation mechanism: Set the total number of proposal boxes required by the decoder to... The capacity space is orthogonally divided into proposal boxes driven by known semantics. With candidate proposal boxes driven by unknown anomalies Based on target scores respectively Pixel-level anomaly scores Perform a descending sort, obtain the corresponding highest-scoring spatial location index, and concatenate them into a composite index set. : (14) (15) (16) in, This represents an index retrieval operation that extracts the spatial locations of the K largest values ​​in a sequence. This represents the concatenation operation of index tensors; S430: Extracting High-Dimensional Spatial Features and Location Priors: Utilizing Joint Index Sets From the dense set of candidate boxes generated by the feature encoder, the coordinates of the corresponding initial proposal boxes are precisely converged. Then, the initial proposal boxes are fed into the decoder as positional priors for subsequent cross-attention calculations and fine regression of defect bounding boxes. (17) in, The dense candidate box coordinate tensor output by the encoder. This indicates an operation to extract the corresponding coordinate position based on the index. These are the coordinates of the joint proposal frame that are ultimately sent to the decoder; S440: Perform decoding transformation and final detection prediction: Calculate the coordinates of the selected joint proposal boxes. As an initial position prior, combined with encoded features The data is fed into a decoding network consisting of multiple layers of cross-attention and self-attention for feature updating and decoding. Subsequently, a multi-task joint prediction network performs class probability mapping and fine-grained bounding box regression in a unified manner, directly outputting the final set of prediction results. : (18) in, This represents a multi-task joint prediction mapping network that includes a feedforward neural network. and The first The probability distribution of the categories and location coordinates of each predicted target.

[0014] In one embodiment, S500 includes: S510: Perform optimal assignment based on known class labels: Obtain the set of prediction results from the S400 output. And introduce a set of real labels that only contain known defect categories. The Hungarian algorithm is used to calculate the minimum matching cost between the predicted results and known defects, and to determine the positive sample allocation relationship of known categories. S520: Calculate the discriminative supervision loss for known categories: for those successfully matched in S510 The prediction results are used to calculate the cross-entropy classification loss between the predicted probability distribution and the true class. And the location regression loss between the predicted bounding box and the true bounding box. ; S530: Calculate the anomaly separation loss based on background manifold constraints: To overcome the limitations of known class labels and endow the model with the ability to detect unknown defects, an anomaly feature separation loss is introduced. The collaborative anomaly metric score output from S300 Apply differential constraints: (19) When the image to be tested is a normal image, The value is 0 when the image under test is an abnormal image. =1; S540: Perform multi-task joint optimization and parameter update: Sum the above loss terms with weights to construct the total loss for end-to-end training. : (20) in, , , The hyperparameter weights are minimized using the backpropagation algorithm. The gradient descent optimizer is used to iteratively update the overall network parameters to complete the training of the industrial defect open set target detection model.

[0015] In one embodiment, S600 includes: S610: During inference, the input image is fed into the trained network model, directly passing through the feature interaction and decoding networks from S100 to S400, and outputting the original detection result set. (twenty one) in, To predict the known class probability distribution of the head output, For collaborative anomaly measurement scores, The coordinates of the original defect bounding box obtained from regression; S620: Perform open set classification calibration: based on a set classification confidence threshold. With the abnormal score threshold Perform open set attribute discrimination for each prediction box: (twenty two) in, and These represent the maximum confidence level in the predicted probability distribution and its corresponding index of the known defect category, respectively. Indicates an unknown category of defect outside the distribution; This indicates normal background noise; S630: Perform redundant prediction box filtering: For the detection results after open set classification calibration, non-maximum suppression is used to calculate the spatial intersection-union ratio between each bounding box. When the intersection-union ratio between prediction boxes of the same class is greater than the set overlap threshold, the prediction box with the highest comprehensive score is retained, and the remaining redundant overlapping boxes are suppressed and removed. S640: Output the final detection result: The target bounding box coordinate set after redundancy filtering and its corresponding defect attributes are reverse-mapped to the scale of the original input image, and the final industrial defect detection report is output and saved to achieve accurate interception of unknown defects in real complex environments.

[0016] A background-defect co-modeling defect detection system for open worlds, used to execute a background-defect co-modeling defect detection method for open worlds, includes: The data preprocessing module is used to acquire an industrial defect dataset containing images of the images to be detected and images of normal samples without defects, and to preprocess and construct a standardized feature tensor. The feature extraction and interaction module is used to extract the features to be detected and the normal background reference features using the feature extraction network. Based on the normal background reference features, a robust manifold representing the defect-free state of the background features is constructed. The feature to be detected and the robust manifold of the background features are aligned and fused through the defect-background interaction learning network to sharpen the discrimination boundary between the background and the defect. The collaborative anomaly measurement module is used to execute a global-local collaborative anomaly measurement strategy based on the feature interaction reconstruction results, and generate a collaborative anomaly measurement score to measure the degree of anomalies in the overall image and local regions. The decoding and prediction module is used to design a hybrid proposal box generation mechanism based on semantics and anomalies. It calculates the target score of the known category and the score of the unknown anomaly, generates a joint proposal box through a two-branch hybrid sorting, and feeds the joint proposal box as a position prior into the decoding network to output the detection result. The training optimization module is used to construct a multi-task joint loss optimization objective based on known class supervision and abnormal prior guidance. During the training phase, it uses samples with known class labels for supervised learning, forcing the model to establish a high sensitivity to abnormal background features, so as to achieve the ability to capture unknown defects outside the distribution during the testing phase. The inference post-processing module is used to input the industrial image to be detected into the trained model, obtain the original prediction box containing the known category probability and anomaly score, remove redundant predictions through open set classification and discrimination logic and post-processing algorithm, and output the final defect category and bounding box coordinates.

[0017] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a background-defect co-modeling defect detection method for an open world.

[0018] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a background-defect co-modeling defect detection method for an open world.

[0019] The aforementioned background-defect joint modeling defect detection method and system for open worlds sharpens the distinction between background and defects by constructing a robust manifold of background features and using a defect-background interaction learning network for feature alignment and fusion. Secondly, it employs a global-local collaborative anomaly measurement strategy to generate collaborative anomaly measurement scores, accurately characterizing the differences in anomaly distribution across multiple scales and enhancing the robustness of detecting small, unknown defects. Thirdly, it designs a hybrid proposal box generation mechanism driven by both semantics and anomalies, generating joint proposal boxes as positional priors through dual-branch hybrid sorting, effectively overcoming the bottleneck of closed-set detection and preventing unknown defects from being misclassified as background. Finally, it constructs a multi-task joint loss optimization objective, forcing the model to establish high sensitivity to abnormal background features using only known class labels, achieving the ability to capture out-of-distribution unknown defects under end-to-end learning. This significantly improves the detection capability and overall robustness of unknown defects in open scenarios. Attached Figure Description

[0020] Figure 1 This is a flowchart of a background-defect joint modeling and defect detection method for open worlds according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of a background-defect joint modeling and defect detection method for open worlds in one embodiment of the present invention. Figure 3 This is a diagram showing the detection results of the training method in one embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0022] This invention provides a background-defect joint modeling defect detection method and system for open worlds, aiming to solve the problem that existing industrial defect detection methods are limited by the closed-set assumption, leading to the easy misclassification of unknown defects as background during testing and resulting in missed detections. This invention actively constructs a robust manifold of background features by introducing a normal feature library, and combines bi-branch feature interaction and global-local collaborative anomaly measurement to achieve precise decoupling of background and defects in the feature space. Simultaneously, it proposes a hybrid proposal box generation mechanism based on semantic and anomaly dual-driven approaches, as well as an orthogonal discrimination post-processing strategy, effectively improving the joint detection accuracy and robustness of known and unknown defects in open scenarios. The following provides a detailed description of the various technical features and implementation schemes of this invention. The overall flowchart is shown below. Figure 1 As shown in the diagram, the overall principle is illustrated below. Figure 2 As shown, the technical solution adopted in this invention is as follows: (a) S100: Obtain an industrial defect dataset containing the image to be detected and normal sample images without defects, and perform preprocessing to construct a standardized feature tensor.

[0023] Specifically, a pre-constructed industrial defect dataset containing the industrial image to be detected and the defect-free normal sample image is obtained. Adaptive contrast enhancement, multi-scale size normalization and tensor normalization preprocessing operations are performed on the industrial image to be detected and the defect-free normal sample image respectively to construct a standardized feature tensor that meets the input requirements of the detection model.

[0024] In one embodiment, S100 includes: S110: Acquire real-world images of defective industrial components. and its corresponding defect categories and true bounding box annotation information. To construct the sample set to be tested ,in This indicates the total number of samples to be tested; S120: Obtain images of defect-free, normal components. This is used as a benchmark reference set to provide prior information on the distribution of normal background features. ,in This represents the total number of normal samples, which is used to build the normal feature library later. S130: Sample set to be tested With reference set The images in the dataset undergo uniform spatial scale alignment and normalization, outputting a normalized tensor of the features to be detected that satisfies the input specifications of the detection network. (1) (2) in, The image to be detected after preprocessing. This is the normal image after preprocessing. This represents a spatial geometric transformation function that includes multi-scale random scaling and horizontal flipping. This represents the image tensor quantization and pixel normalization processing functions.

[0025] (ii) S200: The feature extraction network is used to extract the features to be detected and the normal background reference features respectively. Based on the normal background reference features, a robust manifold representing the defect-free state of the background features is constructed. The feature to be detected and the robust manifold of the background features are aligned and fused through the defect-background interaction learning network to sharpen the discrimination boundary between the background and the defect.

[0026] Specifically, the standardized tensor of features to be detected and the normal feature tensor are input into a pre-trained feature extraction network to extract multi-scale features to be detected and normal background reference features. The normal background reference features are used to construct a robust manifold representing the defect-free state. Subsequently, through a defect-background interaction learning network, a cross-attention mechanism is introduced to align and fuse the features to be detected and the robust manifold of background features, fully learning and sharpening the discrimination boundary between the background and defects.

[0027] In one embodiment, S200 includes: S210: The preprocessed image to be detected... Compared with normal images A convolutional backbone network with shared input parameters extracts features, and channel dimensionality reduction is performed through a projection layer to obtain multi-scale features for testing. Compared with normal reference features : (3) (4) in, This indicates a joint feature extraction operation involving convolutional layers and channel-wise dimension reduction projection; S220: In order to establish an absolute metric for open set detection within the feature space, multiple batches of extracted normal reference features will be used. Flattening the matrix in the spatial dimension and stitching it across batches, a robust manifold matrix representing the distribution of pure background features is constructed. : (5) in, This represents a flattening operation that converts the spatial dimension of the feature map to the sequence dimension. This indicates an operation that stitches together the flattened features of multiple normal samples along the sample dimensions; S230: Introduces a defect-background feature interactive learning mechanism to learn the features to be tested. Mapped to query vector The robust manifold matrix of background features Mapped to a key-value pair matrix and The cross-attention mechanism is used to extract the most relevant normal visual semantics from the background manifold to the features to be detected, and then feature alignment and fusion are performed. (6) in, This indicates the cross-attention fusion operation. This is the feature representation after interactive reconstruction.

[0028] Through this reconstruction process, the network forces the use of a clean background manifold to represent the current image, thereby effectively widening the distance between unknown defects and normal background in the feature space, and fully learning and sharpening the discrimination boundary between the background and defects.

[0029] (III) S300: Based on the feature interaction reconstruction results, execute the global-local collaborative anomaly measurement strategy to generate a collaborative anomaly measurement score for measuring the degree of anomaly in the overall image and local regions.

[0030] Specifically, based on the aforementioned feature interaction reconstruction results, a global-local collaborative anomaly measurement strategy is executed. Global representations of the robust manifolds of the detected features and background features are extracted separately, global feature deviation is calculated, and a global anomaly score is generated. Simultaneously, the similarity residual between the detected features and the background manifold is calculated in local regions to generate pixel-level local anomaly feature maps. Finally, a collaborative verification fusion mechanism is used to weightedly fuse the global anomaly score and the extreme value responses of the local anomaly maps, outputting a comprehensive anomaly measurement score. This achieves effective differentiation between normal backgrounds and potential unknown defects at both global and local scales.

[0031] In one embodiment, S300 includes: S310: Perform global anomaly measurement: for features to be detected Global average pooling is performed to obtain the global query vector, and the robust manifold matrix of background features is calculated. The mean feature vector along the sample dimension is used to obtain the global feature residual, which is then input into a scoring network to generate a global anomaly score that measures the degree of global anomaly. : (7) (8) (9) in, This indicates a global average pooling operation. This indicates the calculation of the mean. This represents a multilayer perceptron network used to calculate feature deviation. S320: Perform local anomaly measurement: Calculate the feature to be detected at each spatial location of the feature map. Interactive reconstruction features output from S200 Pixel-by-pixel comparison is performed, the maximum similarity at each location is taken, and its residual is calculated to generate a pixel-level anomaly feature map reflecting the distribution of local anomalies. : (10) in, This function represents the cosine similarity between query features and manifold features. S330: Constructing a collaborative verification and fusion mechanism: Extracting local anomaly feature maps The global extreme response component and its correlation with the global anomaly score. Weighted fusion is performed to generate the final collaborative anomaly measurement score. : (11) in, This indicates the extraction of the maximum value from the local anomaly feature map. and These are the preset fusion weight coefficients.

[0032] This collaborative verification mechanism effectively compensates for the problem that single-scale methods are easily affected by local noise, and achieves accurate and robust measurement of unknown defects.

[0033] (iv) S400: Design a hybrid proposal box generation mechanism based on semantics and anomaly dual drive. Calculate the target score of known categories and the score of unknown anomalies respectively. Generate a joint proposal box through dual-branch hybrid sorting. The joint proposal box is sent as a position prior to the decoding network and the detection result is output.

[0034] Specifically, a hybrid proposal box generation mechanism based on semantic and anomaly dual-driven approaches is designed. During the feature encoding stage, the known class targetability score and the unknown pixel-level anomaly score of the feature to be detected are calculated separately. Through a dual-branch hybrid sorting mechanism, proposal box extraction slots are independently allocated to known semantic defects and potential unknown anomalies, generating joint proposal boxes. This avoids anomalies of unknown morphology being filtered as background noise during the initial feature screening stage. Subsequently, the joint proposal boxes are fed into the decoding network as location priors, outputting the final classification results of known and unknown class defects and the bounding box position coordinates.

[0035] In one embodiment, S400 includes: S410: Obtain the interactive reconstruction features output by S200 The classification prediction head is used to calculate the target score of each spatial pixel belonging to a known defect category. Simultaneously, the robust manifold matrix of background features constructed in S200 is invoked. , computational interaction reconstructed features With manifold matrix The similarity residuals in the feature space are used to generate pixel-level anomaly scores that reflect the degree of out-of-distribution deviation. : (12) (13) in, It is the Sigmoid activation function. For a linear classification mapping network with known category objectives, This is an operation for calculating similarity residuals; S420: Establish a dual-branch hybrid proposal box allocation mechanism: Set the total number of proposal boxes required by the decoder to... The capacity space is orthogonally divided into proposal boxes driven by known semantics. With candidate proposal boxes driven by unknown anomalies Based on target scores respectively Pixel-level anomaly scores Perform a descending sort, obtain the corresponding highest-scoring spatial location index, and concatenate them into a composite index set. : (14) (15) (16) in, This represents an index retrieval operation that extracts the spatial locations of the K largest values ​​in a sequence. This represents the concatenation operation of index tensors; S430: Extracting High-Dimensional Spatial Features and Location Priors: Utilizing Joint Index Sets From the dense set of candidate boxes generated by the feature encoder, the coordinates of the corresponding initial proposal boxes are precisely converged. Then, the initial proposal boxes are fed into the decoder as positional priors for subsequent cross-attention calculations and fine regression of defect bounding boxes. (17) in, The dense candidate box coordinate tensor output by the encoder. This indicates an operation to extract the corresponding coordinate position based on the index. These are the coordinates of the joint proposal frame that are ultimately sent to the decoder; S440: Perform decoding transformation and final detection prediction: Calculate the coordinates of the selected joint proposal boxes. As an initial position prior, combined with encoded features The data is fed into a decoding network consisting of multiple layers of cross-attention and self-attention for feature updating and decoding. Subsequently, a multi-task joint prediction network performs class probability mapping and fine-grained bounding box regression in a unified manner, directly outputting the final set of prediction results. : (18) in, This represents a multi-task joint prediction mapping network that includes a feedforward neural network. and The first The probability distribution of the categories and location coordinates of each predicted target.

[0036] (v) S500: Construct a multi-task joint loss optimization objective based on known class supervision and abnormal prior guidance. During the training phase, use samples with known class labels for supervised learning to force the model to establish a high sensitivity to abnormal background features, so as to achieve the ability to capture unknown defects outside the distribution during the testing phase.

[0037] Specifically, a multi-task joint loss optimization objective is constructed based on known class supervision and anomaly prior guidance. During the model training phase, supervised learning is performed using only samples with known class labels. By performing optimal label assignment and combining the basic detection loss and anomaly separation loss, the model is forced to master the semantics of known classes while establishing a high sensitivity to "abnormal background" features, thereby achieving the ability to capture unknown defects outside the distribution during the testing phase.

[0038] In one embodiment, S500 includes: S510: Perform optimal assignment based on known class labels: Obtain the set of prediction results from the S400 output. And introduce a set of real labels that only contain known defect categories. The Hungarian algorithm is used to calculate the minimum matching cost between the predicted results and known defects, and to determine the positive sample allocation relationship of known categories. S520: Calculate the discriminative supervision loss for known categories: for those successfully matched in S510 The prediction results are used to calculate the cross-entropy classification loss between the predicted probability distribution and the true class. And the location regression loss between the predicted bounding box and the true bounding box. This ensures that the model can accurately identify and locate predefined known industrial defects; S530: Calculate the anomaly separation loss based on background manifold constraints: To overcome the limitations of known class labels and endow the model with the ability to detect unknown defects, an anomaly feature separation loss is introduced. The collaborative anomaly metric score output from S300 Apply differential constraints: (19) When the image to be tested is a normal image, The value is 0 when the image under test is an abnormal image. The value is 1; under this mechanism, the network learns that any sample containing defects, other than the normal background, is forced to have a higher anomalous score. This causes the model to produce a high anomalous response in future tests, even when encountering unseen, unknown types of defects, due to their deviation from the background manifold. S540: Perform multi-task joint optimization and parameter update: Sum the above loss terms with weights to construct the total loss for end-to-end training. : (20) in, , , The hyperparameter weights are minimized using the backpropagation algorithm. The gradient descent optimizer is used to iteratively update the overall network parameters to complete the training of the industrial defect open set target detection model.

[0039] (vi) S600: Input the industrial image to be detected into the trained model, obtain the original prediction box containing the known category probability and anomaly score, remove redundant predictions through open set classification and discrimination logic and post-processing algorithm, and output the final defect category and bounding box coordinates.

[0040] In one embodiment, S600 includes: S610: During inference, the input image is fed into the trained network model, directly passing through the feature interaction and decoding networks from S100 to S400, and outputting the original detection result set. (twenty one) in, To predict the known class probability distribution of the head output, For collaborative anomaly measurement scores, The coordinates of the original defect bounding box obtained from regression; S620: Perform open set classification calibration: based on a set classification confidence threshold. With the abnormal score threshold Perform open set attribute discrimination for each prediction box: (twenty two) in, and These represent the maximum confidence level in the predicted probability distribution and its corresponding index of the known defect category, respectively. Indicates an unknown category of defect outside the distribution; This represents normal background noise; through this discrimination criterion, the model achieves accurate classification of known defects, discovery of unknown anomalies, and background noise filtering within a unified logical framework. S630: Perform redundant prediction box filtering: For the detection results after open set classification calibration, non-maximum suppression is used to calculate the spatial intersection-union ratio between each bounding box. When the intersection-union ratio between prediction boxes of the same class is greater than the set overlap threshold, the prediction box with the highest comprehensive score is retained, and the remaining redundant overlapping boxes are suppressed and removed. S640: Output the final detection result: The target bounding box coordinate set after redundancy filtering and its corresponding defect attributes are reverse-mapped to the scale of the original input image, and the final industrial defect detection report is output and saved to achieve accurate interception of unknown defects in real complex environments.

[0041] Specifically, the training method detection results are shown in the figure below. Figure 3 As shown in the figure, six subplots illustrate the model's detection results under different local viewpoints on complex industrial components. Green boxes represent known defect categories, while red boxes represent unknown defect categories. It can be observed that the proposed method can accurately detect known defects while also accurately locating unknown defects. This visualization demonstrates that the proposed framework exhibits good robustness and generalization ability in real-world complex industrial scenarios. On one hand, the model effectively suppresses interference from background factors such as complex metal textures; on the other hand, it maintains strong perception and localization accuracy even for unseen defect patterns, thus validating the effectiveness of the method in open-set object detection tasks.

[0042] The above proposes a background-defect joint modeling defect detection method for open worlds, which effectively overcomes the limitations of closed-set detection and improves the ability to discover and detect unknown out-of-distribution defects. Its advantages are as follows: 1. Constructing a dual-driven hybrid proposal box mechanism to overcome the bottleneck of closed-set detection: Addressing the problem that traditional algorithms tend to filter unknown defects as background, this invention designs a dual-branch orthogonal sampling mechanism based on known semantics and unknown anomaly scores. This mechanism independently extracts proposal boxes for potential anomalies, ensuring that various unknown defects are successfully detected.

[0043] 2. Designing a global-local collaborative anomaly metric to improve detection accuracy and robustness: This invention proposes a collaborative verification strategy that integrates global feature deviation and local reconstruction residuals. By accurately characterizing the distribution differences between defects and normal backgrounds at multiple scales, it effectively overcomes the problem of susceptibility to noise interference at a single scale, significantly enhancing the ability to accurately detect small, unknown defects.

[0044] 3. Introducing multi-task joint separation constraints to achieve efficient end-to-end learning: Within a unified framework, this invention can force the network to establish an opposition relationship between clean background and abnormal defects by using only known defect labels through anomaly separation loss.

[0045] In one embodiment, a background-defect co-modeling defect detection system for open worlds is also provided, for performing a background-defect co-modeling defect detection method for open worlds, including: The data preprocessing module is used to acquire an industrial defect dataset containing images of the images to be detected and images of normal samples without defects, and to preprocess and construct a standardized feature tensor. The feature extraction and interaction module is used to extract the features to be detected and the normal background reference features using the feature extraction network. Based on the normal background reference features, a robust manifold representing the defect-free state of the background features is constructed. The feature to be detected and the robust manifold of the background features are aligned and fused through the defect-background interaction learning network to sharpen the discrimination boundary between the background and the defect. The collaborative anomaly measurement module is used to execute a global-local collaborative anomaly measurement strategy based on the feature interaction reconstruction results, and generate a collaborative anomaly measurement score to measure the degree of anomalies in the overall image and local regions. The decoding and prediction module is used to design a hybrid proposal box generation mechanism based on semantics and anomalies. It calculates the target score of the known category and the score of the unknown anomaly, generates a joint proposal box through a two-branch hybrid sorting, and feeds the joint proposal box as a position prior into the decoding network to output the detection result. The training optimization module is used to construct a multi-task joint loss optimization objective based on known class supervision and abnormal prior guidance. During the training phase, it uses samples with known class labels for supervised learning, forcing the model to establish a high sensitivity to abnormal background features, so as to achieve the ability to capture unknown defects outside the distribution during the testing phase. The inference post-processing module is used to input the industrial image to be detected into the trained model, obtain the original prediction box containing the known category probability and anomaly score, remove redundant predictions through open set classification and discrimination logic and post-processing algorithm, and output the final defect category and bounding box coordinates.

[0046] Specific limitations regarding the open-world background-defect co-modeling defect detection system can be found in the limitations of the open-world background-defect co-modeling defect detection method described above, and will not be repeated here. Each module in the aforementioned open-world background-defect co-modeling defect detection system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0047] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a background-defect co-modeling defect detection method for an open world.

[0048] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of a background-defect co-modeling defect detection method for an open world.

[0049] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0050] The foregoing has provided a detailed description of the background-defect co-modeling defect detection method and system for open world environments provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention, and the descriptions of the embodiments above are merely for the purpose of helping to understand the core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A defect detection method for background-defect joint modeling in an open world, characterized in that, Includes the following steps: S100: Obtain an industrial defect dataset containing the image to be detected and defect-free normal sample images, and preprocess it to construct a standardized feature tensor; S200: The feature extraction network extracts the features to be detected and the normal background reference features respectively. Based on the normal background reference features, a robust manifold representing the defect-free state of the background features is constructed. The feature to be detected and the robust manifold of the background features are aligned and fused through the defect-background interaction learning network to sharpen the discrimination boundary between the background and the defect. S300: Based on the feature interaction reconstruction results, a global-local collaborative anomaly measurement strategy is executed to generate a collaborative anomaly measurement score for measuring the degree of anomalies in the overall image and local regions. S400: Design a hybrid proposal box generation mechanism based on semantics and anomalies. It calculates the target score of the known category and the score of the unknown anomaly respectively, generates a joint proposal box through a two-branch hybrid sorting, and feeds the joint proposal box as a position prior into the decoding network to output the detection result. S500: Construct a multi-task joint loss optimization objective based on known class supervision and abnormal prior guidance. During the training phase, supervised learning is performed using samples with known class labels, forcing the model to establish a high sensitivity to abnormal background features, so as to achieve the ability to capture unknown defects outside the distribution during the testing phase. S600: Input the industrial image to be detected into the trained model, obtain the original prediction box containing the known category probability and anomaly score, remove redundant predictions through open set classification and discrimination logic and post-processing algorithm, and output the final defect category and bounding box coordinates.

2. The method according to claim 1, characterized in that, S100 includes: S110: Acquire real-world images of defective industrial components. and its corresponding defect categories and true bounding box annotation information. To construct the sample set to be tested ,in This indicates the total number of samples to be tested; S120: Obtain images of defect-free, normal components. This is used as a benchmark reference set to provide prior information on the distribution of normal background features. ,in This represents the total number of normal samples, which is used to build the normal feature library later. S130: Sample set to be tested With reference set The images in the dataset undergo uniform spatial scale alignment and normalization, outputting a normalized tensor of the features to be detected that satisfies the input specifications of the detection network. （1） （2） in, The image to be detected after preprocessing. This is the normal image after preprocessing. This represents a spatial geometric transformation function that includes multi-scale random scaling and horizontal flipping. This represents the image tensor quantization and pixel normalization processing functions.

3. The method according to claim 2, characterized in that, S200 includes: S210: The preprocessed image to be detected... Compared with normal images A convolutional backbone network with shared input parameters extracts features, and channel dimensionality reduction is performed through a projection layer to obtain multi-scale features for testing. Compared with normal reference features : （3） （4） in, This indicates a joint feature extraction operation involving convolutional layers and channel-wise dimension reduction projection; S220: In order to establish an absolute metric for open set detection within the feature space, multiple batches of extracted normal reference features will be used. Flattening the matrix in the spatial dimension and stitching it across batches, a robust manifold matrix representing the distribution of pure background features is constructed. : （5） in, This represents a flattening operation that converts the spatial dimension of the feature map to the sequence dimension. This indicates an operation that stitches together the flattened features of multiple normal samples along the sample dimensions; S230: Introduces a defect-background feature interactive learning mechanism to learn the features to be tested. Mapped to query vector The robust manifold matrix of background features Mapped to a key-value pair matrix and The cross-attention mechanism is used to extract the most relevant normal visual semantics from the background manifold to the features to be detected, and then feature alignment and fusion are performed. （6） in, This indicates the cross-attention fusion operation. This is the feature representation after interactive reconstruction.

4. The method according to claim 3, characterized in that, The S300 includes: S310: Perform global anomaly measurement: for features to be detected Global average pooling is performed to obtain the global query vector, and the robust manifold matrix of background features is calculated. The mean feature vector along the sample dimension is used to obtain the global feature residual, which is then input into a scoring network to generate a global anomaly score that measures the degree of global anomaly. : （7） （8） （9） in, This indicates a global average pooling operation. This indicates the calculation of the mean. This represents a multilayer perceptron network used to calculate feature deviation. S320: Perform local anomaly measurement: Calculate the feature to be detected at each spatial location of the feature map. Interactive reconstruction features output from S200 Pixel-by-pixel comparison is performed, the maximum similarity at each location is taken, and its residual is calculated to generate a pixel-level anomaly feature map reflecting the distribution of local anomalies. : （10） in, This function represents the cosine similarity between query features and manifold features. S330: Constructing a collaborative verification and fusion mechanism: Extracting local anomaly feature maps The global extreme response component and its correlation with the global anomaly score. Weighted fusion is performed to generate the final collaborative anomaly measurement score. : （11） in, This indicates the extraction of the maximum value from the local anomaly feature map. and These are the preset fusion weight coefficients.

5. The method according to claim 4, characterized in that, The S400 includes: S410: Obtain the interactive reconstruction features output by S200 The classification prediction head is used to calculate the target score of each spatial pixel belonging to a known defect category. Simultaneously, the robust manifold matrix of background features constructed in S200 is invoked. , computational interaction reconstructed features With manifold matrix The similarity residuals in the feature space are used to generate pixel-level anomaly scores that reflect the degree of out-of-distribution deviation. : （12） （13） in, It is the Sigmoid activation function. For a linear classification mapping network with known category objectives, This is an operation for calculating similarity residuals; S420: Establish a dual-branch hybrid proposal box allocation mechanism: Set the total number of proposal boxes required by the decoder to... The capacity space is orthogonally divided into proposal boxes driven by known semantics. With candidate proposal boxes driven by unknown anomalies Based on target scores respectively Pixel-level anomaly scores Perform a descending sort, obtain the corresponding highest-scoring spatial location index, and concatenate them into a composite index set. : （14） （15） （16） in, This represents an index retrieval operation that extracts the spatial locations of the K largest values ​​in a sequence. This represents the concatenation operation of index tensors; S430: Extracting High-Dimensional Spatial Features and Location Priors: Utilizing Joint Index Sets From the dense set of candidate boxes generated by the feature encoder, the coordinates of the corresponding initial proposal boxes are precisely converged. Then, the initial proposal boxes are fed into the decoder as positional priors for subsequent cross-attention calculations and fine regression of defect bounding boxes. （17） in, The dense candidate box coordinate tensor output by the encoder. This indicates an operation to extract the corresponding coordinate position based on the index. These are the coordinates of the joint proposal frame that are ultimately sent to the decoder; S440: Perform decoding transformation and final detection prediction: Calculate the coordinates of the selected joint proposal boxes. As an initial position prior, combined with encoded features The data is fed into a decoding network consisting of multiple layers of cross-attention and self-attention for feature updating and decoding. Subsequently, a multi-task joint prediction network performs class probability mapping and fine-grained bounding box regression in a unified manner, directly outputting the final set of prediction results. : （18） in, This represents a multi-task joint prediction mapping network that includes a feedforward neural network. and The first The probability distribution of the categories and location coordinates of each predicted target.

6. The method according to claim 5, characterized in that, The S500 includes: S510: Perform optimal assignment based on known class labels: Obtain the set of prediction results from the S400 output. And introduce a set of real labels that only contain known defect categories. The Hungarian algorithm is used to calculate the minimum matching cost between the predicted results and known defects, and to determine the positive sample allocation relationship of known categories. S520: Calculate the discriminative supervision loss for known categories: for those successfully matched in S510 The prediction results are used to calculate the cross-entropy classification loss between the predicted probability distribution and the true class. And the location regression loss between the predicted bounding box and the true bounding box. ; S530: Calculate the anomaly separation loss based on background manifold constraints: To overcome the limitations of known class labels and endow the model with the ability to detect unknown defects, an anomaly feature separation loss is introduced. The collaborative anomaly metric score output from S300 Apply differential constraints: （19） When the image to be tested is a normal image, The value is 0 when the image under test is an abnormal image. =1; S540: Perform multi-task joint optimization and parameter update: Sum the above loss terms with weights to construct the total loss for end-to-end training. : （20） in, , , The hyperparameter weights are minimized using the backpropagation algorithm. The gradient descent optimizer is used to iteratively update the overall network parameters to complete the training of the industrial defect open set target detection model.

7. The method according to claim 6, characterized in that, The S600 includes: S610: During inference, the input image is fed into the trained network model, directly passing through the feature interaction and decoding networks from S100 to S400, and outputting the original detection result set. （21） in, To predict the known class probability distribution of the head output, For collaborative anomaly measurement scores, The coordinates of the original defect bounding box obtained from regression; S620: Perform open set classification calibration: based on a set classification confidence threshold. With the abnormal score threshold Perform open set attribute discrimination for each prediction box: （22） in, and These represent the maximum confidence level in the predicted probability distribution and its corresponding index of the known defect category, respectively. Indicates an unknown category of defect outside the distribution; This indicates normal background noise; S630: Perform redundant prediction box filtering: For the detection results after open set classification calibration, non-maximum suppression is used to calculate the spatial intersection-union ratio between each bounding box. When the intersection-union ratio between prediction boxes of the same class is greater than the set overlap threshold, the prediction box with the highest comprehensive score is retained, and the remaining redundant overlapping boxes are suppressed and removed. S640: Output the final detection result: The target bounding box coordinate set after redundancy filtering and its corresponding defect attributes are reverse-mapped to the scale of the original input image, and the final industrial defect detection report is output and saved to achieve accurate interception of unknown defects in real complex environments.

8. A background-defect co-modeling defect detection system for open worlds, used to perform the method according to any one of claims 1 to 7, characterized in that, include: The data preprocessing module is used to acquire an industrial defect dataset containing images of the images to be detected and images of normal samples without defects, and to preprocess and construct a standardized feature tensor. The feature extraction and interaction module is used to extract the features to be detected and the normal background reference features using the feature extraction network. Based on the normal background reference features, a robust manifold representing the defect-free state of the background features is constructed. The feature to be detected and the robust manifold of the background features are aligned and fused through the defect-background interaction learning network to sharpen the discrimination boundary between the background and the defect. The collaborative anomaly measurement module is used to execute a global-local collaborative anomaly measurement strategy based on the feature interaction reconstruction results, and generate a collaborative anomaly measurement score to measure the degree of anomalies in the overall image and local regions. The decoding and prediction module is used to design a hybrid proposal box generation mechanism based on semantics and anomalies. It calculates the target score of the known category and the score of the unknown anomaly, generates a joint proposal box through a two-branch hybrid sorting, and feeds the joint proposal box as a position prior into the decoding network to output the detection result. The training optimization module is used to construct a multi-task joint loss optimization objective based on known class supervision and abnormal prior guidance. During the training phase, it uses samples with known class labels for supervised learning, forcing the model to establish a high sensitivity to abnormal background features, so as to achieve the ability to capture unknown defects outside the distribution during the testing phase. The inference post-processing module is used to input the industrial image to be detected into the trained model, obtain the original prediction box containing the known category probability and anomaly score, remove redundant predictions through open set classification and discrimination logic and post-processing algorithm, and output the final defect category and bounding box coordinates.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images