Open end-to-end semi-supervised inspection method and system for adaptive unknown defect identification

Abstract

The application discloses an open end-to-end semi-supervised detection method and system for adaptive unknown defect identification, relates to the technical field of industrial defect detection, and comprises three parts of an image acquisition module, a defect detection module and a data storage module. The defect detection module adopts an open semi-supervised defect detection model based on YOLOv11, completes defect detection under the condition of a small amount of labeled data, and identifies newly-appeared unknown defects through an open set identification mechanism, so that the detection problems caused by insufficient labeled data and continuous evolution of defect types in an industrial scene are solved at the same time. The application faces the application pain points of difficult acquisition of labeled samples in an industrial field, high labeling cost and continuous evolution of defect types, and through semi-supervised learning combined with an open set identification mechanism, the identification and positioning of newly-appeared unknown defects are realized while the dependence on artificial labeled data is reduced, performance reduction of a model when defect distribution changes is avoided, and therefore the detection precision and detection efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of industrial defect detection technology, specifically to an open end-to-end semi-supervised detection method and system for adaptive identification of unknown defects. Background Technology

[0002] Defect detection technology in industrial manufacturing is a core element in ensuring product pass rates, and its technological development has evolved from manual inspection to automated inspection. Traditional manual visual inspection methods rely on the operator's experience and judgment, resulting in inherent drawbacks such as strong subjectivity and inconsistent inspection standards. With the development of computer vision technology, image processing-based automated inspection systems have gradually become the mainstream solution. These systems acquire product surface images using high-precision industrial cameras and utilize digital image processing algorithms to identify and classify defects. A typical industrial defect detection process includes four key steps: image acquisition, preprocessing, feature extraction, and defect judgment. Among these, the feature extraction stage directly determines the system's detection accuracy and robustness.

[0003] Current mainstream deep learning-based defect detection methods primarily employ fully supervised learning frameworks, requiring a large amount of labeled data to train convolutional neural network models. While these methods can achieve high detection accuracy in specific scenarios, they suffer from significant technical bottlenecks: First, the model's capabilities are severely limited by the quantity and quality of labeled data, demanding high levels of expertise from process engineers in labeling. Furthermore, existing supervised learning frameworks are essentially closed-set recognition systems, where output layer neurons are fixed to known defect categories. When input samples contain defect types not covered by the training set, the system will force them into incorrect known categories. This characteristic can accumulate systematic false detection risks in dynamically changing industrial environments. Although recent research has attempted to introduce semi-supervised learning to reduce label dependence, these improvements still haven't overcome the inherent limitations of closed-set recognition and cannot meet the actual needs of open-set recognition in industrial scenarios. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes an open end-to-end semi-supervised detection method and system for adaptive unknown defect identification. It aims to overcome the limitations of traditional supervised and semi-supervised learning frameworks in identifying closed sets and effectively identifying unknown defects. This invention achieves accurate classification and localization of known defects as well as reliable identification of unknown defects.

[0005] Therefore, the present invention provides the following technical solution: On the one hand, this invention provides an open end-to-end semi-supervised detection method for adaptive unknown defect identification, comprising: An open semi-supervised defect detection model is established; the open semi-supervised defect detection model includes a teacher model and a student model based on the YOLOv11 network; the teacher model includes a teacher open set recognizer that identifies defects based on unlabeled defect image data; the student model includes a student open set recognizer that identifies defects based on unlabeled defect image data and labeled defect image data; the teacher open set recognizer and the student open set recognizer have the same structure; The teacher open set identifier includes: a positioning reliability screening module and an open set classification discrimination module; the positioning reliability screening module evaluates the spatial consistency and geometric reliability of the detection box based on the positioning reliability threshold, and filters out unstable or low-quality candidate boxes; the open set classification discrimination module includes a closed set classifier and a binary verification mechanism; the binary verification mechanism is used to independently determine whether a sample truly belongs to a given known category, or should be rejected as an unknown category; Based on the aforementioned open semi-supervised defect detection model, defect detection is performed on the real-time acquired defect image data to obtain defect detection results.

[0006] Furthermore, the teacher model also includes a teacher feature extraction module that extracts features based on unlabeled defect image data, and the student model also includes a student feature extraction module that extracts features based on unlabeled defect image data and labeled defect image data. The teacher feature extraction module has the same structure as the student feature extraction module. The teacher feature extraction module uses the backbone and neck parts of YOLOv11 to perform multi-level feature extraction, and combines spatial coordinate attention mechanism and multi-scale feature fusion to send the obtained features to the teacher open set recognizer.

[0007] Furthermore, in the positioning reliability screening module, the positioning reliability is: ;in, t c This represents the raw confidence score of the student model output. Used to map this value to The probability value within the interval, Represents the perfect intersection-union ratio between two predicted bounding boxes, where and Bounding box prediction results for the teacher model and the student model, respectively; For the For a sample, when its location reliability At that time, retain its location reliability and record it as . ;when When, set it to zero, that is .

[0008] Furthermore, in the open set classification and discrimination module, the logit value output by the given detection head is... The closed-set classifier uses the Softmax function to determine which known categories a sample belongs to. The probability of: ; in, For classification confidence; The categorical binary validation output is constructed using the Sigmoid activation function: ; in, This represents the Sigmoid activation function; positive probability. Representing samples and categories The degree of matching, negative probability Used to quantify the degree to which a sample does not belong to this category; The output of the closed-set classifier is fused with the output of the binary verification, thus expanding the classification range. Class expansion to Class, in which Represents all possible categories; Open set classification confidence distribution for: ; Among them, the The class is an unknown defect category, which is obtained by accumulating the rejection probabilities of each known category.

[0009] Furthermore, the teacher model also includes: a pseudo-label allocation module for generating pseudo-labels; the pseudo-label allocation module generates pseudo-labels based on unlabeled data using a dual-threshold category adaptive label allocation mechanism; the dual-threshold category adaptive label allocation mechanism optimizes the utilization rate of pseudo-labels by dynamically adjusting the thresholds and adaptively updates pseudo-labels for different categories.

[0010] Furthermore, in the dual-threshold category adaptive label allocation mechanism, pseudo-labels are classified through three confidence levels: high-confidence pseudo-labels, low-confidence pseudo-labels, and unreliable pseudo-labels, with two time-varying thresholds set for each category. and And satisfy ; In each training iteration At that time, based on the teacher model's confidence prediction on open sets of unlabeled mini-batch data, the global smoothing confidence threshold was calculated using exponential moving average. : ; in, For EMA attenuation factor, This indicates the number of unlabeled samples in this small batch. This indicates that the teacher network is iterating. Time for sample The given confidence level for the open set; For each category The average confidence level is updated using an exponential moving average:

[0011] in, Indicates sample Classified as The probability of classifying an open set; Global threshold based on category confidence level Scaling is performed to obtain the category. Dynamic upper threshold : ; category lower threshold Defined as a fixed proportion of the upper threshold: ;in As a proportionality coefficient, in iteration Time as category Assigned category-aware weights Defined as:

[0012] in, This indicates a weak level of supervision.

[0013] Furthermore, the student model also includes: a dynamic weight optimization module for updating weights; the dynamic weight optimization module updates the weights of the student model through a loss function; the student model updates the parameters of the teacher model through an exponentially weighted average. The loss function is: ;in, The supervised loss is used to measure the difference between the adaptive model's predictions and the ground truth annotations in classification and localization. ;in This represents the predicted value and the actual predicted value of the student model; For classifying losses, To regress the loss, For distribution focus loss; For known pseudo-label loss of unlabeled samples, ; For the unknown pseudo-label loss of unlabeled samples, ; The classification items employ a strict hard-label strategy, participating in training only when the confidence level of the open set reaches the upper threshold. The known classes are... Unknown class : ; ; in, CE Represents cross-entropy loss; The regression term uses either a high-confidence classification mechanism or a very high-confidence localization mechanism: ; ; in, Intersection, union, and comparison; The focus term of the distribution employs a dual-threshold hierarchical supervision approach: strong supervision for high confidence levels and weak supervision for medium confidence levels. ; ; in, It is a computational method that transforms bounding box regression in object detection from predicting a single, definite value into predicting a continuous probability distribution.

[0014] Furthermore, after obtaining the weights of the student model, the process also includes: calculating the gradient of the weights of the current student model, and applying a perturbation to the weights according to the gradient direction; The disturbance calculation formula is: ;in, For student network weight, For the disturbance amplitude, The gradient of the loss function with the current weights.

[0015] Furthermore, it also includes: acquiring and storing defect image data and defect detection results.

[0016] In another aspect, the present invention also provides an open end-to-end semi-supervised detection system for adaptive unknown defect identification, comprising: an image acquisition module, a defect detection module, and a data storage module; The image acquisition module acquires defect image data in real time through a camera device and transmits the acquired data to the server in real time. The defect detection module includes an open semi-supervised defect detection model; the open semi-supervised defect detection model includes a teacher model and a student model based on the YOLOv11 network; the teacher model includes: a teacher feature extraction module for extracting features from unlabeled defect image data, a teacher open set recognizer for identifying defects in the extracted features, and a pseudo-label assignment module for generating pseudo-labels; the student model includes: a student feature extraction module for extracting features from both unlabeled and labeled defect image data, a student open set recognizer for identifying defects in the extracted features, and a dynamic weight optimization module for updating weights; the teacher feature extraction module and the student feature extraction module have the same structure; the teacher open set recognizer and the student open set recognizer have the same structure; In the teacher feature extraction module, the backbone and neck parts of the YOLOv11 network are used for multi-level feature extraction, and spatial coordinate attention mechanism and multi-scale feature fusion are combined. The teacher open set identifier includes: a positioning reliability screening module and an open set classification discrimination module; the positioning reliability screening module evaluates the spatial consistency and geometric reliability of the detection box based on the positioning reliability threshold, and filters out unstable or low-quality candidate boxes; the open set classification discrimination module includes a closed set classifier and a binary verification mechanism; the binary verification mechanism is used to independently determine whether a sample truly belongs to a given known category, or should be rejected as an unknown category; The pseudo-label allocation module generates pseudo-labels based on unlabeled data using a dual-threshold category adaptive label allocation mechanism. The dynamic weight optimization module updates the weights of the student model through a loss function; the student model updates the parameters of the teacher model through an exponentially weighted average. The data storage module is responsible for storing defect detection data obtained from the defect detection module and raw data obtained from the image acquisition module, and periodically updating model parameters with new defect data to optimize detection performance.

[0017] The above technical solution has the following beneficial effects: (1) This invention proposes an open set detector that can distinguish between labeled defects and unknown defect categories. When the model encounters new and unseen defect types, it can identify them in a timely and accurate manner, thus ensuring that it can maintain efficient detection capabilities when facing new situations in practical applications. This design significantly improves the adaptability and generalization ability of the model, especially in industrial environments where defect types are constantly changing.

[0018] (2) This invention proposes a dual-threshold category adaptive pseudo-label allocation mechanism. By dynamically adjusting the threshold during training and adaptively updating for different categories, it realizes the reasonable use of pseudo-labels with different reliability, thereby maximizing the learning effect of unlabeled data and improving the detection performance and generalization ability of the model.

[0019] (3) This invention introduces a dynamic weight optimization mechanism. By applying small adversarial perturbations to the student model during training, the robustness of the model under inputs of unlabeled data, noise interference, and unknown defects is improved, thereby enhancing its adaptability to complex industrial scenarios. At the same time, the parameters of the student model updated through adversarial training are fed back to the teacher model through consistency constraints, and the teacher model is updated synchronously using an exponential moving average method. The adversarial perturbations of this invention are more effective in mining the supervision information of unlabeled data. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an overall block diagram of the real-time industrial defect detection system in this embodiment of the invention; Figure 2 This is an open semi-supervised defect detection model in this embodiment of the invention. Detailed Implementation

[0022] This invention provides an open, end-to-end semi-supervised detection method and system for adaptive unknown defect identification, aiming to overcome the dependence on a large number of labeled samples in traditional industrial defect detection methods, as well as the challenges of identifying unknown defects. This method can progressively improve model accuracy during training, thereby effectively addressing the challenges of diverse and unknown defects in industry.

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] like Figure 1 As shown, this invention proposes an open end-to-end semi-supervised detection method for adaptive unknown defect identification, which specifically includes the following steps: S1. Collect defect image data in real time and transmit the collected data to the server in real time.

[0026] In practice, camera devices are installed above and below the production line to collect defect image data in real time. This data is then transmitted to a server for storage and processing via a high-speed data transmission interface. The camera devices utilize high-resolution industrial cameras, capable of accurately capturing minute surface defects and ensuring high-quality image data under various environmental conditions. All collected data is transmitted to the server immediately. Subsequently, this data is divided: a small subset of images from a specific production line is classified as a labeled dataset (containing several defect categories, with roughly balanced numbers for each category), while the remaining images and all images from other production lines are classified as an unlabeled dataset (covering multiple defect categories, including unknown types of defects). For the labeled dataset, defects in the images are manually labeled, generating corresponding .txt files containing the coordinates of the manually labeled defect bounding boxes and the defect category label for each box. The unlabeled dataset is left unprocessed.

[0027] S2. Establish an open semi-supervised defect detection model; like Figure 2As shown, the open semi-supervised defect detection model includes a teacher model and a student model based on the YOLOv11 network. The teacher model includes: a teacher feature extraction module for extracting features from unlabeled data, a teacher open set recognizer for identifying defects in the extracted features, and a pseudo-label assignment module for generating pseudo-labels. The student model includes: a student feature extraction module for extracting features from both unlabeled and labeled data, a student open set recognizer for identifying defects in the extracted features, and a dynamic weight optimization module for updating weights. The teacher feature extraction module and the student feature extraction module have the same structure; the teacher open set recognizer and the student open set recognizer have the same structure. In the feature extraction module, in order to enhance the model's ability to perceive defects at different scales, YOLOv11's backbone and neck parts are used for multi-level feature extraction. The spatial coordinate attention mechanism and multi-scale feature fusion are combined to improve the model's feature representation ability in complex backgrounds.

[0028] In the backbone, convolutional operations are used to extract basic features such as edges, textures, colors, and shapes from the image, and downsampling is used to gradually aggregate global information, enabling the model to better understand and distinguish differences in the image. As the network layers deepen, the model gradually extracts richer feature information, providing a foundation for subsequent object detection tasks.

[0029] In the neck region, by fusing low-level detail features and high-level semantic features, the model can not only accurately locate large-scale defects but also effectively capture the details of small-scale defects. This approach enables the model to identify defects of different scales, thereby improving its ability to detect complex targets.

[0030] To overcome the problems of traditional semi-supervised defect detection methods that rely solely on classification confidence, are difficult to reliably identify new types of defects, and are prone to misclassifying unknown defects as known categories, this invention proposes an open set recognizer.

[0031] The open set identifier consists of two parts: a positioning reliability screening module and an open set classification and discrimination module. The positioning reliability screening module introduces a location reliability threshold to evaluate the spatial consistency and geometric reliability of the detection boxes, filtering out unstable or low-quality candidate boxes and avoiding false label noise caused by relying solely on classification scores. In the open set classification and discrimination module, an intra-class positive and negative verification mechanism is added to the traditional known class probability output, enabling fine-grained confirmation of known classes and explicit rejection of unknown defects, thus expanding the decision space from C classes to C+1 classes (unknown classes). Through the dual constraints of "positioning threshold + innovative classifier," the reliability of unknown defect identification and the stability of detection and positioning can be significantly improved under conditions where defect distribution in industrial scenarios evolves over time, unknown defects continuously appear, and annotation is delayed.

[0032] Specifically, in the location reliability screening module, the location reliability is: ; in, t c This represents the raw confidence score of the student model output. Used to map this value to The probability value within the interval, The larger the value, the higher the confidence level that the predicted bounding box contains the defective target. This represents the Complete Intersection over Union (CIoU) metric between two predicted bounding boxes, where and The bounding box prediction results of the teacher detector and the student detector are shown separately. A higher CIoU value indicates stronger consistency between the two detectors in their localization results, thus suggesting a more reliable localization estimate of the target. During training, the localization reliability score is also shown. This is used to evaluate the reliability of each pseudo-label generated by the adaptive detector. Since the adaptive detector and the main detector may output different bounding boxes for the same target, the localization difference between them can be significant, especially if one detector only focuses on local information of the target or fails to capture key details. Therefore, a localization reliability threshold is introduced. This is used to filter out samples with low location reliability. Specifically, for the first... For a sample, when its location reliability At that time, retain its location reliability and record it as . ;when When, set it to zero, that is .

[0033] The open set classification module aims to improve the model's ability to distinguish between known and unknown defects by combining traditional closed set classification with a binary verification mechanism. This is based on the logit value output by the detection head. The standard closed-set classifier uses the Softmax function to determine which known categories a sample belongs to. The probability of: ; in, To assess classification confidence, the Softmax function has a significant limitation: for samples far from the training distribution, it tends to produce overconfident predictions, leading to unreliable classification confidence, especially in scenarios with novel, defective inputs. To mitigate this problem, a binary verification mechanism is introduced to independently determine whether a sample truly belongs to a given, known category. This should be rejected as an unknown category. First, a sigmoid activation function is used to construct the category-level binary validation output: ; in, This represents the Sigmoid activation function. Positive probability. Representing samples and categories The degree of matching, negative probability This is used to quantify the degree to which a sample does not belong to the category. This category-level decoupling mechanism can more finely distinguish between in-distribution and out-of-distribution samples, thereby enhancing the model's robustness to misclassification.

[0034] By fusing the Softmax classification signal with the binary verification output, the classification range is expanded from... Class expansion to Class, in which This represents all possible categories. The final open set classification confidence distribution is calculated using the following joint "acceptance-rejection" mechanism. : ; Among them, the The category is an unknown defect category, which is obtained by accumulating the rejection probabilities of each known category, thereby achieving explicit identification of unknown defects.

[0035] In the pseudo-label allocation module, a dual-threshold category adaptive label allocation mechanism is used for label allocation. This mechanism optimizes the utilization rate of pseudo-labels by dynamically adjusting the thresholds and adaptively updates pseudo-labels for different categories, thereby ensuring the full application of pseudo-labels of each category in training and enhancing the diversity and quality of training data.

[0036] Specifically, pseudo-labels are classified using three levels of confidence: high-confidence pseudo-labels, low-confidence pseudo-labels, and unreliable pseudo-labels. This mechanism sets two time-varying thresholds for each category. and And satisfy This allows for the stratification and confidence-aware utilization of unlabeled samples based on their open set classification probabilities.

[0037] In each training iteration First, based on the teacher model's confidence prediction on open sets of unlabeled mini-batch data, the global smoothing confidence threshold is calculated using exponential moving average (EMA). : ; in, For EMA attenuation factor, This indicates the number of unlabeled samples in this small batch. This indicates that the teacher model is iterating. Time for sample The open set confidence level is given. Using EMA can maintain the continuity of threshold changes over time and stabilize the threshold evolution process.

[0038] To achieve category adaptability, for each category The average confidence level is also updated using the EMA:

[0039] in, Indicates sample Classified as The open set classification probability is used to characterize the dynamic changes of the confidence features of each class during training. Furthermore, the global threshold is adjusted based on the relative difficulty (or confidence level) of each class. Scaling is performed to obtain the category. Dynamic upper threshold : ; This strategy assigns lower thresholds to categories with lower confidence distributions, thereby improving the fairness of screening and training stability among different defect categories. lower threshold Defined as a fixed proportion of the upper threshold: ;in As a proportionality coefficient, in iteration Time as category Assigned category-aware weights Defined as:

[0040] in, This indicates weak supervision strength, used to assign lower weights to samples with medium confidence levels, in order to achieve stratified utilization of unlabeled samples.

[0041] In the dynamic weight optimization module, the student's model weights are obtained by updating the loss function, which consists of three parts: supervised loss. Known pseudo-label loss With unknown pseudo-label loss .

[0042] Monitoring losses Used to measure the difference between adaptive model predictions and ground truth annotations in classification and localization, derived from classification. cls ,return box and distribution focus dfl It consists of three parts: ; in This represents the student model's predicted value and the actual predicted value. For unlabeled samples, a known pseudo-label loss is defined. With unknown pseudo-label loss Both have the same structure, consisting of a categorical term, a regression term, and a distribution focus term: ; The classification items employ a strict hard-label strategy, participating in training only when the confidence level of the open set reaches an upper threshold (the known class is...). Unknown class ): ; ; in, CE This represents the cross-entropy loss. The regression term employs a "high confidence in classification or extremely high confidence in localization" trigger mechanism to preserve reliable localization supervision. ; ; The focus items of the distribution are subject to hierarchical supervision with a dual threshold: strong supervision for high confidence and weak supervision for medium confidence (unknown class isomorphism is handled). ; ; The final total loss is obtained by weighting the supervision loss and the two types of pseudo-label losses: .

[0043] Next, adversarial perturbation data augmentation technology was introduced into the student model weights, replacing the strong and weak augmentation methods commonly used in traditional semi-supervised models; The disturbance calculation formula is as follows: ;in, For student network weight, For the disturbance amplitude, The gradient of the loss function with the current weights.

[0044] By calculating the gradient of the current student feature extraction network weights and applying perturbations to the weights according to the direction of this gradient, the aim is to enhance the model's adaptability to noise and unlabeled data, allowing the model to continuously "challenge" itself during training. This method effectively avoids the student network overfitting to labeled data and improves its generalization ability to unknown defects. The student network gradually updates the parameters of the teacher network through EMA, enabling the teacher model to continuously improve its detection performance.

[0045] S3. Store the acquired defect detection results data (including detection box position, category label, confidence level, etc.) and the original image data, and periodically update the model parameters with new defect data (all unlabeled) to optimize detection performance.

[0046] The methods described in the above embodiments have the following technical advantages: (1) An open set detector is proposed, which can distinguish between labeled defects and unknown defect categories. When the model encounters new and unseen defect types, it can identify them in a timely and accurate manner, thus ensuring that it can maintain efficient detection capabilities when facing new situations in practical applications. This design significantly improves the adaptability and generalization ability of the model, especially in industrial environments where defect types are constantly changing.

[0047] (2) A dual-threshold class adaptive pseudo-label allocation mechanism is proposed. By dynamically adjusting the threshold during training and adaptively updating for different categories, the reasonable use of pseudo-labels with different reliability is realized, thereby maximizing the learning effect of unlabeled data and improving the detection performance and generalization ability of the model.

[0048] (3) A dynamic weight optimization module was introduced. This technique enhances the model's robustness in handling unknown data and noise by introducing small perturbations into the training data, thereby improving the model's adaptability to complex and changing industrial scenarios. Compared with traditional augmentation methods, adversarial perturbation can more effectively help the model learn from unlabeled data, further improving the accuracy of defect detection.

[0049] Corresponding to the detection methods in the above embodiments, the present invention also provides an open end-to-end semi-supervised detection system for adaptive unknown defect identification, including: an image acquisition module, an open defect detection module, and a data storage module; The image acquisition module acquires defect image data in real time through a camera device and transmits the acquired data to the server in real time. The defect detection module includes an open semi-supervised defect detection model; this model comprises a teacher model and a student model based on the YOLOv11 network; the teacher model includes: a teacher feature extraction module for extracting features from unlabeled data, a teacher open set recognizer for identifying defects in the extracted features, and a pseudo-label assignment module for generating pseudo-labels; the student model includes: a student feature extraction module for extracting features from both unlabeled and labeled data, a student open set recognizer for identifying defects in the extracted features, and a dynamic weight optimization module for updating weights; the teacher feature extraction module and the student feature extraction module have the same structure; the teacher open set recognizer and the student open set recognizer have the same structure. In the teacher feature extraction module, the backbone and neck parts of the YOLOv11 network are used for multi-level feature extraction, and spatial coordinate attention mechanism and multi-scale feature fusion are combined. The teacher open set recognizer includes: a location reliability screening module and an open set classification and discrimination module; the location reliability screening module evaluates the spatial consistency and geometric reliability of the detection box based on the location reliability threshold, and filters out unstable or low-quality candidate boxes; the open set classification and discrimination module includes a closed set classifier and a binary verification mechanism; the binary verification mechanism is used to independently determine whether a sample truly belongs to a given known category, or should be rejected as an unknown category; The pseudo-label allocation module generates pseudo-labels based on unlabeled data using a dual-threshold category adaptive label allocation mechanism. The dynamic weight optimization module updates the weights of the student model through the loss function; the student model updates the parameters of the teacher model through an exponentially weighted average. The data storage module is responsible for storing defect detection data obtained from the defect detection module and raw data obtained from the image acquisition module, and periodically updating model parameters with new defect data to optimize detection performance.

[0050] The detection system of this invention corresponds to the detection method in the above embodiments, so the description is relatively simple. For related similarities, please refer to the description of the detection method in the above embodiments, which will not be described in detail here.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An open end-to-end semi-supervised detection method for adaptive unknown defect identification, characterized in that, include: Establish an open semi-supervised defect detection model; The open semi-supervised defect detection model includes a teacher model and a student model based on the YOLOv11 network; The teacher model includes: a teacher open set recognizer that identifies defects based on unlabeled defect image data; the student model includes: a student open set recognizer that identifies defects based on both unlabeled and labeled defect image data; the teacher open set recognizer and the student open set recognizer have the same structure. The teacher open set identifier includes: a positioning reliability screening module and an open set classification discrimination module; the positioning reliability screening module evaluates the spatial consistency and geometric reliability of the detection box based on the positioning reliability threshold, and filters out unstable or low-quality candidate boxes; the open set classification discrimination module includes a closed set classifier and a binary verification mechanism; the binary verification mechanism is used to independently determine whether a sample truly belongs to a given known category, or should be rejected as an unknown category; Based on the aforementioned open semi-supervised defect detection model, defect detection is performed on the real-time acquired defect image data to obtain defect detection results.

2. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 1, characterized in that, The teacher model further includes a teacher feature extraction module that extracts features based on unlabeled defect image data, and the student model further includes a student feature extraction module that extracts features based on unlabeled defect image data and labeled defect image data. The teacher feature extraction module has the same structure as the student feature extraction module. The teacher feature extraction module uses the backbone and neck parts of YOLOv11 to perform multi-level feature extraction, and combines spatial coordinate attention mechanism and multi-scale feature fusion. The obtained features are sent to the teacher open set recognizer.

3. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 1, characterized in that, In the positioning reliability screening module, the positioning reliability is: ;in, t c This represents the raw confidence score of the student model output. Used to map this value to The probability value within the interval, Represents the perfect intersection-union ratio between two predicted bounding boxes, where and Bounding box prediction results for the teacher model and the student model, respectively; For the For a sample, when its location reliability At that time, retain its location reliability and record it as . ;when When, set it to zero, that is .

4. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 3, characterized in that, In the open set classification and discrimination module, given the logit value output by the detection head... The closed-set classifier uses the Softmax function to determine which known categories a sample belongs to. The probability of: ； in, For classification confidence; The categorical binary validation output is constructed using the Sigmoid activation function: ； in, This represents the Sigmoid activation function; positive probability. Representing samples and categories The degree of matching, negative probability Used to quantify the degree to which a sample does not belong to this category; The output of the closed-set classifier is fused with the output of the binary verification, thus expanding the classification range. Class expansion to Class, in which Represents all possible categories; Open set classification confidence distribution for: ； Among them, the The class is an unknown defect category, which is obtained by accumulating the rejection probabilities of each known category.

5. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 1, characterized in that, The teacher model also includes: a pseudo-label allocation module for generating pseudo-labels; the pseudo-label allocation module generates pseudo-labels based on unlabeled data using a dual-threshold category adaptive label allocation mechanism; the dual-threshold category adaptive label allocation mechanism optimizes the utilization rate of pseudo-labels by dynamically adjusting the thresholds and adaptively updates pseudo-labels for different categories.

6. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 5, characterized in that, In the dual-threshold adaptive label allocation mechanism, pseudo-labels are classified through three confidence levels: high-confidence pseudo-labels, low-confidence pseudo-labels, and unreliable pseudo-labels. Two time-varying thresholds are set for each category. and And satisfy ; In each training iteration At that time, based on the teacher model's confidence prediction on open sets of unlabeled mini-batch data, the global smoothing confidence threshold was calculated using exponential moving average. : ; in, For EMA attenuation factor, This indicates the number of unlabeled samples in this small batch. This indicates that the teacher network is iterating. Time for sample The given confidence level for the open set; For each category The average confidence level is updated using an exponential moving average: in, Indicates sample Classified as The probability of classifying an open set; Global threshold based on category confidence level Scaling is performed to obtain the category. Dynamic upper threshold : ; category lower threshold Defined as a fixed proportion of the upper threshold: ;in As a proportionality coefficient, in iteration Time as category Assigned category-aware weights Defined as: in, This indicates a weak level of supervision.

7. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 1, characterized in that, The student model further includes: a dynamic weight optimization module for updating weights; the dynamic weight optimization module updates the weights of the student model through a loss function; the student model updates the parameters of the teacher model through an exponentially weighted average. The loss function is: ;in, The supervised loss is used to measure the difference between the adaptive model's predictions and the ground truth annotations in classification and localization. ;in This represents the predicted value and the actual predicted value of the student model; For classifying losses, To regress the loss, For distribution focus loss; For known pseudo-label loss of unlabeled samples, ; For the unknown pseudo-label loss of unlabeled samples, ; The classification items employ a strict hard-label strategy, participating in training only when the confidence level of the open set reaches the upper threshold. The known classes are... Unknown class : ； ； in, CE Represents cross-entropy loss; The regression term uses either a high-confidence classification mechanism or a very high-confidence localization mechanism: ； ； in, Intersection, union, and comparison; The focus term of the distribution employs a dual-threshold hierarchical supervision approach: strong supervision for high confidence levels and weak supervision for medium confidence levels. ； ； in, It is a computational method that transforms bounding box regression in object detection from predicting a single, definite value into predicting a continuous probability distribution.

8. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 7, characterized in that, After obtaining the weights of the student model, the process also includes: calculating the gradient of the current student model's weights and applying a perturbation to the weights according to the gradient direction; The disturbance calculation formula is: ;in, For student network weight, For the disturbance amplitude, The gradient of the loss function with the current weights.

9. The open end-to-end semi-supervised detection method for adaptive unknown defect identification according to claim 1, characterized in that, Also includes: Acquire and store defect image data and defect detection results.

10. An open end-to-end semi-supervised inspection system for adaptive unknown defect identification, characterized in that, include: Image acquisition module, defect detection module, and data storage module; The image acquisition module acquires defect image data in real time through a camera device and transmits the acquired data to the server in real time. The defect detection module includes an open semi-supervised defect detection model; the open semi-supervised defect detection model includes a teacher model and a student model based on the YOLOv11 network; the teacher model includes: a teacher feature extraction module for extracting features from unlabeled defect image data, a teacher open set recognizer for identifying defects in the extracted features, and a pseudo-label assignment module for generating pseudo-labels; the student model includes: a student feature extraction module for extracting features from both unlabeled and labeled defect image data, a student open set recognizer for identifying defects in the extracted features, and a dynamic weight optimization module for updating weights; the teacher feature extraction module and the student feature extraction module have the same structure; the teacher open set recognizer and the student open set recognizer have the same structure; In the teacher feature extraction module, the backbone and neck parts of the YOLOv11 network are used for multi-level feature extraction, and spatial coordinate attention mechanism and multi-scale feature fusion are combined. The teacher open set identifier includes: a positioning reliability screening module and an open set classification discrimination module; the positioning reliability screening module evaluates the spatial consistency and geometric reliability of the detection box based on the positioning reliability threshold, and filters out unstable or low-quality candidate boxes; the open set classification discrimination module includes a closed set classifier and a binary verification mechanism; the binary verification mechanism is used to independently determine whether a sample truly belongs to a given known category, or should be rejected as an unknown category; The pseudo-label allocation module generates pseudo-labels based on unlabeled data using a dual-threshold category adaptive label allocation mechanism. The dynamic weight optimization module updates the weights of the student model through a loss function; the student model updates the parameters of the teacher model through an exponentially weighted average. The data storage module is responsible for storing defect detection data obtained from the defect detection module and raw data obtained from the image acquisition module, and periodically updating model parameters with new defect data to optimize detection performance.

Images