Defect detection model optimization method and device, storage medium and electronic equipment

Abstract

The invention discloses a defect detection model optimization method, which comprises the steps of constructing a defect sample database based on an image of a to-be-detected industrial part, and selecting to execute an incremental learning mode or a full-quantity training mode on an initial detection model based on a performance state of the initial detection model and a newly added data state in the defect sample database, wherein the incremental learning mode comprises the steps of freezing a part of network layers of the model, only training a feature extraction network layer, and updating the model by using newly added data; the full-amount training mode comprises the step of training all network layers of the model by using all available data; performance verification is carried out on the optimized model, and if verification is passed, the optimized model is used for visual defect detection of industrial parts; and collecting low-confidence samples through visual defect detection, marking the low-confidence samples, and feeding back the marked low-confidence samples to the defect sample database so as to update the defect sample database. By means of the method, resource consumption in the model updating process can be reduced, and meanwhile catastrophic forgetting is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial visual inspection, and in particular to a defect detection optimization method and device, a storage medium and an electronic device. BACKGROUND

[0002] In the field of industrial visual inspection, defect detection methods based on deep learning face two core challenges. On the one hand, in actual industrial production environments, especially on high-yield production lines, defect samples themselves are small probability events, resulting in a lack of defect data available for model training. At the same time, parts on the production line often have different shapes, materials and processes, and this "heterogeneous part" characteristic makes the defect morphology varied, further increasing the difficulty of model generalization. On the other hand, when a new defect pattern appears, the traditional solution is to use periodic full model retraining. This solution requires training the entire model from scratch after combining new and old data, which not only has high computational cost and long time consumption, but also causes training costs to continue to grow as data accumulates. More seriously, due to the imbalance of new and old data distribution and quantity, catastrophic forgetting problems are likely to occur during full retraining, that is, the model significantly reduces the ability to recognize previously learned old defects while learning new defect features. In addition, full training requires a large number of new samples to ensure effectiveness, resulting in a long cycle from discovering model defects to completing updates, making it difficult for the system to respond to new quality problems in the production process. These factors together restrict the stable and economic operation of intelligent detection systems in industrial environments that need to be continuously adapted.

[0003] Therefore, the industrial field urgently needs a model continuous optimization method that can significantly reduce model update costs, avoid catastrophic forgetting, and quickly adapt to new defects while ensuring recognition performance. SUMMARY

[0004] Based on the above problems, the embodiments of the present application provide a defect detection model optimization method, device, storage medium and electronic device, which can significantly reduce the consumption of computing resources in the model update process by fusing the dual-path dynamic optimization mechanism of incremental learning and full training, and constructing a data closed loop, effectively avoiding catastrophic forgetting, while realizing the rapid response to new defects in industry and the continuous and stable improvement of model performance, so as to achieve the efficient balance of optimization cost and performance guarantee.

[0005] In a first aspect, the embodiments of the present application provide a defect detection model optimization method, which comprises: constructing a defect sample database based on an image of an industrial part to be detected, the defect sample database being used to store annotated industrial part defect sample data; selecting an incremental learning mode or a full training mode to optimize an initial detection model based on a performance state of the initial detection model and a state of newly added data in the defect sample database, wherein: the incremental learning mode comprises: freezing part of a network layer of the model, training only a feature extraction network layer, and updating the model using the newly added data; the full training mode comprises: training all network layers of the model using all available data; performing performance verification on the optimized model, and if the verification is passed, using the optimized model for visual defect detection of the industrial part; the visual defect detection is used to detect appearance defects of the industrial part; collecting low-confidence samples through the visual defect detection, feeding the annotated low-confidence samples back to the defect sample database to update the defect sample database; wherein the updated defect sample database is used to provide annotated industrial part defect sample data for a next round of model optimization process.

[0006] In a possible implementation, based on the performance state of the initial detection model and the state of the newly added data in the defect sample database, the initial detection model is selected to execute the incremental learning mode or the full training mode, which comprises: When the amount of the newly added data in the defect sample database is less than a first preset threshold, the incremental learning mode is selected to be executed; When the performance index of the initial detection model is improved by less than a second preset threshold after the incremental learning mode is executed for a plurality of times, the full training mode is selected to be executed.

[0007] In a possible implementation, before the defect sample database is constructed based on the image of the industrial part to be detected, the method further comprises: obtaining annotated defect sample data.

[0008] In a possible implementation, when the incremental learning mode is executed, the training data used further comprises part of historical data extracted from the defect sample database.

[0009] In a possible implementation, the model obtained through the full training mode is used as a reference model of the incremental learning mode.

[0010] In a possible implementation, the low-confidence samples comprise samples whose detection confidence for known defects is less than a third preset threshold, and / or samples whose detection confidence for new defects is less than a fourth preset threshold.

[0011] In a possible implementation, after the performance verification is performed on the optimized model, the method further comprises: If the verification fails, a model retraining process is triggered, which includes checking data quality, adjusting training parameters, or directly triggering a full training mode.

[0012] In a second aspect, the embodiments of the present application provide a defect detection model training device, which comprises a construction module, an execution module, and an updating module, wherein: The construction module is configured to construct a defect sample database, which is used to store annotated defect sample data. The execution module is configured to select an incremental learning mode or a full training mode for the initial detection model based on the performance state of the initial detection model and the state of the newly added data in the defect sample database, wherein the incremental learning mode comprises freezing part of the network layers of the model, training only the feature extraction network layer, and updating the model using the newly added data; and the full training mode comprises training all network layers of the model using all available data. The updating module is configured to perform performance verification on the optimized model, and if the verification passes, the model is deployed to an online detection system; low confidence samples are collected by the deployed online detection system, are annotated, and are fed back to the defect sample database to update the content of the defect sample database.

[0013] In a third aspect, the embodiments of the present application provide a computer storage medium, which stores a plurality of instructions, and the instructions are adapted to be loaded by a processor and execute the steps of the above method.

[0014] In a fourth aspect, the embodiments of the present application provide an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is adapted to be loaded by the processor and execute the steps of the above method.

[0015] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following: By constructing a two-level optimization architecture that coordinates incremental learning and full training, the training cost is significantly reduced and stabilized in the long term while ensuring the model's recognition performance. The incremental learning path, through an innovative approach of freezing the parameters of the feature classification network and optimizing only the feature extraction network, not only reduces the computational cost of a single model update to less than half that of traditional full training, but more importantly, it makes the optimization cost only related to the amount of new data, thus freeing it from dependence on the total amount of historical data. This effectively solves the problem of linear growth in training cost with data accumulation in traditional methods. Simultaneously, this... The solution mitigates catastrophic forgetting through a structured parameter freezing strategy, ensuring the model maintains stable recognition capabilities of previously acquired defect knowledge while learning new defect features. Furthermore, an intelligent triggering mechanism based on performance monitoring enables a shift from fixed-cycle training to on-demand training, shortening the model optimization cycle and improving system response speed. Finally, by establishing a complete data loop from low-confidence sample identification to annotation and retraining, an intelligent system capable of autonomously identifying problems in practical applications and driving self-improvement is constructed. Ultimately, this achieves a comprehensive technical effect of controllable cost, stable performance, and continuous evolution of the industrial defect detection model throughout its lifecycle. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A system architecture diagram of a defect detection model optimization system provided in this application embodiment; Figure 2 A flowchart illustrating a defect detection model optimization method provided in an embodiment of this application; Figure 3 A logic block diagram for optimizing a defect detection model provided in an embodiment of this application; Figure 4 A structural block diagram of a defect detection model optimization device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the features and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0020] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0021] As mentioned earlier, deep learning-based defect detection methods have become an important component of intelligent manufacturing in the field of industrial visual inspection. However, these methods face severe challenges in practical applications. First, in industrial production environments with high yield rates, defect samples are inherently low-probability events, resulting in extremely scarce defect data suitable for model training. Simultaneously, parts on the production line often possess different shapes, materials, and processes; this "heterogeneous part" characteristic leads to diverse defect morphologies, further increasing the difficulty of model generalization.

[0022] Existing technical solutions primarily employ a strategy of periodically retraining the entire model. The specific process includes: collecting an initial training dataset for model training; accumulating misclassified samples and new defect samples during model deployment and operation; merging the old and new datasets and retraining the model from scratch once a certain number of new labeled samples have been collected; and finally replacing the old online model with the newly trained model. This traditional approach has three significant drawbacks: training costs increase linearly with data accumulation, requiring retraining the entire network for each model update, consuming enormous computational resources and time; during retraining on merged datasets, the imbalance in the distribution and quantity of old and new data leads to catastrophic forgetting, meaning the model loses its ability to identify old defects while learning new knowledge; and full training requires accumulating a large number of new samples to ensure effectiveness, resulting in an excessively long cycle from discovering model defects to completing model updates, severely hindering system optimization.

[0023] In view of this, this application provides a defect detection model optimization method, apparatus, storage medium, and electronic device. The aim is to construct a hierarchical, intelligent continuous optimization system, dividing the model optimization strategy into two levels: conventional optimization based on incremental learning and deep optimization based on full training. The system comprises four core modules: an online detection and intelligent perception module responsible for real-time visual detection and screening of low-confidence samples; a data management module providing a human-computer interaction interface for quality inspection experts to label and manage samples; a hierarchical optimization strategy execution module providing two parallel optimization paths; and a performance monitoring and decision support module continuously monitoring model performance indicators and providing decision support for engineers. In specific implementation, the system dynamically selects the optimization path according to a preset strategy. When the amount of new data is small, an incremental learning mode is adopted, freezing the feature fusion and feature classification parts of the model and training only the feature extraction backbone network to quickly adapt to new defects with low computational cost. When the model performance reaches a bottleneck, a full training mode is initiated, retraining the model using all data to find the global optimum. This dual-path collaborative working mechanism forms a dual-loop structure of "inner loop incremental learning + outer loop full training," ensuring both model agility and daily optimization while maximizing model performance. Through modular design and a two-level optimization strategy, this solution effectively resolves the contradiction between model optimization cost and performance in industrial scenarios. It provides the ability to quickly respond to new defects while preserving performance improvement potential through full-data training, forming a long-term, self-improving, intelligent model lifecycle management solution. This provides a more economical, efficient, and stable technical solution for the field of industrial visual inspection.

[0024] Please see Figure 1 , Figure 1 This is an exemplary system architecture diagram of a defect detection model optimization method provided in an embodiment of this application.

[0025] likeFigure 1 As shown, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links, etc.

[0026] Terminal 101 can interact with server 103 via network 102 to receive messages from or send messages to server 103. Alternatively, terminal 101 can interact with server 103 via network 102 to receive messages or data sent to server 103 by other users. Terminal 101 can be hardware or software. When terminal 101 is hardware, it can be various electronic devices, including but not limited to smartwatches, smartphones, tablets, laptops, and desktop computers. When terminal 101 is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0027] In this embodiment, terminal 101 can construct a defect sample database based on the image of the industrial part to be detected. The defect sample database is used to store labeled defect sample data of industrial parts. Based on the performance status of the initial detection model and the status of newly added data in the defect sample database, the initial detection model is optimized by selecting either incremental learning mode or full training mode. Incremental learning mode includes freezing some network layers of the model, training only the feature extraction network layer, and updating the model using the newly added data. Full training mode includes training all network layers of the model using all available data. The optimized model is then validated. If the validation is successful, the optimized model is used for visual defect detection of industrial parts. Visual defect detection is used to detect appearance defects of industrial parts. Low-confidence samples are collected through visual defect detection, and these low-confidence samples are labeled and fed back to the defect sample database to update the defect sample database. The updated defect sample database is used to provide labeled defect sample data of industrial parts for the next round of model optimization.

[0028] Server 103 can be a business server providing various services. It should be noted that server 103 can be either hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0029] Alternatively, the system architecture may not include server 103. In other words, server 103 may be an optional device in the embodiments of this specification. That is, the method provided in the embodiments of this specification can be applied to a system structure that only includes terminal 101. The embodiments of this application do not limit this.

[0030] It should be understood that Figure 1 The number of terminals, networks, and servers shown is only illustrative; the number can be any number of terminals, networks, and servers depending on the implementation requirements.

[0031] Please see Figure 2 , Figure 2 This is a flowchart illustrating a defect detection model optimization method provided in an embodiment of this application. The execution entity in this embodiment can be an electronic device performing defect detection model optimization, a processor within the electronic device performing the defect detection model optimization method, or a defect detection model optimization service within the electronic device performing the defect detection model optimization method. For ease of description, the following uses a processor within an electronic device as an example to illustrate the specific execution process of the defect detection model optimization method.

[0032] like Figure 2 As shown, the defect detection model optimization method can include at least: S201. Construct a defect sample database based on the image of the industrial part to be inspected. The defect sample database is used to store labeled defect sample data of industrial parts.

[0033] Specifically, in industrial visual inspection scenarios, defect samples are characterized by scarcity, diversity, and continuous growth, necessitating the establishment of a dedicated database for systematic management. In one possible implementation, before constructing a defect sample database based on images of the industrial parts to be inspected, images of the industrial parts are acquired using image acquisition equipment. These images are then annotated, and standardized annotation files are generated. In practice, raw data is first collected through two parallel channels: first, quality inspectors use high-resolution industrial cameras at a dedicated imaging station to acquire multi-angle images of defective workpieces, saving them as RAW format files and annotating the data; second, high-resolution industrial cameras deployed on the production line automatically acquire workpiece images, forming a raw data pool to be annotated, which is then manually annotated once a certain quantity is reached or periodically. This data originates from newly discovered defect samples on the production line or unverified samples accumulated during the system's historical operation. All collected data is annotated using tools such as Labellim or Labelme to perform fine-grained edge annotation on the defects of the workpieces, generating JSON format annotation files that fully record the defect location coordinates and category information. All labeled data is stored uniformly in a defect sample database. The database categorizes and manages samples according to multiple dimensions such as part type, defect category, and collection date, forming a structured sample library. This database not only provides high-quality data support for subsequent model optimization but also provides data traceability capabilities for model performance analysis and optimization decisions through comprehensive metadata records.

[0034] Please see Figure 3 , Figure 3 This is a logic block diagram illustrating an optimized defect detection model provided in an embodiment of this application. For example... Figure 3 As shown, after labeling the data, all labeled data needs to be stored in the defect sample database to construct the defect sample database.

[0035] S202. Based on the performance status of the initial detection model and the status of newly added data in the defect sample database, select either incremental learning mode or full training mode for the initial detection model. Incremental learning mode includes freezing some network layers of the model, training only the feature extraction network layer, and updating the model using the newly added data. Full training mode includes training all network layers of the model using all available data.

[0036] Specifically, an intelligent training path decision-making mechanism is established to optimize resource allocation by quantitatively evaluating model performance and data status. In practice, the system continuously monitors two key indicators: first, model performance, primarily based on mAP50 on the validation set during forward propagation, while also tracking trends in false negative and false positive rates; second, the data status in the defect sample database, focusing on the number of new samples, category distribution, and their proportional relationship with historical data. In one possible implementation, when the amount of new data in the defect sample database is below a first preset threshold, incremental learning mode is selected; when the performance improvement of the initial detection model is below a second preset threshold after multiple consecutive executions of incremental learning mode, full training mode is selected. The first preset threshold is generally set to 10% of the average number of defect samples of each type in the defect sample database. If the number of new samples is less than 10% of the average number of defect samples of each type in the database, and the model performance does not show a significant decline, incremental learning mode is automatically selected. In this mode, the system employs a layered parameter freezing strategy, specifically freezing all parameters of the feature fusion network and feature classification network, while only granting trainable access to the feature extraction network. Subsequently, new data is used to perform forward and backward propagation calculations on this network, updating the weight parameters through gradient descent, thereby enabling the model to quickly adapt to new defect features. When executing the incremental learning mode, the training data also includes a portion of historical data extracted from the defect sample database. The system selects samples from each existing defect category in the database at a certain proportion (e.g., randomly selecting a fixed number or a certain percentage from each category). The purpose of this is to allow the model to continuously "review" and "consolidate" previously learned defect patterns while learning new defect features, especially those important defect types with few samples. This ensures that the model retains the ability to recognize old knowledge as much as possible during parameter updates, which is one of the core technical means to mitigate catastrophic forgetting. Simultaneously, the model obtained through the full-scale training mode will serve as the baseline model for the incremental learning mode. When the model's performance improvement falls below the second preset threshold, it generally means that the model's mAP50 improvement on the validation set is less than 1% for three consecutive times, or the number of new samples has reached the threshold for triggering full training. At this time, the system will issue a performance bottleneck alarm and initiate full training mode. Alternatively, full training mode can be manually activated at this time; this embodiment does not impose any special limitations on this. In this mode, the system releases the parameter freeze state of all network layers and uses all available data in the defect sample database to perform end-to-end training on the feature extraction network, feature fusion network, and feature classification network, finding the optimal parameter combination for the model through global optimization.The entire decision-making process combines quantitative indicators with trend analysis. It can be automatically triggered by preset thresholds, or engineers can make manual decisions based on comprehensive evaluation after the system issues an alert, ensuring that each model update achieves the best balance between cost and performance.

[0037] Please continue reading. Figure 3 ,like Figure 3 As shown, the defect sample database provides all training samples to the full training mode for training, re-initializes and trains the model end-to-end, maximizes the potential of the model, and uses this model as the benchmark model for subsequent incremental training.

[0038] S203. Perform performance verification on the optimized model. If the verification is successful, the optimized model will be used for visual defect detection of industrial parts.

[0039] Specifically, after incremental learning or full training is completed, the system comprehensively evaluates the optimized model on a validation set independent of the training set. This validation set contains low-confidence samples of known defects. Performance validation employs k-fold cross-validation, using mean precision (mAP50) as the core metric, while strictly controlling the false negative rate below 3% and the false positive rate to not exceed a preset threshold. During the validation process, the system generates a detailed performance analysis report, showcasing key metrics such as precision and recall for each defect category.

[0040] In one possible implementation, if the validation fails, a model retraining process is triggered. This process includes checking data quality, adjusting training parameters, or directly triggering full-scale training. Specifically, the system first automatically generates a detailed failure analysis report, marking the specific performance indicators that fail to meet standards and their gaps from those standards. If the validation fails, the system automatically marks the training as a failure and triggers the corresponding handling mechanism—for models that fail incremental learning, full-scale training is recommended; for models that fail full-scale training, engineers are prompted to check data quality or adjust hyperparameters. Subsequently, a tiered investigation and handling mechanism is initiated: the first step is to check data quality. The system automatically analyzes the consistency of annotations, class balance, and sample validity of the training dataset, identifying potential data problems by calculating indicators such as bounding box overlap and class distribution entropy. If an excessively high annotation error rate or severe missing samples in a specific class is found, the system returns to the data management module requiring re-annotation or supplementary data collection. If the data quality check is successful, the system proceeds to the training parameter adjustment stage. Based on successful configurations from historical training records, the system optimizes and adjusts the current hyperparameter set. Typical adjustments include: dynamically increasing data augmentation strength to enhance sample diversity, moderately reducing the learning rate to stabilize the convergence process, or adjusting the weight coefficients of each category in the loss function to improve sample imbalance. The adjusted parameters will be applied to the next training task. If the above targeted optimizations still fail to pass validation, the system will eventually trigger full training mode. At this time, high-performance computing resources are automatically allocated, all data in the defect sample database is loaded, and end-to-end full training is performed on all network layers of the model. The entire retraining process adopts a progressive processing logic, ensuring that each optimization attempt is based on the analysis of the previous failure. A maximum retry threshold is set. If multiple consecutive retraining attempts still fail to pass validation, the system will stop the automated process and send a manual intervention request for engineers to conduct in-depth technical investigation.

[0041] S204. Collect low-confidence samples through visual defect detection, and feed the labeled low-confidence samples back to the defect sample database to update the defect sample database; wherein, the updated defect sample database is used to provide labeled industrial part defect sample data for the next round of model optimization process.

[0042] For details, please continue reading Figure 3 ,like Figure 3As shown, if the model passes validation (the main criteria are mAP50 ≥ 0.85 and a false negative rate meeting the standard), the deployment process begins: first, the model weight file is converted to a format suitable for inference; then, it is integrated into the online detection system through a version control system; finally, it is released in a phased manner within a specific time period, gradually replacing the original model and continuously monitoring its online performance. After deployment, the online detection system outputs detection results and constructs a data feedback loop by analyzing the model inference results in real time. Low-confidence samples include samples with a detection confidence level below the third preset threshold for known defects, and / or samples with a detection confidence level below the fourth preset threshold for new defects. Specifically, the system sets a double-reset confidence threshold: for detection results of known defect types, samples with a confidence level below 0.9 (the third preset threshold) will be marked as low-confidence samples; for new defect types not defined by the system, samples with a confidence level below 0.7 (the fourth preset threshold) will be classified as suspected new defect samples. These samples, along with their corresponding original images, are automatically collected and temporarily stored in the review area. Quality inspectors use a dedicated interface to review, confirm, and accurately label them. The labeled qualified samples are then imported into the corresponding category of the defect sample database, enabling continuous expansion and updating of the defect sample database and providing high-quality data for the next round of model optimization.

[0043] This application provides a defect detection model optimization method. This solution, through a collaborative optimization mechanism integrating incremental learning and full training, significantly reduces model update costs while effectively curbing catastrophic forgetting. An intelligent decision-making mechanism based on model performance and data status enables a shift from fixed-cycle training to on-demand training, greatly improving optimization efficiency and resource utilization. By introducing historical data playback and full training baseline reset strategies, existing knowledge is reinforced while achieving a step-by-step leap in model performance. Furthermore, a robust verification mechanism and clear data screening criteria ensure system stability, while a complete low-confidence sample retrieval loop drives continuous self-evolution, ultimately forming a cost-effective, responsive, and continuously optimized industrial inspection lifecycle management solution.

[0044] Please see Figure 4 , Figure 4 This is a structural block diagram of a defect detection model optimization device provided in an embodiment of this application. Figure 4 As shown: The defect detection model optimization device 400 includes: a construction module 410, an execution module 420, a performance verification module 430, and an update module 440. Wherein: Module 410 is used to build a defect sample database based on the image of the industrial part to be inspected. The defect sample database is used to store labeled defect sample data of industrial parts. The execution module 420 is used to select either incremental learning mode or full training mode to optimize the initial detection model based on the performance status of the initial detection model and the status of newly added data in the defect sample database. The incremental learning mode includes freezing some network layers of the model, training only the feature extraction network layer, and updating the model using the newly added data. The full training mode includes training all network layers of the model using all available data. The performance verification module 430 is used to verify the performance of the optimized model. If the verification is successful, the optimized model will be used for visual defect detection of industrial parts. Visual defect detection is used to detect appearance defects of industrial parts. The update module 440 is used to collect low-confidence samples through visual defect detection, and to feed the low-confidence samples back to the defect sample database after labeling them in order to update the defect sample database. The updated defect sample database is used to provide labeled industrial part defect sample data for the next round of model optimization.

[0045] In some possible embodiments, the execution module 420 includes: The first selection unit is used to select the incremental learning mode when the amount of newly added data in the defect sample database is lower than the first preset threshold. The second selection unit is used to select to execute the full training mode when the performance index of the initial detection model is lower than the second preset threshold after multiple consecutive executions of the incremental learning mode.

[0046] In some possible embodiments, the defect detection model optimization device 400 further includes: An image acquisition unit is used to acquire images of industrial parts to be inspected through an image acquisition device. The annotation unit is used to annotate the image of the industrial part to be tested and generate a standardized annotation file.

[0047] In some possible embodiments, when performing incremental learning mode, the training data used also includes a portion of historical data extracted from a defect sample database.

[0048] In some possible implementations, the model obtained through full training mode will serve as the baseline model for incremental learning mode.

[0049] In some possible embodiments, low-confidence samples include samples whose confidence in detecting known defects is lower than a third preset threshold, and / or samples whose confidence in detecting new defects is lower than a fourth preset threshold.

[0050] In some possible embodiments, the defect detection model optimization device 400 further includes: The retraining unit is used to trigger the model retraining process if the validation fails. The retraining process includes: checking data quality, adjusting training parameters, or directly triggering the full training mode.

[0051] It should be noted that the defect detection model optimization device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the defect detection model optimization method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the defect detection model optimization device and the defect detection model optimization method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0052] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0053] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 may include: at least one processor 501, at least one network interface 504, user interface 503, memory 505, and at least one communication bus 502.

[0054] The communication bus 502 is used to enable communication between these components.

[0055] The user interface 503 may include a display screen and a camera. Optional user interfaces 503 may include standard wired interfaces and wireless interfaces.

[0056] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0057] The processor 501 may include one or more processing cores. The processor 501 connects to various parts within the electronic device 500 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by calling data stored in the memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0058] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a defect detection model optimization application.

[0059] exist Figure 5In the illustrated electronic device 500, the user interface 503 is mainly used to provide an input interface for the user and acquire user input data; while the processor 501 can be used to call the defect detection model optimization application stored in the memory 505 and specifically perform the following operations: construct a defect sample database based on the image of the industrial part to be detected, the defect sample database is used to store labeled industrial part defect sample data; based on the performance status of the initial detection model and the status of newly added data in the defect sample database, select to execute incremental learning mode or full training mode to optimize the initial detection model, wherein: the incremental learning mode includes: freezing some network layers of the model, training only the feature extraction network layer, and updating the model using new data; the full training mode includes: training all network layers of the model using all available data; perform performance verification on the optimized model, and if the verification is successful, use the optimized model for visual defect detection of industrial parts; visual defect detection is used to detect appearance defects of industrial parts; collect low confidence samples through visual defect detection, and feed the labeled low confidence samples back to the defect sample database to update the defect sample database; wherein, the updated defect sample database is used to provide labeled industrial part defect sample data for the next round of model optimization process.

[0060] In some possible embodiments, when the processor 501 executes the performance status of the initial detection model and the newly added data status in the defect sample database, and selects to execute either incremental learning mode or full training mode for the initial detection model, it specifically performs the following: When the amount of newly added data in the defect sample database is less than the first preset threshold, the incremental learning mode is selected; when the performance index of the initial detection model is less than the second preset threshold after multiple consecutive executions of the incremental learning mode, the full training mode is selected.

[0061] In some possible embodiments, before the processor 501 executes the construction of a defect sample database based on the image of the industrial part to be inspected, it is also used to perform: Obtain labeled defect sample data.

[0062] In some possible embodiments, when performing incremental learning mode, the training data used also includes a portion of historical data extracted from a defect sample database.

[0063] In some possible implementations, the model obtained through full training mode will serve as the baseline model for incremental learning mode.

[0064] In some possible embodiments, low-confidence samples include samples whose confidence in detecting known defects is lower than a third preset threshold, and / or samples whose confidence in detecting new defects is lower than a fourth preset threshold.

[0065] In some possible embodiments, after performing performance verification on the optimized model, processor 501 is also used to perform: If the verification fails, the model retraining process is triggered. The retraining process includes: checking data quality, adjusting training parameters, or directly triggering the full training mode.

[0066] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the above-described instructions. Figure 2 One or more steps in the illustrated embodiment. If the constituent modules of the above-described defect detection model optimization device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0067] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid state disks (SSDs)).

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation schemes can be combined arbitrarily.

[0069] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A defect detection model optimization method, the method being applied to industrial visual defect detection, characterized in that, The method includes: A defect sample database is constructed based on images of industrial parts to be inspected. The defect sample database is used to store labeled defect sample data of industrial parts. Based on the performance status of the initial detection model and the status of newly added data in the defect sample database, the initial detection model is optimized by selecting either incremental learning mode or full training mode. The incremental learning mode includes freezing some network layers of the model, training only the feature extraction network layer, and updating the model using the newly added data. The full training mode includes training all network layers of the model using all available data. The optimized model is then validated for performance. If the validation is successful, the optimized model is used for visual defect detection of industrial parts. The visual defect detection is used to detect appearance defects of industrial parts. Low-confidence samples are collected through visual defect detection, and these low-confidence samples are labeled and fed back into the defect sample database to update the defect sample database. The updated defect sample database is used to provide labeled industrial part defect sample data for the next round of model optimization.

2. The method as described in claim 1, characterized in that, Based on the performance status of the initial detection model and the status of newly added data in the defect sample database, the initial detection model is selected to execute either incremental learning mode or full training mode, including: When the amount of newly added data in the defect sample database is lower than the first preset threshold, the incremental learning mode is selected to be executed. When the performance index of the initial detection model is lower than the second preset threshold after multiple consecutive executions of the incremental learning mode, the full training mode is selected to be executed.

3. The method as described in claim 1, characterized in that, Before constructing the defect sample database based on the image of the industrial part to be inspected, the method further includes: Images of industrial parts to be inspected are obtained through image acquisition equipment. The image of the industrial part to be tested is annotated, and a standardized annotation file is generated.

4. The method as described in claim 1, characterized in that, When executing the incremental learning mode, the training data used also includes some historical data extracted from the defect sample database.

5. The method as described in claim 1, characterized in that, The model obtained through the full training mode will serve as the benchmark model for the incremental learning mode.

6. The method as described in claim 1, characterized in that, The low-confidence samples include samples whose confidence in detecting known defects is lower than a third preset threshold, and / or samples whose confidence in detecting new defects is lower than a fourth preset threshold.

7. The method as described in claim 1, characterized in that, After verifying the performance of the optimized model, the method further includes: If the verification fails, the model retraining process is triggered. The retraining process includes: checking data quality, adjusting training parameters, or directly triggering the full training mode.

8. A defect detection model optimization device, characterized in that, The device includes: A construction module is used to build a defect sample database based on images of industrial parts to be inspected. The defect sample database is used to store labeled defect sample data of industrial parts. An execution module is used to select either incremental learning mode or full training mode to optimize the initial detection model based on the performance status of the initial detection model and the status of newly added data in the defect sample database. The incremental learning mode includes freezing some network layers of the model, training only the feature extraction network layer, and updating the model using the newly added data. The full training mode includes training all network layers of the model using all available data. The performance verification module is used to verify the performance of the optimized model. If the verification is successful, the optimized model is used for visual defect detection of industrial parts. Visual defect detection is used to detect appearance defects of industrial parts. The update module is used to collect low-confidence samples through the visual defect detection, and to feed the labeled low-confidence samples back to the defect sample database to update the defect sample database; wherein, the updated defect sample database is used to provide labeled industrial part defect sample data for the next round of model optimization process.

9. A storage medium, characterized in that, The storage medium stores a plurality of instructions adapted for loading by a processor and executing the steps of the method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method as described in any one of claims 1 to 7.

Images