A method and system for detecting defects in industrial products

Abstract

This invention provides a method and system for detecting defects in industrial products. It acquires defect-free images and generates defect samples by fusing a stable diffusion model, low-rank adaptive technology, and a control network. These defect samples are then merged with real defect samples from the industrial production process to construct a comprehensive sample library, providing high-quality data support for training the detection model. An improved YOLOv8 model is constructed and trained based on the comprehensive sample library. A closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filtering fusion scheme are employed to respond to illumination fluctuations in real time. Images of the products to be inspected are acquired and input into the trained improved YOLOv8 model, which outputs defect information. Specifically, the above method significantly improves detection accuracy, stability, and efficiency.

Description

Technical Field

[0001] This invention belongs to the field of industrial product defect detection technology, and specifically relates to a method and system for industrial product defect detection. Background Technology

[0002] In the field of intelligent manufacturing, visual inspection of appearance defects and precision dimensions is a core link in ensuring the quality stability of high-end products (such as precision parts such as automotive sensor components) and improving the automation efficiency of production lines. Its inspection accuracy, environmental adaptability and inspection efficiency directly determine the quality control level and market competitiveness of industrial products, and are one of the key supporting technologies for promoting the high-quality development of the high-end manufacturing industry.

[0003] Currently, the field of visual inspection for appearance defects and precision dimensions still faces three major technological bottlenecks: First, the challenge of training and optimizing deep learning models with small sample defect data. In industrial scenarios, the number of product defect samples is often limited due to factors such as production process control and sample collection difficulty. Traditional deep learning models rely on a large amount of labeled data, which easily leads to overfitting in small sample scenarios, resulting in insufficient model recognition accuracy and generalization ability, and failing to cover diverse micro-defect types. Second, the shortcomings of image acquisition and processing technology under dynamic and complex lighting environments. In industrial production lines, lighting conditions are prone to fluctuations with changes in production processes and environment. Ordinary image acquisition systems lack adaptive adjustment capabilities, and their anti-interference image processing algorithms have weak performance, easily leading to a decline in the quality of acquired images and blurred defect features, directly affecting the accuracy of inspection results. Third, the problem of insufficient real-time data processing and precise positioning in multi-station collaborative inspection systems. High-speed production lines require multi-sensor, multi-station collaboration to complete multi-dimensional inspection. Existing technologies struggle to achieve efficient fusion of multi-source data and suffer from insufficient defect location accuracy and high data processing latency, failing to meet the dual requirements of real-time inspection and precise judgment in high-speed production lines.

[0004] Existing visual inspection technologies struggle to overcome the aforementioned bottlenecks: traditional machine vision methods rely on manually designed features, resulting in poor adaptability to complex microscopic defects; conventional deep learning models are not optimized for small sample scenarios, leading to large model sizes, slow inference speeds, and difficulty in adapting to the pace of high-speed production lines; and ordinary lighting adjustment systems and anti-interference algorithms lack dynamic response capabilities, making them unable to cope with lighting fluctuations in industrial environments. Summary of the Invention

[0005] Based on this, the present invention provides an industrial product defect detection method and system, which aims to solve at least one of the above-mentioned technical problems.

[0006] A first aspect of this invention provides a method for detecting defects in industrial products, the method comprising: Defect-free images are acquired, and defect samples are generated by fusing a stable diffusion model, low-rank adaptation technique, and control network. The defect samples are merged with real defect samples from the industrial production process to construct a comprehensive sample library; Construct an improved YOLOv8 model and train it using the comprehensive sample library. The system acquires images of the products to be inspected, inputs them into a trained improved YOLOv8 model, and outputs defect information. It employs a closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filter fusion scheme to respond to illumination fluctuations in real time in order to acquire images of the products to be inspected. The improved YOLOv8 model employs a combination of modulated deformable convolution and the C2fCIB module in the feature extraction stage; introduces the Dy_Sample dynamic upsampling operator in the feature fusion stage; integrates the advantages of self-attention and convolution through the ACmix module in the feature enhancement stage; and optimizes the downsampling process in the feature transfer stage.

[0007] Furthermore, the step of acquiring defect-free images and generating defect samples by fusing a stable diffusion model, low-rank adaptation technology, and a control network includes: Collect defect-free images and extract GLCM material feature vectors, Canny edge maps, and high-precision depth maps to construct a defect-free basic feature library; A fusion-stable diffusion model is selected as the basic generative model, and a low-rank adaptation technique is used to train the fusion-stable diffusion model. The training data comes from the defect-free basic feature library and is supplemented with a preset number of labeled real defect samples. During training, the material and structural feature parameters of the product are injected into the low-rank adapter parameter space of the fusion stable diffusion model to ensure that the base material properties and structural morphology of the generated samples are highly consistent with the real product, with a structural similarity of no less than 90%, and finally a product-specific LoRA model is trained. Based on the product's structural integrity defects, appearance defects, and size-related defects, a multimodal ControlNet control condition is constructed and works in conjunction with the product's dedicated LoRA model to generate defect samples.

[0008] Furthermore, in the step of using a closed-loop adaptive illumination adjustment system with an improved global adaptive gamma correction algorithm and a Gaussian filter fusion scheme to respond to illumination fluctuations in real time and acquire images of the product to be inspected, the closed-loop adaptive illumination adjustment system monitors the fluctuations in ambient illumination in real time through an ambient light sensor and feeds the data back to the central processing unit. The central processing unit calculates compensation instructions based on the preset brightness target value using a control algorithm, and drives the programmable LED light source controller to dynamically adjust the brightness of the multiple LED light sources integrated around the imaging system, thereby actively canceling ambient light interference and ensuring that the light intensity projected onto the sample surface is constant.

[0009] Furthermore, in the step of acquiring images of the product to be inspected by employing a closed-loop adaptive illumination adjustment system with an improved global adaptive gamma correction algorithm and a Gaussian filtering fusion scheme to respond to illumination fluctuations in real time, the normalized average gray value of the image is mapped to an adaptive gamma value related to the overall brightness of the image. Specifically, when the average gray value is low, a gamma value less than 1 is automatically generated to expand the dynamic range of the dark area and enhance the defect details hidden in the shadows; when the average gray value is high, a gamma value greater than 1 is generated to suppress overexposure in the highlight area and restore lost texture. In addition, a weighting function based on image histogram characteristics or local contrast is introduced to fine-tune the gamma value to preserve the contrast between small target defects and the background.

[0010] Furthermore, in the step of acquiring images of the product to be inspected by employing a closed-loop adaptive illumination adjustment system with an improved global adaptive gamma correction algorithm and a Gaussian filter fusion scheme to respond to illumination fluctuations in real time, a Gaussian filter weight matrix with a standard deviation σ and a kernel size of k×k is used to smooth the image. The weight matrix elements are calculated according to the formula... Calculate and reduce image noise interference, where x and y are coordinates relative to the kernel center, with values ​​ranging from [value range missing]. Integer step size.

[0011] Furthermore, the architecture of the improved YOLOv8 model includes a backbone network, a neck, and a head. The backbone network includes an input layer, a convolutional layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, and an SPPF layer connected in sequence. The first feature extraction layer consists of a first modulated deformable convolutional module and a first C2fCIB module. The second feature extraction layer consists of a second modulated deformable convolutional module and a second C2fCIB module. The third feature extraction layer consists of a third modulated deformable convolutional module and a third C2fCIB module. The fourth feature extraction layer consists of a fourth modulated deformable convolutional module and a fourth C2fCIB module. The neck layer comprises, in sequence, a first Dy_Sample, a first stitching layer, a fifth C2fCIB module, a second Dy_Sample, a second stitching layer, a C2fMLLABlock, a fifth modulated deformable convolutional module, a third stitching layer, a sixth C2fCIB module, a seventh C2fCIB module, a fourth stitching layer, and an eighth C2fCIB module. The output of the second C2fCIB module is connected to the input of the second stitching layer, the output of the third C2fCIB module is connected to the input of the first stitching layer, the output of the SPPF layer is connected to the inputs of the first Dy_Sample and the fourth stitching layer, the output of the C2fMLLABlock is also connected to the first ACmix module, the output of the sixth C2fCIB module is also connected to the second ACmix module, and the output of the eighth C2fCIB module is connected to the third ACmix module. The head includes three detection heads, wherein the first ACmix module is connected to the first detection head, the second ACmix module is connected to the second detection head, and the third ACmix module is connected to the third detection head.

[0012] Furthermore, in the C2fCIB module, the input features pass through the first CBS module. The features passed through the CBS module are split into multiple branches, each branch is connected to an independent CIB, and each branch extracts local features of different dimensions in parallel. Then, the outputs of all CIB branches are concatenated by Concat to integrate the feature information of multiple branches. Finally, the concatenated features pass through the second CBS module, where convolutional fusion of channel features and adjustment of the number of output channels are performed to obtain the output features of the module. The CBS module is a module that combines convolutional layers, batch normalization, and activation functions, while CIB is a lightweight convolutional unit.

[0013] A second aspect of the present invention provides an industrial product defect detection system for implementing an industrial product defect detection method provided in the first aspect of the present invention, the system comprising: The acquisition module is used to acquire defect-free images and generate defect samples by fusing a stable diffusion model, low-rank adaptation technology and a control network. The merging module is used to merge the defect samples with real defect samples from the industrial production process to build a comprehensive sample library. The building module is used to build an improved YOLOv8 model and train the improved YOLOv8 model based on the comprehensive sample library. The input module is used to acquire images of the product to be inspected. It inputs a trained improved YOLOv8 model and outputs defect information. It employs a closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filter fusion scheme to respond to illumination fluctuations in real time in order to acquire images of the product to be inspected. The improved YOLOv8 model employs a combination of modulated deformable convolution and the C2fCIB module in the feature extraction stage; introduces the Dy_Sample dynamic upsampling operator in the feature fusion stage; integrates the advantages of self-attention and convolution through the ACmix module in the feature enhancement stage; and optimizes the downsampling process in the feature transfer stage.

[0014] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the industrial product defect detection method provided in the first aspect.

[0015] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the industrial product defect detection method provided in the first aspect.

[0016] The present invention provides an industrial product defect detection method and system. This method acquires defect-free images and generates defect samples by fusing a stable diffusion model, low-rank adaptive technology, and a control network. The defect samples are then merged with real defect samples from the industrial production process to construct a comprehensive sample library, providing high-quality data support for training the detection model. An improved YOLOv8 model is constructed and trained based on the comprehensive sample library. A closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filtering fusion scheme are employed to respond to illumination fluctuations in real time. Images of the product to be inspected are acquired, input into the trained improved YOLOv8 model, and defect information is output. Specifically, the above method significantly improves detection accuracy, stability, and efficiency. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the implementation of an industrial product defect detection method according to Embodiment 1 of the present invention. Figure 2 A schematic diagram illustrating the architecture for improving the YOLOv8 model; Figure 3 This is a schematic diagram of the C2fCIB module architecture; Figure 4 This is a structural block diagram of an industrial product defect detection system provided in Embodiment 2 of the present invention; Figure 5 This is a structural block diagram of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0018] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0019] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0021] Example 1 Please see Figure 1 , Figure 1 The present invention illustrates an industrial product defect detection method provided in Embodiment 1, which specifically includes steps S01 to S04.

[0022] Step S01: Acquire defect-free images and generate defect samples by fusing a stable diffusion model, low-rank adaptation technology, and a control network.

[0023] Specifically, defect-free images are acquired, and GLCM material feature vectors, Canny edge maps, and high-precision depth maps are extracted to construct a defect-free basic feature library. In this embodiment of the invention, defect-free high-definition images of the target product acquired by an industrial telecentric lens are used as the data foundation. To inject prior knowledge of the product's physics and geometry, the Image Feature Extractor in the SD-WebUI plugin is used to extract the GLCM material feature vectors of the product, including surface attributes such as the matte texture of plastic and the reflective properties of metal. The Canny edge map of the product is extracted through ControlNet, including the spatial layout relationships and contour shapes of key components, generating a black-and-white edge map for spatial constraints. Simultaneously, to provide prior knowledge of three-dimensional morphology, a monocular depth estimation model is used to generate a high-precision depth map. Based on the above multimodal feature extraction results, a standardized defect-free basic feature library is constructed, providing a foundation for subsequent controllable generation. The Stable Diffusion model is selected as the basic generative model, and the Low Rank Adaptation (LoRA) technique is used to train the Stable Diffusion model. The training data comes from the defect-free basic feature library and is supplemented with a preset number of labeled real defect samples. During training, key parameters such as rank (8-16) and learning rate (2e-4~5e-4) are fine-tuned. The material and structural feature parameters of the product are injected into the low-rank adapter parameter space of the fusion stable diffusion model to ensure that the base material properties and structural morphology of the generated samples are highly consistent with the real products, with a structural similarity of no less than 90%. Finally, a product-specific LoRA model (.safetensors format) is trained. Based on the product's structural integrity defects, appearance defects, and size-related defects, a multimodal ControlNet control condition is constructed and works in conjunction with the product-specific LoRA model to generate defect samples. It should be noted that for structural integrity defects such as perforation defects, breakage, missing studs, glue coating, and broken positioning posts, a semantic segmentation mask is used as a control condition. This mask is generated by annotating key structures in the defect-free base image using the LabelMe annotation tool or the MaskR-CNN segmentation model, constraining defects to be generated only within specific component areas to ensure that the defect morphology conforms to the physical structural constraints. To address visual defects and surface flaws such as missing QR codes on nameplates, rubber coating on nuts, foreign objects on fisheye needles, burrs around inserts, and surface contamination, a combined control condition is employed, using a high-precision Canny edge profile extracted by ControlNet and a GLCM material feature vector. The Canny edge profile precisely constrains the shape and orientation of surface defects, ensuring a natural connection with component edges; the GLCM material feature vector accurately constrains the material properties of the defect area, ensuring consistency with the original product material and preventing material distortion. To address size-related defects, a product depth map generated by a binocular telecentric camera or a depth estimation model is used as a control condition to precisely constrain the three-dimensional shape and depth characteristics of the defects, simulating real physical phenomena such as concavity, convexity, and indentation.

[0024] Furthermore, a standardized defect description system should be established, and quantitative prompt generation specifications should be designed to accurately guide the generation direction. Among them, single defect prompts should clearly define the defect category, relative location, and morphological characteristics, while multi-defect prompts should be combined based on the defect coexistence rules in real production scenarios.

[0025] During the generation phase, the system integrates and loads the basic model, product-specific LoRA model, and multimodal ControlNet control conditions based on platforms such as Automatic1111 WebUI. By setting core parameters such as the DPM++2MKarras sampling method and a sampling step count of 25-30, a sample set of generated defects covering appearance identification and structural integrity is collaboratively generated. Finally, manual screening is required to remove samples that do not conform to physical laws, ensuring the high authenticity and usability of the generated data.

[0026] Step S02: Merge the defect samples with real defect samples from the industrial production process to construct a comprehensive sample library.

[0027] Understandably, using defect-free images captured by industrial telecentric lenses as initial input, high-fidelity defect samples are generated through StableDiffusion+LoRA+ControlNet technology. These samples are then merged with real defect samples collected in production to construct a comprehensive sample library. This sample library is then divided into 2:8 ratios and directly used for the pre-training and fine-tuning of the YOLOv8 model, ensuring that the model can acquire sufficient and diverse defect features during the training phase.

[0028] Step S03: Construct an improved YOLOv8 model and train the improved YOLOv8 model based on the comprehensive sample library.

[0029] In industrial product defect detection, the accurate identification of small flaws and irregular defects faces many challenges, such as small targets being easily submerged by complex backgrounds, defects being difficult to distinguish from normal textures, and details being lost during multi-scale feature transfer. In addition, the real-time detection capability must be taken into account to meet the online detection needs of the production line.

[0030] In this embodiment of the invention, an improved architecture based on YOLOv8 is proposed, which achieves efficient detection of industrial defects through end-to-end optimization. Please refer to [link to relevant documentation]. Figure 2 This diagram illustrates the improved YOLOv8 model architecture. In the feature extraction stage, a combination of modulated deformable convolution and the C2fCIB module is used to efficiently preserve local details of minor defects while dynamically focusing on irregular defect regions, enhancing the recognizability of small target features. In the feature fusion stage, the Dy_Sample dynamic upsampling operator is introduced to replace the traditional fixed interpolation method, specifically restoring small target details during feature amplification and avoiding information ambiguity. In the feature enhancement stage, the ACmix module integrates the advantages of self-attention and convolution, collaboratively modeling global context and local details to effectively distinguish defects from complex background textures. In the feature transfer stage, modulated deformable convolution optimizes the downsampling process, ensuring that multi-scale defect features are not lost during transfer. This improved architecture significantly enhances the detection accuracy of irregular defects and minor flaws in industrial products, reduces false negatives and missed detections, and maintains real-time detection capabilities, meeting the practical application needs of online inspection on production lines.

[0031] Specifically, the improved architecture of the YOLOv8 model includes a backbone network, a neck, and a head. The backbone network includes an input layer, a convolutional layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, and an SPPF layer connected in sequence. The first feature extraction layer consists of a first modulated deformable convolutional module and a first C2fCIB module. The second feature extraction layer consists of a second modulated deformable convolutional module and a second C2fCIB module. The third feature extraction layer consists of a third modulated deformable convolutional module and a third C2fCIB module. The fourth feature extraction layer consists of a fourth modulated deformable convolutional module and a fourth C2fCIB module. The neck layer comprises, in sequence, a first Dy_Sample, a first stitching layer, a fifth C2fCIB module, a second Dy_Sample, a second stitching layer, a C2fMLLABlock, a fifth modulated deformable convolutional module, a third stitching layer, a sixth C2fCIB module, a seventh C2fCIB module, a fourth stitching layer, and an eighth C2fCIB module. The output of the second C2fCIB module is connected to the input of the second stitching layer, the output of the third C2fCIB module is connected to the input of the first stitching layer, the output of the SPPF layer is connected to the inputs of the first Dy_Sample and the fourth stitching layer, the output of the C2fMLLABlock is also connected to the first ACmix module, the output of the sixth C2fCIB module is also connected to the second ACmix module, and the output of the eighth C2fCIB module is connected to the third ACmix module. The head includes three detection heads, wherein the first ACmix module is connected to the first detection head, the second ACmix module is connected to the second detection head, and the third ACmix module is connected to the third detection head.

[0032] It's important to note that in existing YOLOv8 architectures, the modulated deformable convolutions in the backbone network are all ordinary convolutions with fixed sampling positions within a rectangular grid. This limits their ability to capture features of irregular defects in industrial scenarios (such as curved scratches and irregular dents)—fixed sampling may miss the edges of small imperfections or only cover part of the defect area, resulting in incomplete feature representation. Replacing these with modulated deformable convolutions allows the convolution kernel sampling positions to adapt to the defect shape (e.g., shifting towards the scratch endpoints) by learning dynamic offsets and attention masks. Simultaneously, the mask suppresses background texture interference, focusing more accurately on the defect area. This is particularly important for small imperfections (such as micrometer-sized spots): ordinary convolutions may "average" small targets due to fixed sampling, while dynamic sampling can specifically amplify the feature signals of small targets, improving the discriminative power of underlying features.

[0033] Meanwhile, the ordinary convolution used for downsampling in the original Head may lose key features of small targets during feature dimensionality reduction due to fixed sampling, "compressing" the features of minor defects into noise. Replacing it with modulated deformable convolution allows for the preservation of core defect features through dynamic offset during downsampling, focusing the features of tiny weld points on the sampling center and ensuring that the signal of small targets is not diluted when features are transferred from high resolution to low resolution. This is particularly important for multi-scale defect detection, preventing small target features from being "covered" by large target features during dimensionality reduction.

[0034] For more details, please refer to Figure 3This is a schematic diagram of the C2fCIB module architecture. In the C2fCIB module, the input features pass through the first CBS module. The features after passing through the CBS module are split into multiple branches, and each branch is connected to an independent CIB. Each branch extracts local features of different dimensions in parallel. Then, the outputs of all CIB branches are concatenated by Concat to integrate the feature information of multiple branches. Finally, the concatenated features pass through the second CBS module, where convolutional fusion of channel features and adjustment of the number of output channels are performed to obtain the module's output features. The CBS module is a combination of convolutional layers, batch normalization, and activation functions, while CIB is a lightweight convolutional unit.

[0035] It's important to note that the original C2f module's Bottleneck uses ordinary convolution for spatial and channel blending, which is computationally expensive because the complexity of ordinary convolution is related to the square of the number of channels, and cross-channel information fusion may blur the local details of minor defects. In contrast, the CIB module in C2fCIB employs a compact structure of spatially blended depthwise convolution and channel-blended pointwise convolution. The c-grouped depthwise convolution allows each channel to independently capture spatial features, avoiding cross-channel information interference and accurately preserving the local texture of minor defects. Pointwise convolution achieves channel fusion at low cost, integrating only necessary semantic information (such as the color / brightness features of defects) and reducing redundant computation. For industrial scenarios, the computational cost of depthwise convolution is only 1 / c of ​​that of ordinary convolution, ensuring real-time performance while enhancing the ability to preserve details of small targets—for example, when detecting tiny solder defects on a circuit board, it can simultaneously preserve the edge shape and grayscale anomalies of the solder.

[0036] Furthermore, the original head upsampling uses nearest interpolation with fixed rules, which only enlarges the feature map by pixel copying or interpolation, resulting in limited ability to recover details of minor imperfections. For example, tiny scratches in the low-resolution feature map are easily blurred into "noise" after fixed upsampling, making them difficult to distinguish from the background. Dy_Sample adjusts the upsampling sampling position by learning dynamic offsets: when upsampling the low-resolution features of the P5 layer (SPPF layer) to the high-resolution P4 layer (third C2fCIB module), it can actively cluster the sampling points towards the pixel position of the scratch, rather than uniformly interpolating. This dynamism is crucial for industrial small target detection: it can selectively preserve the fine cracks on the material surface while enlarging the feature map, avoiding the "detail dilution" caused by ordinary upsampling, and providing clearer underlying features for subsequent feature fusion.

[0037] Furthermore, the feature fusion of the original YOLOv8 only integrates through simple Concat and C2f, lacking collaborative modeling of global context and local details. For example, in complex backgrounds, normal textures may be misclassified as defects due to similar local details but different global distributions. ACmix solves this problem through a hybrid mechanism of "self-attention + convolution". The self-attention branch captures global dependencies, such as whether the texture of a certain area is consistent with the surrounding normal areas, helping to distinguish between "defects" and "normal textures". The convolution branch preserves local details of defect edges, avoiding the loss of small target features caused by global modeling. At the same time, ACmix's lightweight design of grouping and depthwise convolution ensures that it does not increase the amount of computation too much, meeting the needs of industrial real-time inspection.

[0038] Furthermore, the modified version uses features enhanced by ACmix. ACmix's enhancements allow the features of the input detection head to simultaneously include the edge and texture details of the defect, as well as the differences from the surrounding environment. This enables the detection head to accurately locate small targets that depend on local details during prediction, while also relying on the global context to reduce false positives, ultimately improving the recall and accuracy for minor flaws.

[0039] Step S04: Obtain an image of the product to be inspected, input the trained improved YOLOv8 model, and output defect information.

[0040] Among them, a closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filter fusion scheme are used to respond to illumination fluctuations in real time in order to acquire images of the products to be inspected.

[0041] Specifically, the closed-loop adaptive illumination adjustment system monitors fluctuations in ambient illuminance in real time using an ambient light sensor and feeds the data back to the central processing unit. Based on a preset brightness target value, this unit uses a control algorithm to calculate compensation commands and drives a programmable LED light source controller to dynamically adjust the brightness of multiple LED light sources (such as ring lights and strip lights) integrated around the imaging system. This actively counteracts ambient light interference, ensuring a constant illumination intensity projected onto the sample surface. Furthermore, the system can be upgraded with an intelligent search algorithm. Using quality evaluation indicators such as local contrast and information entropy of the acquired image as optimization targets, it automatically searches for and locks the optimal light source combination and angle parameters that most clearly highlight defect features, ultimately providing stable, uniform, and highly targeted lighting conditions for image acquisition from the root cause.

[0042] The improved global adaptive gamma correction algorithm first maps the normalized average gray value of the image to an adaptive gamma value related to the overall brightness of the image: when the image is generally dark (low average gray value), a gamma value less than 1 is automatically generated to expand the dynamic range of dark areas and enhance the details of defects hidden in shadows; conversely, when the image is generally bright (high average gray value), a gamma value greater than 1 is generated to suppress overexposure in highlight areas and restore lost textures. Furthermore, to avoid the potential decrease in local contrast or loss of highlight / dark detail that may result from simple global correction, this improved algorithm introduces a weighting function based on image histogram characteristics or local contrast to fine-tune the gamma value. This allows for better preservation and highlighting of the contrast between small target defects and complex backgrounds on the basis of global enhancement, significantly improving the usability of the image under extreme or uneven lighting conditions.

[0043] In addition, a Gaussian filter weight matrix with standard deviation σ and kernel size k×k is used to smooth the image. The elements of the weight matrix are calculated according to the formula. Calculate and reduce image noise interference, where x and y are coordinates relative to the kernel center, with values ​​ranging from [value range missing]. Integer step size.

[0044] In practical applications, the trained and improved YOLOv8 model is deployed to an edge detection system equipped with a telecentric lens and an industrial camera. After the product to be inspected enters the inspection area, it is positioned, fixed, image acquired and preprocessed, and then input into the model to complete inference. The model outputs the defect category, location, size and confidence level, and after comparing it with the preset quality standard, it automatically completes the qualification judgment. Qualified products flow directly into the next process, while unqualified products trigger an audible and visual alarm and simultaneously record the defect information to the database.

[0045] Finally, a closed-loop optimization is formed through "detection feedback - sample update - model iteration": "Model missed / false detection defects" or "new types of defects" discovered in actual detection are manually labeled and added to the "comprehensive sample library"; based on the updated sample library, the YOLOv8 model is fine-tuned and improved to enhance the model's detection accuracy for rare and complex defects; simultaneously, based on defect statistics, the ControlNet control conditions and Prompt descriptions are optimized in reverse to generate defect samples that are more suitable for the current production scenario, continuously improving the sample library. Through the cycle of "input generation → training modeling → detection application → feedback optimization", the two technologies are dynamically coordinated to ensure that the system's detection capabilities adapt to the continuous changes in the production scenario.

[0046] In summary, the industrial product defect detection method in the above embodiments of the present invention acquires defect-free images and generates defect samples by fusing a stable diffusion model, low-rank adaptive technology, and a control network. These defect samples are then merged with real defect samples from the industrial production process to construct a comprehensive sample library, providing high-quality data support for training the detection model. An improved YOLOv8 model is constructed and trained based on the comprehensive sample library. A closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filtering fusion scheme are employed to respond to illumination fluctuations in real time. Images of the product to be inspected are acquired, and the images are input into the trained improved YOLOv8 model to output defect information. Specifically, the above method significantly improves detection accuracy, stability, and efficiency.

[0047] Example 2 Please see Figure 4 , Figure 4 This is a structural block diagram of an industrial product defect detection system 200 provided in Embodiment 2 of the present invention. The industrial product defect detection system 200 specifically includes: a data acquisition module 21, a data merging module 22, a data construction module 23, and an input module 24, wherein: Acquisition module 21 is used to acquire defect-free images and generate defect samples by fusing a stable diffusion model, low-rank adaptation technology and control network; The merging module 22 is used to merge the defect samples with real defect samples in the industrial production process to construct a comprehensive sample library; Module 23 is used to construct an improved YOLOv8 model. Based on the comprehensive sample library, the improved YOLOv8 model is trained. The architecture of the improved YOLOv8 model includes a backbone network, a neck, and a head. The backbone network includes an input layer, a convolutional layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, and an SPPF layer connected in sequence. The first feature extraction layer consists of a first modulated deformable convolutional module and a first C2fCIB module. The second feature extraction layer consists of a second modulated deformable convolutional module and a second C2fCIB module. The third feature extraction layer consists of a third modulated deformable convolutional module and a third C2fCIB module. The fourth feature extraction layer consists of a fourth modulated deformable convolutional module and a fourth C2fCIB module. The neck layer comprises, in sequence, a first Dy_Sample, a first stitching layer, a fifth C2fCIB module, a second Dy_Sample, a second stitching layer, a C2fMLLABlock, a fifth modulated deformable convolutional module, a third stitching layer, a sixth C2fCIB module, a seventh C2fCIB module, a fourth stitching layer, and an eighth C2fCIB module. The output of the second C2fCIB module is connected to the input of the second stitching layer, the output of the third C2fCIB module is connected to the input of the first stitching layer, the output of the SPPF layer is connected to the inputs of the first Dy_Sample and the fourth stitching layer, the output of the C2fMLLABlock is also connected to the first ACmix module, the output of the sixth C2fCIB module is also connected to the second ACmix module, and the output of the eighth C2fCIB module is connected to the third ACmix module. The head includes three detection heads, wherein the first ACmix module is connected to the first detection head, the second ACmix module is connected to the second detection head, and the third ACmix module is connected to the third detection head; In the C2fCIB module, the input features pass through the first CBS module. The features passed through the CBS module are split into multiple branches, and each branch is connected to an independent CIB. Each branch extracts local features of different dimensions in parallel. Then, the outputs of all CIB branches are concatenated by Concat to integrate the feature information of multiple branches. Finally, the concatenated features pass through the second CBS module, where convolutional fusion of channel features and adjustment of the number of output channels are performed to obtain the output features of the module. The CBS module is a module that combines convolutional layers, batch normalization, and activation functions, while CIB is a lightweight convolutional unit. Input module 24 is used to acquire images of the product to be inspected, input the trained improved YOLOv8 model, and output defect information. It adopts a closed-loop adaptive illumination adjustment system and an improved global adaptive gamma correction algorithm and Gaussian filtering fusion scheme to respond to illumination fluctuations in real time in order to acquire images of the product to be inspected. The closed-loop adaptive illumination adjustment system monitors the fluctuations of ambient illumination in real time through an ambient light sensor and feeds the data back to the central processing unit. The central processing unit calculates compensation instructions based on the preset brightness target value using a control algorithm, and drives the programmable LED light source controller to dynamically adjust the brightness of the multiple LED light sources integrated around the imaging system, thereby actively canceling ambient light interference and ensuring that the light intensity projected onto the sample surface is constant. The normalized average gray value of the image is mapped to an adaptive gamma value that is related to the overall brightness of the image. When the average gray value is low, a gamma value less than 1 is automatically generated to expand the dynamic range of dark areas and enhance the details of defects hidden in shadows; when the average gray value is high, a gamma value greater than 1 is generated to suppress overexposure in highlight areas and restore lost textures. In addition, a weighting function based on image histogram characteristics or local contrast is introduced to fine-tune the gamma value in order to preserve the contrast between small target defects and the background. The image is smoothed using a Gaussian filter weight matrix with standard deviation σ and kernel size k×k. The elements of the weight matrix are calculated according to the formula... Calculate and reduce image noise interference, where x and y are coordinates relative to the kernel center, with values ​​ranging from [value range missing]. Integer step size; The improved YOLOv8 model employs a combination of modulated deformable convolution and the C2fCIB module in the feature extraction stage; introduces the Dy_Sample dynamic upsampling operator in the feature fusion stage; integrates the advantages of self-attention and convolution through the ACmix module in the feature enhancement stage; and optimizes the downsampling process in the feature transfer stage.

[0048] Furthermore, in some optional embodiments of the present invention, the acquisition module 21 includes: The extraction unit is used to acquire defect-free images and extract GLCM material feature vectors, Canny edge maps, and high-precision depth maps to build a defect-free basic feature library. The training unit is used to select the fusion stable diffusion model as the basic generative model and train the fusion stable diffusion model using low-rank adaptation technology. The training data comes from the defect-free basic feature library and is supplemented by a preset number of labeled real defect samples. The injection unit is used during the training process to inject the material and structural feature parameters of the product into the low-rank adapter parameter space of the fusion stable diffusion model, ensuring that the base material properties and structural morphology of the generated samples are highly consistent with the real product, with a structural similarity of no less than 90%, and finally training a product-specific LoRA model. The collaborative unit is used to construct multimodal ControlNet control conditions based on the product's structural integrity defects, appearance defects, and size-related defects, and works in conjunction with the product's dedicated LoRA model to generate defect samples.

[0049] Example 3 In another aspect, the present invention also proposes an electronic device, please refer to [link to relevant documentation]. Figure 5The image shows an electronic device according to Embodiment 3 of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor. When the processor 10 executes the computer program 30, it implements the industrial product defect detection method described above.

[0050] In some embodiments, the processor 10 may be a central processing unit (CPU), controller, microcontroller, microprocessor or other data processing chip, used to run program code stored in memory 20 or process data, such as executing access restriction programs.

[0051] The memory 20 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 20 can be an internal storage unit of an electronic device, such as the hard disk of the electronic device. In other embodiments, the memory 20 can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 20 can include both internal and external storage units of the electronic device. The memory 20 can be used not only to store application software and various types of data of the electronic device, but also to temporarily store data that has been output or will be output.

[0052] It should be pointed out that, Figure 5 The structure shown does not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0053] This invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the industrial product defect detection method described above.

[0054] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0055] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0056] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0057] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0058] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for detecting defects in industrial products, characterized in that, The method includes: Defect-free images are acquired, and defect samples are generated by fusing a stable diffusion model, low-rank adaptation technology, and a control network. The defect samples are merged with real defect samples from the industrial production process to construct a comprehensive sample library; Construct an improved YOLOv8 model and train it based on the comprehensive sample library. The system acquires images of the products to be inspected, inputs them into a trained improved YOLOv8 model, and outputs defect information. It employs a closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filter fusion scheme to respond to illumination fluctuations in real time in order to acquire images of the products to be inspected. The improved YOLOv8 model employs a combination of modulated deformable convolution and the C2fCIB module in the feature extraction stage; introduces the Dy_Sample dynamic upsampling operator in the feature fusion stage; integrates the advantages of self-attention and convolution through the ACmix module in the feature enhancement stage; and optimizes the downsampling process in the feature transfer stage.

2. The industrial product defect detection method according to claim 1, characterized in that, The steps of acquiring defect-free images and generating defect samples by fusing a stable diffusion model, low-rank adaptation technology, and a control network include: Collect defect-free images and extract GLCM material feature vectors, Canny edge maps, and high-precision depth maps to construct a defect-free basic feature library; A fusion-stable diffusion model is selected as the basic generative model, and a low-rank adaptation technique is used to train the fusion-stable diffusion model. The training data comes from the defect-free basic feature library and is supplemented with a preset number of labeled real defect samples. During training, the material and structural feature parameters of the product are injected into the low-rank adapter parameter space of the fusion stable diffusion model to ensure that the base material properties and structural morphology of the generated samples are highly consistent with the real product, with a structural similarity of no less than 90%, and finally a product-specific LoRA model is trained. Based on the product's structural integrity defects, appearance defects, and size-related defects, a multimodal ControlNet control condition is constructed and works in conjunction with the product-specific LoRA model to generate defect samples.

3. The industrial product defect detection method according to claim 2, characterized in that, In the step of acquiring images of the product under inspection by adopting a closed-loop adaptive illumination adjustment system and an improved global adaptive gamma correction algorithm and Gaussian filtering fusion scheme to respond to illumination fluctuations in real time, the closed-loop adaptive illumination adjustment system monitors the fluctuations of ambient illumination in real time through an ambient light sensor and feeds the data back to the central processing unit. The central processing unit calculates compensation instructions based on the preset brightness target value using a control algorithm, and drives the programmable LED light source controller to dynamically adjust the brightness of the multiple LED light sources integrated around the imaging system, thereby actively canceling ambient light interference and ensuring that the light intensity projected onto the sample surface is constant.

4. The industrial product defect detection method according to claim 3, characterized in that, In the step of acquiring images of the product to be inspected, a closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filter fusion scheme are adopted to respond to illumination fluctuations in real time. In this step, the normalized average gray value of the image is mapped to an adaptive gamma value related to the overall brightness of the image. Specifically, when the average gray value is low, a gamma value less than 1 is automatically generated to expand the dynamic range of dark areas and enhance the details of defects hidden in shadows; when the average gray value is high, a gamma value greater than 1 is generated to suppress overexposure in highlight areas and restore lost textures. In addition, a weighting function based on image histogram characteristics or local contrast is introduced to fine-tune the gamma value to preserve the contrast between small target defects and the background.

5. The industrial product defect detection method according to claim 4, characterized in that, In the step of acquiring images of the product to be inspected, a closed-loop adaptive illumination adjustment system combined with an improved global adaptive gamma correction algorithm and a Gaussian filter fusion scheme is used to respond to illumination fluctuations in real time. The image is smoothed using a Gaussian filter weight matrix with a standard deviation σ and a kernel size of k×k. The weight matrix elements are calculated according to the formula... Calculate and reduce image noise interference, where x and y are coordinates relative to the kernel center, with values ​​ranging from [value range missing]. Integer step size.

6. The method for detecting defects in industrial products according to claim 5, characterized in that, The architecture of the improved YOLOv8 model includes a backbone network, a neck, and a head. The backbone network includes an input layer, a convolutional layer, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, and an SPPF layer connected in sequence. The first feature extraction layer consists of a first modulated deformable convolutional module and a first C2fCIB module. The second feature extraction layer consists of a second modulated deformable convolutional module and a second C2fCIB module. The third feature extraction layer consists of a third modulated deformable convolutional module and a third C2fCIB module. The fourth feature extraction layer consists of a fourth modulated deformable convolutional module and a fourth C2fCIB module. The neck layer comprises, in sequence, a first Dy_Sample, a first stitching layer, a fifth C2fCIB module, a second Dy_Sample, a second stitching layer, a C2fMLLABlock, a fifth modulated deformable convolutional module, a third stitching layer, a sixth C2fCIB module, a seventh C2fCIB module, a fourth stitching layer, and an eighth C2fCIB module. The output of the second C2fCIB module is connected to the input of the second stitching layer, the output of the third C2fCIB module is connected to the input of the first stitching layer, the output of the SPPF layer is connected to the inputs of the first Dy_Sample and the fourth stitching layer, the output of the C2fMLLABlock is also connected to the first ACmix module, the output of the sixth C2fCIB module is also connected to the second ACmix module, and the output of the eighth C2fCIB module is connected to the third ACmix module. The head includes three detection heads, wherein the first ACmix module is connected to the first detection head, the second ACmix module is connected to the second detection head, and the third ACmix module is connected to the third detection head.

7. The method for detecting defects in industrial products according to claim 6, characterized in that, In the C2fCIB module, the input features pass through the first CBS module. The features passed through the CBS module are split into multiple branches, each branch is connected to an independent CIB, and each branch extracts local features of different dimensions in parallel. Then, the outputs of all CIB branches are concatenated by Concat to integrate the feature information of multiple branches. Finally, the concatenated features pass through the second CBS module, where convolutional fusion of channel features and adjustment of the number of output channels are performed to obtain the output features of the module. The CBS module is a combination of convolutional layers, batch normalization, and activation functions, while CIB is a lightweight convolutional unit.

8. An industrial product defect detection system, characterized in that, The system for implementing the industrial product defect detection method as described in any one of claims 1-7 includes: The acquisition module is used to acquire defect-free images and generate defect samples by fusing a stable diffusion model, low-rank adaptation technology and a control network. The merging module is used to merge the defect samples with real defect samples from the industrial production process to build a comprehensive sample library. The building module is used to build an improved YOLOv8 model and train the improved YOLOv8 model based on the comprehensive sample library. The input module is used to acquire images of the product to be inspected. It inputs a trained improved YOLOv8 model and outputs defect information. It employs a closed-loop adaptive illumination adjustment system, an improved global adaptive gamma correction algorithm, and a Gaussian filter fusion scheme to respond to illumination fluctuations in real time in order to acquire images of the product to be inspected. The improved YOLOv8 model employs a combination of modulated deformable convolution and the C2fCIB module in the feature extraction stage; introduces the Dy_Sample dynamic upsampling operator in the feature fusion stage; integrates the advantages of self-attention and convolution through the ACmix module in the feature enhancement stage; and optimizes the downsampling process in the feature transfer stage.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the industrial product defect detection method as described in any one of claims 1-7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the industrial product defect detection method as described in any one of claims 1-7.

Images