AI-based large-scale model-based method for detecting surface defects in industrial products

Abstract

This application relates to the field of industrial inspection technology and discloses a method for detecting surface defects in industrial products based on an AI large-scale model. The method includes the following steps: S1, preprocessing the surface image of the industrial product to generate illumination-invariant features and multi-scale features; S2, using the AI ​​large-scale model to extract deep features from the preprocessed image and generating a multi-level defect feature map through multi-scale feature fusion; S3, optimizing the AI ​​large-scale model using knowledge distillation technology to generate a lightweight model; S4, deploying the lightweight model to an edge device for surface defect detection; and S5, optimizing the confidence level determination during the detection process by dynamically adjusting the detection threshold. By combining illumination-invariant feature extraction and multi-scale feature decomposition, the method achieves the effect of extracting detailed surface features under complex lighting conditions, solves the problem of detection instability caused by changes in lighting, and improves the robustness of the detection.

Description

Technical Field

[0001] This invention relates to the field of industrial inspection technology, specifically to a method for detecting surface defects in industrial products based on an AI large model. Background Technology

[0002] Surface defect inspection of industrial products is a crucial step in ensuring product quality and production efficiency in modern manufacturing. Existing technologies mainly include manual visual inspection and automated inspection methods based on machine vision. Manual visual inspection relies on worker experience and sensitivity; while still used in some small-scale production scenarios, its efficiency and consistency are low when facing large-scale assembly line operations. Automated inspection technologies based on machine vision extract and analyze features from industrial surface images using rule-based image processing algorithms, improving automation while reducing the shortcomings of human intervention. These technologies show good applicability in scenarios with stable lighting conditions and relatively simple surface textures and defect features, and improve inspection efficiency to some extent. However, with the increasing complexity and diversity of industrial production environments, existing technologies are gradually facing limitations in their adaptability.

[0003] Existing technologies still have some shortcomings in practical applications. Traditional rule-based machine vision methods often struggle to handle complex textures and diverse defect features, especially in industrial environments with varying lighting conditions and diverse materials, where their accuracy and robustness are significantly limited. While deep learning-based defect detection technologies have made significant progress in automation and detection accuracy, the generalization ability of existing methods is relatively limited. Models are typically optimized for specific defect types or scenarios, and their detection performance tends to degrade when applied to scenarios with changes in material, lighting, or defect type. Furthermore, deep learning methods require large-scale labeled data, while defect data in industrial settings is often unevenly distributed and costly to collect and label, increasing the difficulty of model optimization. Simultaneously, deep learning models typically require significant computing resources, limiting their large-scale deployment on resource-constrained edge devices and making it difficult to meet the real-time detection needs of industrial environments. These issues restrict the widespread application of existing technologies in complex industrial environments. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an AI-based large model-based method for detecting surface defects in industrial products. This method solves the problems of insufficient robustness of existing industrial product surface defect detection technologies under complex lighting conditions, inadequate extraction of diverse defect features, limited model generalization ability, and low real-time detection efficiency of edge devices.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting surface defects in industrial products based on an AI large-scale model, comprising the following steps:

[0006] S1. Perform data preprocessing on the surface images of industrial products to generate illumination-invariant features and multi-scale features;

[0007] S2. Use a large AI model to extract deep features from the preprocessed image and generate a multi-level defect feature map through multi-scale feature fusion.

[0008] S3. Optimize large AI models using knowledge distillation technology to generate lightweight models;

[0009] S4. Deploy the lightweight model to edge devices for surface defect detection;

[0010] S5. Optimize the confidence determination during the detection process by dynamically adjusting the detection threshold.

[0011] Preferably, in step S1, the extraction of illumination-invariant features during data preprocessing includes:

[0012] The pixel gradient of the input image is calculated to obtain the gradient direction and magnitude of the pixel value. The pixel gradient is then normalized to eliminate the influence of illumination intensity and generate an illumination-invariant feature map.

[0013] Preferably, in step S1, the multi-scale feature decomposition of data preprocessing includes:

[0014] High-frequency and low-frequency features are extracted from the input image through multi-scale decomposition, and the high-frequency and low-frequency features are weighted and fused according to the set weight parameters to generate a fused feature map containing multi-scale information.

[0015] Preferably, in step S2, deep feature extraction is performed using a multi-head self-attention mechanism based on the Transformer architecture, specifically including the following steps:

[0016] a. Generate the query matrix, key matrix, and value matrix;

[0017] b. Generate attention weights through calculations between the query matrix and the key matrix;

[0018] c. Use attention weights to perform a weighted summation of the value matrix to generate deep features;

[0019] d. Extract global and local features using a multi-head attention mechanism.

[0020] Preferably, in step S2, multi-scale feature fusion is achieved through a feature pyramid network, which combines feature maps of different scales into a unified multi-scale feature after weight optimization.

[0021] Preferably, in step S3, the knowledge distillation technique includes the following steps:

[0022] a. Use the teacher model to generate deep features from the input data;

[0023] b. Utilize the student model to learn the feature outputs of the teacher model;

[0024] c. Optimize the student model by minimizing the cross-entropy loss between the true label and the student model output, as well as the difference loss between the teacher model and the student model output.

[0025] Preferably, in step S3, the lightweight model is optimized through model quantization. The model quantization involves discretizing the model's weights, converting the weights from floating-point representation to quantized representation, and then using the quantized weights for matrix operations.

[0026] Preferably, in step S4, the lightweight model in the edge device is deployed using a deep learning inference acceleration tool, which is used to optimize the model's inference speed and computational efficiency.

[0027] Preferably, in step S5, dynamically adjusting the detection threshold includes the following steps:

[0028] a. Statistically analyze the confidence distribution of defect detection results, and calculate the mean and standard deviation of the confidence scores;

[0029] b. Based on the set adjustment parameters, the detection threshold is dynamically adjusted using the mean and standard deviation.

[0030] Preferably, in step S2, the weights for multi-scale feature fusion are set by optimizing the response levels of high-frequency and low-frequency features, and are used to perform weighted synthesis of local detail defects and global features.

[0031] This invention provides a method for detecting surface defects in industrial products based on a large AI model. It offers the following advantages:

[0032] 1. This invention adopts a technical solution that combines illumination-invariant feature extraction and multi-scale feature decomposition, which achieves the effect of extracting surface detail features under complex illumination conditions. Compared with the detection methods in the prior art that rely on fixed illumination conditions or global features, it solves the problem of detection instability caused by illumination changes and improves the robustness of detection.

[0033] 2. This invention uses a large Transformer-based model for deep feature extraction and combines it with a multi-scale feature fusion network to achieve feature representation capabilities that take into account both global information and local details. Compared with existing single-scale or single-network architecture schemes, this invention effectively solves the problem of insufficient extraction of diverse defect features and adapts to the complex industrial surface defect detection needs.

[0034] 3. This invention utilizes knowledge distillation and model quantization techniques to optimize large models into lightweight models, achieving efficient operation on edge devices. Compared to existing solutions that rely on high-performance hardware, this invention solves the problems of limited resources and high computing costs in industrial settings, enabling the deployment and application of lightweight models.

[0035] 4. This invention achieves real-time optimization of the false negative rate and false positive rate by dynamically adjusting the detection threshold. Compared with the existing technology that uses a fixed threshold method to detect defects, it effectively solves the problem of unstable detection accuracy under diverse defect types and production scenarios, and improves the reliability and adaptability of detection in industrial environments. Attached Figure Description

[0036] Figure 1 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Please see the appendix Figure 1 This invention provides a method for detecting surface defects in industrial products based on an AI large model, comprising the following steps:

[0039] S1. Perform data preprocessing on the surface images of industrial products to generate illumination-invariant features and multi-scale features;

[0040] S2. Use a large AI model to extract deep features from the preprocessed image and generate a multi-level defect feature map through multi-scale feature fusion.

[0041] S3. Optimize large AI models using knowledge distillation technology to generate lightweight models;

[0042] S4. Deploy the lightweight model to edge devices for surface defect detection;

[0043] S5. Optimize the confidence determination during the detection process by dynamically adjusting the detection threshold.

[0044] Specifically, by combining large AI models, data preprocessing, model optimization, and dynamic adjustment mechanisms, efficient detection of surface defects in industrial products can be achieved. First, the system preprocesses the input industrial product surface image to generate illumination-invariant and multi-scale features. This step eliminates the influence of illumination intensity and separates local details from global background features, providing more robust input features for subsequent models. Next, after deep feature extraction by the AI ​​large model, the system uses a multi-head self-attention mechanism to comprehensively model the global and local information of the image. At the same time, through a multi-scale feature fusion network, it further integrates information at different levels and scales to generate multi-level feature maps suitable for defect detection. Then, to improve the operating efficiency in industrial scenarios, the system uses knowledge distillation technology to transfer the key knowledge of the large model to a lightweight model, and combines model quantization technology to further reduce storage and computing requirements, making the model adaptable to edge devices. Finally, in the actual detection process, the system adopts a dynamic adjustment mechanism to adaptively optimize the detection threshold based on the real-time confidence distribution to balance the false negative rate and the false positive rate, ensuring stable detection capabilities in complex industrial environments. Each step is interconnected, forming a complete and efficient defect detection process from feature input to optimization deployment to real-time detection.

[0045] In step S1, illumination-invariant feature extraction during data preprocessing includes:

[0046] The pixel gradient of the input image is calculated to obtain the gradient direction and magnitude of the pixel value, and the pixel gradient is normalized to eliminate the influence of illumination intensity and generate an illumination-invariant feature map.

[0047] In step S1, the multi-scale feature decomposition of data preprocessing includes:

[0048] High-frequency and low-frequency features are extracted from the input image through multi-scale decomposition, and the high-frequency and low-frequency features are weighted and fused according to the set weight parameters to generate a fused feature map containing multi-scale information.

[0049] Specifically, step S1 primarily involves preprocessing images of industrial product surfaces to extract illumination-invariant and multi-scale features, providing more robust input for subsequent deep feature extraction in large-scale models. In industrial environments, product surfaces may exhibit significant differences in brightness or shadow due to varying lighting conditions, and traditional image processing methods struggle to eliminate the impact of these lighting variations. Furthermore, surface defects vary considerably in type, shape, and size, and single-scale feature extraction methods cannot adequately represent this diversity. Through illumination-invariant feature extraction and multi-scale feature decomposition, this invention effectively reduces the interference of lighting variations, enhances feature expressiveness, and provides high-quality input features for deep learning models.

[0050] Generally, illumination-invariant feature extraction processes the gradient changes of pixels in an image to eliminate the influence of illumination intensity. Alternatively, multi-scale feature decomposition captures local details and global background information by separating high-frequency and low-frequency features of the image. In some embodiments, the present invention can further optimize the weighting of high-frequency and low-frequency features according to defect type and image characteristics, thereby improving the ability to represent different types of defects.

[0051] In this embodiment, for the input image of the industrial product surface, illumination-invariant feature extraction is first performed. Specifically:

[0052] In one possible implementation, the input image For each pixel, its gradient value is calculated to reflect the intensity change within its neighborhood. The formula for calculating the gradient value is:

[0053] ;

[0054] in:

[0055] Indicates the coordinates in the image are Pixel values;

[0056] This represents the intensity change of a pixel in the horizontal direction, i.e., along... Partial derivatives in direction;

[0057] This represents the intensity change of a pixel in the vertical direction, i.e., along... Partial derivatives in direction.

[0058] As an alternative, to eliminate the influence of changes in light intensity on the gradient, the calculated gradient is normalized. The specific formula for normalization is:

[0059] ;

[0060] in:

[0061] This indicates the normalized gradient direction;

[0062] The magnitude of the gradient is represented by the following formula:

[0063] ;

[0064] The purpose of normalization is to ensure that illumination-invariant features primarily depend on the direction of the gradient, rather than being affected by changes in illumination intensity. In some embodiments, the illumination-invariant feature map can be further transformed using a function mapping method. For example, an angle function can be used to map the gradient direction to pixel values ​​to generate an illumination-invariant feature map.

[0065] ;

[0066] The illumination-invariant feature map obtained through the above processing It can effectively capture surface detail features and eliminate the interference of illumination on the detection results. In another implementation, the original gradient information and the normalized gradient information can be directly combined to further improve the performance of the feature map.

[0067] In this embodiment, to address the issue of diverse defect types, a multi-scale feature decomposition technique is employed to process the input image. Specifically:

[0068] In one possible implementation, the illumination-invariant feature map is decomposed into multiple scales using discrete wavelet transform. The input image is decomposed into two parts: high-frequency features and low-frequency features. High-frequency features are used to describe surface details, such as cracks and scratches; low-frequency features retain global background information of the image, such as large-scale surface undulations or depressions.

[0069] Generally, the formula for calculating the discrete wavelet transform is:

[0070] ;

[0071] in:

[0072] Indicates the image at scale ,Location The wavelet component below;

[0073] Represents an illumination-invariant feature map;

[0074] Describe the wavelet basis functions. Its conjugate complex number;

[0075] This represents the scale parameter, which controls the frequency range of the decomposition.

[0076] This represents the translation parameter, which controls the position of the decomposition.

[0077] In one possible implementation, high-frequency features and low frequency characteristics The results were calculated using different wavelet components. Specifically, the high-frequency components retain detailed information, while the low-frequency components retain a smooth background.

[0078] In other embodiments, high-frequency and low-frequency features can be further processed by weighted fusion. The fusion formula is:

[0079] ;

[0080] in:

[0081] This is the fused multi-scale feature map;

[0082] The weighting parameter for high-frequency features has a value range of 0 ≤ ≤1, the specific value can be set according to actual needs.

[0083] As an alternative, in some possible implementations, the weight parameters The weighting can be dynamically adjusted based on the response level of different types of defects. For example, for detecting small scratches, the weighting of high-frequency features can be appropriately increased; while for detecting large-area dents or deformations, the weighting of low-frequency features can be increased.

[0084] The multi-scale feature map generated through the above multi-scale decomposition process It can effectively express the global and local information of surface defects, providing high-quality input for subsequent deep feature extraction of large models.

[0085] In step S2, deep feature extraction is accomplished through a multi-head self-attention mechanism based on the Transformer architecture, specifically including the following steps:

[0086] a. Generate the query matrix, key matrix, and value matrix;

[0087] b. Generate attention weights through calculations between the query matrix and the key matrix;

[0088] c. Use attention weights to perform a weighted summation of the value matrix to generate deep features;

[0089] d. Extract global and local features using a multi-head attention mechanism;

[0090] In step S2, multi-scale feature fusion is achieved through a feature pyramid network, which combines feature maps of different scales into a unified multi-scale feature after weight optimization.

[0091] In step S2, the weights for multi-scale feature fusion are set by optimizing the response levels of high-frequency and low-frequency features, and are used to perform weighted synthesis of local detail defects and global features.

[0092] Specifically, step S2, based on the illumination-invariant feature maps and multi-scale feature maps generated by data preprocessing, utilizes a large AI model to perform deep feature extraction and multi-scale feature fusion on these preprocessed features. The design focus of step S2 is to extract deep semantic features from the input data through a large model with a Transformer architecture, and to combine features at different levels and scales through a multi-scale feature fusion network to generate a multi-level defect feature map containing both global information and local details.

[0093] Generally, surface defects in industrial products include scratches, cracks, and dents. These defects are diverse in morphology, and defects of different sizes and scales require different feature extraction methods for accurate representation. Therefore, using large AI models for deep feature extraction can better capture the complex patterns of surface defects. Simultaneously, combining multi-scale feature fusion can fully utilize features at different scales, improving the overall feature representation capability of the model. As an option, this step employs a Transformer-based multi-head self-attention mechanism for deep feature extraction and uses a Feature Pyramid Network (FPN) for multi-scale feature fusion.

[0094] In this embodiment, deep feature extraction is performed on the preprocessed feature map generated in step S1 using a large model based on the Transformer architecture.

[0095] Specifically, in one possible implementation, the input feature map First, the query matrix is ​​generated through a linear transformation. Key matrix Sum matrix The calculation formula is as follows:

[0096] ;

[0097] in:

[0098] , , These are the query matrix, key matrix, and value matrix, respectively.

[0099] Input feature map;

[0100] , , The weight parameters for the query, key, and value matrices.

[0101] In general, query matrix Bond matrix The dot product measures the correlation between different locations in the input feature map. By normalizing the dot product result, the attention weight matrix can be obtained. The formula for calculating attention weights is:

[0102] ;

[0103] in:

[0104] This is the attention weight matrix;

[0105] is the dimension of the key matrix, used to normalize the dot product result;

[0106] This is the normalization function.

[0107] As an alternative, attention weight matrix can be used. Log-value matrix Perform a weighted summation to generate the output feature matrix:

[0108] ;

[0109] in, This represents the extracted deep features.

[0110] In some embodiments, to further enhance feature extraction capabilities, a multi-head self-attention mechanism is employed to model the input feature map from multiple perspectives. Specifically, the query matrix, key matrix, and value matrix are divided into multiple subspaces. In each subspace, attention weights and weighted summations are calculated, and the results from all subspaces are concatenated to form the multi-head attention output.

[0111] ;

[0112] in:

[0113] and They represent the first Attention weights and value matrices for each subspace;

[0114] Indicates the number of long positions;

[0115] The weights are the linear transformation weights after concatenation.

[0116] Through a multi-head self-attention mechanism, this embodiment can simultaneously capture global information and local detail information in the input feature map.

[0117] In this embodiment, a feature pyramid network (FPN) is used to achieve multi-scale feature fusion for the multi-scale features generated in step S1 and the features obtained through deep feature extraction.

[0118] In one possible implementation, multi-scale feature maps are extracted from different layers of the Transformer model. , ,…, Each feature map corresponds to a different semantic level and scale. To achieve multi-scale fusion, these feature maps are upsampled and weighted layer by layer. The fusion formula is as follows:

[0119] ;

[0120] in:

[0121] This is the fused multi-scale feature map;

[0122] For the first Feature map of the layer;

[0123] For the first Weight parameters of the layer feature map.

[0124] In general, weight parameters The settings can be adjusted according to the importance of features at different levels. In some embodiments, the weight parameters can be automatically determined through backpropagation optimization during the training process.

[0125] As an alternative, to further improve the effect of multi-scale feature fusion, different weight parameters can be set for high-frequency features and low-frequency features respectively. Specifically:

[0126] ;

[0127] in:

[0128] Indicates high-frequency characteristics;

[0129] Indicates low-frequency characteristics;

[0130] The weighting parameter for high-frequency features has a value range of 0 ≤ ≤1.

[0131] Through the multi-scale feature fusion method described above, this embodiment can effectively combine features at different levels and scales to generate a multi-level defect feature map. It has stronger characterization capabilities.

[0132] In step S3, the knowledge distillation technique includes the following steps:

[0133] a. Use the teacher model to generate deep features from the input data;

[0134] b. Utilize the student model to learn the feature outputs of the teacher model;

[0135] c. Optimize the student model by minimizing the cross-entropy loss between the true label and the student model output, as well as the difference loss between the teacher model and the student model output;

[0136] In step S3, the lightweight model is optimized through model quantization. Model quantization involves discretizing the model's weights, converting the weights from floating-point representation to quantized representation, and then using the quantized weights for matrix operations.

[0137] Specifically, the core objective of step S3 is to optimize the generated large model by using knowledge distillation to transfer knowledge from the large model to a lightweight model. This significantly reduces computational complexity and model size while maintaining high detection accuracy. The lightweight optimized model can run more efficiently on resource-constrained edge devices, laying the foundation for real-time surface defect detection in subsequent steps.

[0138] Generally, the high performance of deep learning models often relies on complex structures and a large number of parameters. However, these models have high computational resource requirements, making them difficult to deploy directly on edge devices in industrial scenarios. As an alternative, knowledge distillation techniques can be used to extract key features and knowledge from complex large models (i.e., teacher models) and transfer them to simpler small models (i.e., student models), achieving model lightweighting while maintaining performance as much as possible. In some embodiments, to further reduce model complexity, this invention also incorporates model quantization techniques, converting the model's floating-point weights to fixed-point weights to reduce storage requirements and computational overhead.

[0139] In this embodiment, knowledge distillation is used to optimize a large model into a lightweight model. Specifically, the teacher model is used to generate deep features or predict outputs, and the student model is trained by imitating the behavior of the teacher model, thereby achieving the goal of simplifying the model.

[0140] In one possible implementation, the knowledge distillation process includes the following steps:

[0141] First, the teacher model is used to generate deep features or predicted outputs from the input data, serving as soft labels. Generally, the teacher model's predicted output not only includes classification results but also retains correlation information between different categories. This correlation information is crucial for model optimization.

[0142] Specifically, the predicted probability distribution of the teacher model It is usually calculated using the following formula:

[0143] ;

[0144] in:

[0145] Indicates the first The predicted probability of a class;

[0146] The output of the teacher model represents the first... The original score of the class;

[0147] This represents a temperature parameter used to smooth the predicted distribution.

[0148] As an alternative, in order for the student model to learn the probability distribution of the teacher model's output, this is achieved by minimizing the teacher model's output distribution. Student model output distribution The KL divergence between the two is optimized. Its loss function is expressed as:

[0149] ;

[0150] in:

[0151] The student model represents the first Class prediction probability;

[0152] This represents the difference loss between the teacher model and the student model.

[0153] In addition, the training of the student model also incorporates the cross-entropy loss between the real labels and the model predictions. The final loss function is defined as:

[0154] ;

[0155] in:

[0156] The weighting factor for the loss;

[0157] This represents the total loss from knowledge distillation.

[0158] In some embodiments, to further enhance the learning ability of the student model, both real labels and soft labels can be input simultaneously during training, so that the student model can learn the knowledge of the teacher model more comprehensively.

[0159] In this embodiment, model quantization technology is applied to the student model in order to further reduce the computational complexity and storage requirements of the lightweight model.

[0160] In general, the weight parameters of deep learning models are represented by floating-point numbers, but floating-point operations are resource-intensive. As an alternative, quantizing floating-point weights into low-precision fixed-point numbers can effectively reduce computational complexity and storage requirements.

[0161] In one possible implementation, model quantization is achieved through the following formula:

[0162] ;

[0163] in:

[0164] This represents the quantized weight value;

[0165] Represents the original floating-point weights;

[0166] The quantization step size is represented by the following formula:

[0167] ;

[0168] in, and These are the maximum and minimum values ​​of the weights, respectively. This is for quantization bit width.

[0169] As one possible implementation, the quantized weights and activation values ​​are stored and calculated using integers, thereby reducing the overhead of floating-point calculations.

[0170] In some embodiments, post-quantization training methods can be combined to fine-tune the model after quantization in order to reduce the accuracy loss introduced by quantization.

[0171] In step S4, the lightweight model in the edge device is deployed using a deep learning inference acceleration tool, which is used to optimize the model's inference speed and computational efficiency.

[0172] Specifically, step S4 involves deploying the optimized lightweight model to industrial edge devices to achieve real-time detection of surface defects in industrial products. This step primarily addresses the challenges of limited computing resources and high real-time requirements in industrial environments. By combining deep learning inference acceleration tools (such as TensorRT), this invention can significantly improve the model's inference speed while maintaining detection performance, thereby meeting the real-time processing requirements of pipeline environments.

[0173] Typically, deep learning models running on edge devices require adaptation and optimization for hardware architectures such as GPUs, TPUs, or FPGAs. As an alternative, this invention reduces the storage requirements and computational complexity of lightweight models to a level acceptable for edge devices through model quantization and structural optimization techniques, and further optimizes the model's inference process using inference acceleration tools. In some embodiments, the computation graph structure can also be adjusted for specific hardware platforms to further improve the model's running efficiency.

[0174] In this embodiment, the lightweight model generated in step S3 is deployed to an edge device to meet the application needs of industrial sites.

[0175] Specifically, in one possible implementation, the lightweight model is first transformed and optimized using a deep learning inference acceleration tool such as TensorRT. TensorRT is a high-performance optimization tool for deep learning inference that can rearrange the computation graph of the model and apply a variety of optimization strategies, including layer fusion, accuracy calibration, and dynamic memory management.

[0176] Generally, the first step in model optimization is to convert the lightweight model into an intermediate representation (IR). In TensorRT, this process can be achieved as follows:

[0177] ;

[0178] in:

[0179] This represents the original lightweight model;

[0180] This represents the intermediate representation after conversion.

[0181] As an alternative, layer fusion optimization is further applied to the computation graph of the model based on the intermediate representation. The main purpose of layer fusion is to combine multiple consecutive operations (such as convolution, activation functions, and batch normalization) into a single operation to reduce memory accesses and computational overhead.

[0182] Specifically, assume the original computation graph contains two consecutive operations. and After layer fusion optimization, these operations can be merged into a single operation:

[0183] ;

[0184] Furthermore, in some embodiments, to further improve inference speed, the computational precision of the model can be calibrated. Typically, deep learning models use 32-bit floating-point numbers for computation, but in practical applications, the precision can be reduced to 16-bit floating-point numbers or 8-bit integers. Precision calibration not only significantly reduces computational overhead but also maintains high detection performance in most scenarios.

[0185] In this embodiment, the optimized lightweight model performs real-time surface defect detection tasks on an edge device.

[0186] Specifically, in one possible implementation, the real-time detection process includes the following steps:

[0187] First, surface images collected from the industrial production line are input into the model for inference in batch processing. Assume the input images have a batch size of... tensor ,in:

[0188] Indicates the batch size of the images;

[0189] This refers to the number of channels in an image, such as in an RGB image. and These represent the height and width of the image, respectively.

[0190] The model outputs the defect detection results. ,in This indicates the number of categories or defect types predicted by the model.

[0191] Alternatively, to improve real-time performance, unnecessary steps in the inference process (such as storing intermediate feature maps) can be pruned, thereby reducing memory usage and latency. In some embodiments, the batch size can also be dynamically adjusted based on the pipeline speed. To balance inference speed and throughput.

[0192] Furthermore, to ensure the accuracy of the detection results, this embodiment incorporates a confidence level calculation module during the inference process. Generally, the formula for calculating the confidence level is:

[0193] ;

[0194] in:

[0195] Indicates the first The highest confidence level of each sample;

[0196] Indicates the first The sample belongs to the first The probability of a class.

[0197] Generally, edge devices have limited computing resources due to their hardware architecture (such as NVIDIA Jetson, Google TPU, or other embedded platforms). Therefore, model deployment requires adaptation and optimization based on hardware characteristics.

[0198] Alternatively, when running on edge devices with GPU architectures, TensorRT's CUDA kernels can be used to accelerate convolution operations. The optimization formula for convolution operations is:

[0199] ;

[0200] in:

[0201] This is the output of the convolution operation;

[0202] Input feature map;

[0203] These are the kernel weights;

[0204] For bias.

[0205] In some embodiments, the parallel computing capabilities of the FPGA can also be utilized to improve the throughput of the model by adjusting the structure of the data flow graph.

[0206] In step S5, dynamically adjusting the detection threshold includes the following steps:

[0207] a. Statistically analyze the confidence distribution of defect detection results, and calculate the mean and standard deviation of the confidence scores;

[0208] b. Based on the set adjustment parameters, the detection threshold is dynamically adjusted using the mean and standard deviation.

[0209] Specifically, step S5 aims to optimize the surface defect detection process in step S4 by dynamically adjusting the detection threshold to improve the model's adaptability and stability in industrial applications. Surface defect detection in industrial environments typically requires a balance between false negative and false positive rates, and a single fixed threshold may not be sufficient to meet the complex needs under different industrial conditions. This invention employs a dynamic threshold adjustment mechanism to adaptively adjust the threshold based on the confidence distribution of the detection results, thereby ensuring the reliability of the detection results.

[0210] Generally, dynamic threshold adjustment is optimized based on the confidence distribution of the model output. Alternatively, this invention dynamically calculates the detection threshold by statistically analyzing the mean and standard deviation of the confidence scores, combined with adjustable parameters, enabling the system to adapt to defect characteristics in different scenarios. In some embodiments, the threshold adjustment mechanism can be further optimized by incorporating the global characteristics of batch detection data, thereby enhancing the system's robustness.

[0211] In this embodiment, the process of dynamically adjusting the detection threshold includes two main steps: confidence statistics and threshold calculation.

[0212] In one possible implementation, the model's output on a batch of detection samples is first obtained, with the model outputting the probability distribution of its class for each sample. For each sample, the system extracts the highest confidence value from the output probability distribution as the detection confidence for that sample.

[0213] To calculate the distribution characteristics of the overall detection results, the mean and fluctuation range of the confidence scores for all samples in the current batch can be statistically analyzed. For example, by statistically analyzing the average confidence score of the samples, the overall reliability level of the model in the current batch can be reflected, while by statistically analyzing the dispersion index of the confidence score, the degree of fluctuation in the model's output results can be quantified.

[0214] Specifically, the dynamic threshold adjustment is based on the distribution characteristics of the confidence level. The dynamically adjusted detection threshold can be calculated by combining the center value and fluctuation range of the confidence level. Alternatively, an adjustable parameter can be introduced to control the offset of the detection threshold relative to the confidence level distribution, thereby adapting to the detection needs in different scenarios.

[0215] In some embodiments, to further improve the stability of threshold adjustment, the detection threshold can be smoothed by combining statistical information from the current batch and historical batches. For example, a weighted approach can be used to combine the statistical results of the current batch with historical data to reduce the impact of outlier data from a single batch on the detection threshold.

[0216] In this embodiment, the dynamic adjustment mechanism can be optimized to adapt to the detection needs in different industrial scenarios.

[0217] In one possible implementation, when industrial production lines are fast and real-time requirements are high, a larger batch size can be chosen to ensure the stability of statistical results. Conversely, when there are many categories of defects to be detected and the confidence distribution is complex, a smaller batch size can be chosen to reduce the impact of batch differences on threshold calculation.

[0218] Specifically, the choice of batch size is closely related to the production scenario. For example, in high-speed pipeline inspection, hundreds of images may need to be processed per second. In this case, a larger sample batch can be selected to ensure that the dynamic adjustment of the threshold can fully reflect the global characteristics.

[0219] In some embodiments, the dynamic adjustment mechanism of the present invention can be applied to various industrial scenarios. For example, for mass-produced assembly line products, when the confidence level of the detection results is concentrated in a high range, the threshold can be appropriately increased to reduce false detections; while when the confidence level of the detection results is more dispersed, the threshold can be decreased to reduce false negatives.

[0220] Specifically, when the dynamically adjusted threshold is higher than the system's base threshold, the number of detection results may decrease, but the reliability of the detection results is higher; while when the dynamically adjusted threshold is lower than the base threshold, the system can capture more potential defects, but may need to further process false detection results.

[0221] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting surface defects in industrial products based on an AI large-scale model, characterized in that, Includes the following steps: S1. Perform data preprocessing on the surface images of industrial products to generate illumination-invariant features and multi-scale features; In step S1, the extraction of illumination-invariant features during data preprocessing includes: The pixel gradient of the input image is calculated to obtain the gradient direction and magnitude of the pixel value, and the pixel gradient is normalized to eliminate the influence of illumination intensity and generate an illumination-invariant feature map. The multi-scale feature decomposition of the data preprocessing includes: High-frequency and low-frequency features are extracted from the illumination-invariant feature map by multi-scale decomposition, and the high-frequency and low-frequency features are weighted and fused according to the set weight parameters to generate a fused feature map containing multi-scale information. The weighting parameters can be dynamically adjusted according to the response level of different types of defects; S2. Use a large AI model to extract deep features from the preprocessed image and generate a multi-level defect feature map through multi-scale feature fusion. In step S2, deep feature extraction is accomplished through a multi-head self-attention mechanism based on the Transformer architecture, specifically including the following steps: a. Generate the query matrix, key matrix, and value matrix; b. Generate attention weights through calculations between the query matrix and the key matrix; c. Use attention weights to perform a weighted summation of the value matrix to generate deep features; d. Extract global and local features using a multi-head attention mechanism; In step S2, multi-scale feature fusion is achieved through a feature pyramid network, which combines feature maps of different scales into a unified multi-scale feature after weight optimization. S3. Optimize large AI models using knowledge distillation technology to generate lightweight models; S4. Deploy the lightweight model to edge devices for surface defect detection; S5. Optimize the confidence determination during the detection process by dynamically adjusting the detection threshold.

2. The method for detecting surface defects in industrial products based on an AI large model according to claim 1, characterized in that, In step S3, the knowledge distillation technique includes the following steps: a. Use the teacher model to generate deep features from the input data; b. Utilize the student model to learn the feature outputs of the teacher model; c. Optimize the student model by minimizing the cross-entropy loss between the true label and the student model output, as well as the difference loss between the teacher model and the student model output.

3. The method for detecting surface defects in industrial products based on an AI large model according to claim 1, characterized in that, In step S3, the lightweight model is optimized through model quantization. Model quantization involves discretizing the model's weights, converting the weights from floating-point representation to quantized representation, and then using the quantized weights for matrix operations.

4. The method for detecting surface defects in industrial products based on an AI large model according to claim 1, characterized in that, In step S4, the lightweight model in the edge device is deployed using a deep learning inference acceleration tool, which is used to optimize the model's inference speed and computational efficiency.

5. The method for detecting surface defects in industrial products based on an AI large model according to claim 1, characterized in that, In step S5, dynamically adjusting the detection threshold includes the following steps: a. Statistically analyze the confidence distribution of defect detection results, and calculate the mean and standard deviation of the confidence scores; b. Based on the set adjustment parameters, the detection threshold is dynamically adjusted using the mean and standard deviation.

6. The method for detecting surface defects in industrial products based on an AI large model according to claim 1, characterized in that, In step S2, the weights for multi-scale feature fusion are set by optimizing the response levels of high-frequency and low-frequency features, and are used to perform weighted synthesis of local detail defects and global features.

Images