High-precision AOI system and method for surface defect of complex industrial product

Abstract

The present application relates to the technical field of automated inspection. Disclosed are a high-precision AOI system and method for a surface defect of a complex industrial product, wherein the high-precision AOI system for a surface defect of a complex industrial product comprises: a multi-modal sensor module, an image processing and data fusion module, a feature extraction and high-dimensional data analysis module, and a deep learning detection module; and the high-precision AOI method for a surface defect of a complex industrial product comprises: using a multi-modal sensor module to collect multi-dimensional image data of a product surface, and performing optimization, geometric correction and illumination correction on the multi-dimensional image data by means of an image processing and data fusion module. The present application uses a multi-modal sensor fusion technique to comprehensively collect image data from a visible light sensor, an infrared sensor, a hyperspectral sensor and a 3D depth sensor, thus achieving the comprehensive perception of a surface defect of a complex industrial product, and thereby significantly improving the accuracy and robustness of defect detection.

Description

High-precision AOI detection system and method for surface defects in complex industrial products Technical Field

[0001] This application relates to the field of automated inspection technology, specifically to a high-precision AOI inspection system and method for surface defects of complex industrial products. Background Technology

[0002] With the continuous advancement of modern industry, especially the increasingly sophisticated manufacturing processes of complex industrial products, the detection of surface defects has become particularly important in quality control. Surface defects not only affect the appearance quality of products but may also impact their functionality and reliability. Therefore, improving the accuracy, stability, and efficiency of surface defect detection has become an urgent need in industrial production. Currently, surface defect detection mainly relies on automated optical inspection (AOI) technology. This technology acquires and analyzes image information of the product surface to discover potential defects. Although existing technologies have achieved certain results in some specific applications, current AOI systems and methods still have many shortcomings when dealing with complex industrial products.

[0003] In existing defect detection technologies, most systems rely on a single type of sensor for image acquisition. For example, visible light sensors or infrared imaging sensors are used to acquire images of product surfaces. These single sensors can capture some surface defects under specific conditions, but when faced with complex industrial products, a single sensor often cannot capture all possible defects. For example, infrared sensors can detect cracks caused by thermal differences, but may not be able to capture some small surface cracks. Similarly, visible light sensors can clearly capture macroscopic surface defects, but cannot identify deep or minute surface changes. Due to the lack of comprehensive perception of different types of defects, existing technologies often encounter difficulties in the inspection of products with complex materials and irregular shapes, leading to false detections or missed detections. Furthermore, image processing technology typically relies on traditional algorithms for defect identification; however, the limitations of traditional image processing methods lie in their inability to effectively address the impact of factors such as lighting variations and surface irregularities on image quality. During the production process, surface defects may be obscured or distorted due to factors such as uneven lighting, material differences, or irregular surface morphology. Traditional image processing methods usually cannot automatically adjust for these effects, resulting in unclear or difficult-to-detect defects in the image. Especially for the inspection of complex surfaces or multiple materials, existing methods struggle to maintain high image quality, thus affecting the accurate identification of subsequent defects. Therefore, those skilled in the art propose a high-precision AOI inspection system and method for surface defects in complex industrial products to address these issues. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application provides a high-precision AOI detection system and method for surface defects in complex industrial products, aiming to solve problems such as insufficient multi-sensor data fusion, low image processing accuracy, and low feature extraction efficiency in existing defect detection technologies.

[0005] To achieve the above objectives, this application provides the following technical solution: a high-precision AOI inspection system for surface defects in complex industrial products, comprising:

[0006] A multimodal sensor module is used to acquire multi-dimensional image data of the product surface;

[0007] The image processing and data fusion module is used to perform information optimization, geometric correction, and illumination correction on the raw image data acquired by the multimodal sensor module to ensure image quality.

[0008] The feature extraction and high-dimensional data analysis module is used to perform feature extraction and dimensionality reduction analysis on the data output by the image processing and data fusion module, retaining key information related to defect detection as feature information; and

[0009] The deep learning detection module is used to classify, identify, and locate defects based on the feature information output by the feature extraction and high-dimensional data analysis module.

[0010] Preferably, the multimodal sensor module includes:

[0011] Visible light sensors are used to acquire regular images of product surfaces;

[0012] Infrared imaging sensors are used to capture information about surface defects caused by temperature changes;

[0013] Hyperspectral imaging sensors are used to acquire reflection information at different wavelengths, helping to reveal defects that are difficult to detect using traditional optical methods; and

[0014] 3D depth sensors are used to collect three-dimensional information about the surface of products with complex geometries.

[0015] Preferably, the image processing and data fusion module includes:

[0016] The optical information maximization unit is used to optimize camera parameters, light source position and imaging angle according to Shannon's information theory to maximize the mutual information of defect information.

[0017] The geometry and illumination correction unit is used to perform geometric correction and illumination optimization on the acquired image data based on reflection geometry theory and the radiative transfer equation (RTE).

[0018] Preferably, the optical information maximization unit calculates the mutual information based on the following formula: I(X,Y)=H(X)+H(Y)-H(X,Y);

[0019] Where: I(X,Y) represents the mutual information between random variables X and Y; H(X) and H(Y) represent the entropy of random variables X and Y, respectively; H(X,Y) represents the joint entropy of X and Y.

[0020] Preferably, the geometry and illumination correction unit performs illumination correction based on the following reflection model:

[0021] Lambertian reflection model: R L (θ)=ρ·cos(θ);

[0022] Where: R L (θ) represents the reflectivity of the surface at angle θ; ρ is the reflection coefficient; θ is the angle between the incident ray and the surface normal;

[0023] Phong reflection model: R P (θ,φ)=k s ·(cos(θ)·cos(φ)) n ;

[0024] Where: R P (θ,φ) represents the specular reflectivity of the surface at the incident angle θ and the reflection angle φ; k s is the specular reflection coefficient; n is the specular reflection index, which controls the smoothness of the reflected light.

[0025] Preferably, the feature extraction and high-dimensional data analysis module includes:

[0026] Principal Component Analysis (PCA) unit is used to reduce the dimensionality of multimodal sensor data and extract the main feature orientations;

[0027] t-distributed random neighborhood embedding t-SNE units are used to preserve the local structure between high-dimensional data points in order to optimize defect information extraction;

[0028] The data fusion unit is used to perform weighted averaging of features from different sensors to extract key information about defects.

[0029] Preferably, the data fusion unit performs a weighted average of the feature data based on the following formula:

[0030] Wherein: F final Features after fusion; F i The features extracted by the i-th sensor; w i The weights for each sensor feature.

[0031] Preferably, the deep learning detection module includes:

[0032] Convolutional Neural Network (CNN) units are used for automatic classification and localization of defects from extracted features;

[0033] The adaptive optimization unit dynamically adjusts the weights of the CNN model based on the Adam optimizer to improve the accuracy of defect detection.

[0034] A high-precision AOI inspection method for surface defects in complex industrial products includes the following steps:

[0035] S1. Use a multimodal sensor module to collect multi-dimensional image data of the product surface;

[0036] S2. The multi-dimensional image data is optimized, geometrically corrected, and illuminated by the image processing and data fusion module.

[0037] S3. Extract key information about defects through feature extraction and high-dimensional data analysis modules, and perform dimensionality reduction on the data;

[0038] S4. Use the deep learning detection module to classify, identify and locate defects based on the extracted feature information.

[0039] Preferably, the specific steps of the S2 step optimization, geometric correction, and illumination correction are as follows:

[0040] By optimizing camera parameters, light source position, and imaging angle through the optical information maximization unit, the mutual information of defect information is maximized to ensure image quality; and

[0041] Illumination correction is performed using a geometry and illumination correction unit based on the reflection geometry model and the radiative transfer equation (RTE), reducing interference caused by illumination. Attached Figure Description

[0042] Figure 1 is a schematic diagram of the system architecture of the high-precision AOI inspection system for surface defects of complex industrial products provided in this application;

[0043] Figure 2 is a schematic diagram of the architecture of the multimodal sensor module of this application;

[0044] Figure 3 is a schematic diagram of the image processing and data fusion module of this application;

[0045] Figure 4 is a schematic diagram of the feature extraction and high-dimensional data analysis module of this application;

[0046] Figure 5 is a schematic diagram of the architecture of the deep learning detection module of this application; and

[0047] Figure 6 is a schematic diagram of the method flow of this application. Detailed Implementation

[0048] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0049] Please refer to Figures 1-5. Embodiments of this application provide a high-precision AOI inspection system for surface defects in complex industrial products, including:

[0050] A multimodal sensor module is used to acquire multi-dimensional image data of the product surface;

[0051] Specifically, the multimodal sensor module is a key component for collecting surface data of complex industrial products. By integrating multiple sensor technologies, the module provides comprehensive surface information, ensuring accurate and comprehensive identification of product defects in complex production environments. By collecting data from multiple dimensions such as visible light, infrared, hyperspectral, and three-dimensional depth, the module can accurately detect surfaces under different angles and spectral conditions.

[0052] The multimodal sensor module employs multiple complementary sensor types to acquire product surface information from different perspectives simultaneously. These sensors include visible light sensors, infrared imaging sensors, hyperspectral imaging sensors, and 3D depth sensors. Each sensor has its unique working principle and advantages. By combining multiple modal sensors, the information loss caused by a single sensor type can be minimized, ensuring the comprehensiveness and accuracy of defect detection.

[0053] Visible light sensors are used to capture regular images of product surfaces. Their main function is to provide high-resolution visual data that can reveal most surface defects, such as scratches and stains. These defects are usually quite noticeable on the surface and can typically be detected using standard optical imaging techniques. The quality of the visible light image directly affects the accuracy of defect detection, so proper light source settings and camera exposure control are particularly important.

[0054] Visible light sensors can be equipped with high-resolution imaging systems to capture subtle surface features, employ optical zoom lenses to enhance the imaging system's long-distance shooting capabilities, and automatically adjust the focal length according to surface conditions to obtain the best imaging results.

[0055] The main function of infrared imaging sensors is to identify heat-related defects by capturing changes in surface temperature. In particular, when detecting defects such as thermal cracks, bubbles, and separation of inner and outer layers, infrared imaging sensors provide additional information that traditional visible light sensors cannot provide. Infrared sensors can accurately detect these thermal defects based on temperature differences, and therefore play an important role in product quality inspection.

[0056] Infrared sensors detect the temperature distribution on the surface of materials and use thermal conduction models to calculate possible internal defects, such as thermal cracks caused by materials or manufacturing processes, which can cause uneven temperature distribution on the surface. Infrared sensors can effectively capture these minute temperature differences, thereby helping the system to discover hidden surface defects.

[0057] Infrared imaging sensors can combine temperature data with image data to mark potential thermal cracks or bubbles in the image, thereby improving detection accuracy.

[0058] Hyperspectral imaging sensors identify microscopic changes and defects in surface materials by capturing information from multiple wavelengths. Unlike traditional visible light images, hyperspectral sensors can acquire data in multiple spectral bands (including visible, infrared, and ultraviolet light), thus providing more information for the detection of minute defects on product surfaces. This is especially important for the detection of defects on complex material surfaces.

[0059] Hyperspectral imaging sensors can effectively distinguish subtle differences in surface materials and textures by analyzing reflected light across multiple wavelengths. They can detect defects such as tiny cracks, bubbles, and surface contamination that are difficult to detect by traditional optical sensors. The hyperspectral sensor can generate spectral data for each pixel, and after analysis, it can identify the presence of minute defects in the material by observing changes in the reflection spectrum.

[0060] 3D depth sensors use laser scanning or structured light technology to obtain three-dimensional shape data of an object's surface by measuring the time difference or change of light when it is reflected on the surface. This technology can accurately capture the geometry of complex products and identify defects caused by geometric deformation. For example, when inspecting complex metal surfaces or curved products, 3D depth sensors can identify defects caused by surface deformation, scratches, etc., and display the specific location and shape of the defects through stereoscopic images.

[0061] 3D depth sensors use laser scanning to capture depth information of irregular areas on the product surface in real time. When cracks or dents appear on the product surface, 3D depth sensors can provide detailed depth data to help subsequent image processing and defect recognition modules accurately identify and locate defects.

[0062] By combining 3D depth sensors and visible light image sensors, and using stereo image matching technology, depth information and surface images are combined to further improve the detection accuracy of surface defects with complex shapes.

[0063] This application employs multimodal sensor fusion technology to comprehensively acquire image data from visible light, infrared, hyperspectral, and 3D depth sensors, achieving comprehensive perception of surface defects in complex industrial products. Compared to traditional single-sensor technology, existing technologies often fail to fully capture defect information when dealing with products with varying materials and complex shapes. This application, through multi-dimensional sensor data fusion, significantly improves the accuracy and robustness of defect detection.

[0064] The image processing and data fusion module is used to perform information optimization, geometric correction and illumination correction on the raw image data (i.e., multi-dimensional image data) acquired by the multimodal sensor module to ensure image quality;

[0065] Specifically, after the multimodal sensor module completes data acquisition, the acquired image data will be sent to the image processing and data fusion module for further optimization and processing. The core task of this module is to optimize the acquired image data through techniques such as information maximization, geometric and illumination correction, so as to ensure that it can be accurately processed in the subsequent feature extraction and deep learning detection modules. The image processing and data fusion module, through the cooperation of various algorithms, can effectively reduce the interference caused by factors such as lighting conditions and surface irregularities, thereby improving the accuracy of defect detection.

[0066] The image processing and data fusion module consists of an optical information maximization unit and a geometry and illumination correction unit. The two work closely together to ensure image quality improvement and provide high-quality data support for subsequent feature extraction and deep learning detection. This module eliminates unnecessary background noise and corrects errors caused by changes in surface geometry or illumination conditions by reasonably adjusting optical imaging conditions.

[0067] When performing defect detection, the quality of the image directly affects the effect of subsequent processing. The core objective of the optical information maximization unit is to maximize the defect information in the image, reduce background noise, and ensure the effectiveness of the image information. To achieve this goal, this unit optimizes the optical imaging process based on the mutual information principle in Shannon's information theory.

[0068] Mutual information I(X,Y) can describe the degree of correlation between two sets of data X and Y. In image processing, X represents the original data of the image, and Y represents defect information.

[0069] Maximizing optical information is achieved by optimizing imaging conditions, such as adjusting the position of the light source, the camera's exposure time, and the imaging angle, in order to maximize the representation of defect information in the image. The formula for calculating mutual information is as follows: I(X,Y)=H(X)+H(Y)-H(X,Y);

[0070] Where: I(X,Y) is the mutual information between random variables X and Y; H(X) and H(Y) represent the entropy of X and Y respectively; H(X,Y) is the joint entropy of X and Y.

[0071] The optical information maximization unit dynamically adjusts imaging parameters based on real-time image quality feedback. In this way, the system can automatically adjust imaging settings for different product surface conditions and lighting conditions, ensuring optimal image quality in every working environment.

[0072] The optical information maximization unit can also automatically optimize parameter settings based on historical data through machine learning algorithms, thereby improving the intelligence level of the detection process. For example, the system learns and records the optimal illumination configuration under different environmental conditions, thereby achieving automated illumination adjustment.

[0073] The main task of the geometry and illumination correction unit is to eliminate errors caused by factors such as complex surface shapes and uneven illumination. Irregular surface shapes and changes in reflectivity can affect the acquired image data. Especially when processing products with protrusions, depressions or complex curved surfaces, changes in illumination conditions can also lead to inconsistent appearance of defects in the image, further increasing the difficulty of detection.

[0074] The geometry and illumination correction unit performs corrections based on reflection geometry theory and the radiative transfer equation (RTE). First, reflection geometry theory is based on the laws of light reflection. It determines the reflection characteristics of an object by measuring the surface reflection coefficient. Different reflection models are used for different types of surfaces. For example, the Lambertian reflection model is used for planar surfaces, while the Phong reflection model is used for surfaces with specular gloss.

[0075] The Lambertian reflection model is used to describe the illumination behavior of diffuse reflective surfaces, and the formula is as follows: R L (θ)=ρ·cos(θ);

[0076] Where: R L (θ) is the reflectivity of the surface at angle θ; ρ is the reflectance coefficient of the surface, which reflects the light absorption characteristics of the surface; θ is the angle between the incident ray and the surface normal.

[0077] For surfaces with a specular finish, the Phong reflection model is more suitable, as shown in the following formula: R P(θ,φ)=k s ·(cos(θ)·cos(φ)) n ;

[0078] Where: R P (θ,φ) is the specular reflectivity of the surface at the incident angle θ and the reflection angle φ; k s is the specular reflection coefficient, representing the specular gloss; n is the specular reflection index, which controls the intensity and diffusion of reflection.

[0079] In addition, the radiative transfer equation (RTE) is used to describe the process of light propagating on the surface of an object. It accurately simulates the reflection and scattering of light through mathematical modeling. The formula for RTE is as follows:

[0080] Where: I is the radiation intensity; σ a σ is the absorption coefficient, which describes the light absorption characteristics of a material; s Ω is the scattering coefficient, describing the light scattering ability of a material; p(Ω,Ω′) is the scattering term, describing the direction of light scattering; Ω is the direction of the incident light; Ω′ is the direction of the outgoing light; ∫ 4π p(Ω,Ω′)I(Ω′)dΩ′ is an integral term representing the propagation and energy transfer of light during the scattering process.

[0081] This application optimizes image quality through an image processing and data fusion module, and successfully eliminates interference caused by uneven illumination and irregular surface shape by employing optical information maximization and geometric correction techniques. Compared with common image processing methods in the prior art, traditional methods often ignore errors caused by illumination changes and complex surfaces, while the technical solution of this application greatly improves the accuracy of image data and provides more reliable data support for subsequent defect identification.

[0082] The feature extraction and high-dimensional data analysis module is used to perform feature extraction and dimensionality reduction analysis on the data output by the image processing and data fusion module, retaining key information related to defect detection.

[0083] Specifically, after the image processing and data fusion module completes the image data optimization and correction, the next key step is to perform feature extraction and high-dimensional data analysis on the optimized data. This module aims to extract key information related to defects from complex datasets and transform it into a data format that is easy for deep learning models to process. The feature extraction and high-dimensional data analysis module not only helps to reduce redundant data, but also improves the efficiency and accuracy of subsequent processing through dimensionality reduction and data fusion techniques.

[0084] The feature extraction and high-dimensional data analysis module includes two key units: Principal Component Analysis (PCA) and t-SNE (t-distributed random neighborhood embedding). Through the collaborative work of these two units, the system can effectively reduce the amount of data while ensuring data fidelity, thereby improving the speed and accuracy of subsequent detection. The module's workflow includes extracting and fusing features from the data acquired by the multimodal sensor module, and performing dimensionality reduction on the data to make defect information more prominent and reduce noise and redundant information.

[0085] The data input to the feature extraction module is usually high-dimensional, especially data from multimodal sensors, such as visible light images, infrared images, hyperspectral images, and 3D depth data. This data usually contains a lot of redundant information. Therefore, the main task of the principal component analysis (PCA) unit is to perform dimensionality reduction on this data, extract the most important feature information, and retain the main components that are highly relevant to defect detection.

[0086] The PCA unit reduces the dimensionality of the data in the following way: Assuming the input data matrix is ​​X with dimensions m×n (where m is the number of samples and n is the number of features per sample), by performing Singular Value Decomposition (SVD) on X, we can obtain the following decomposition form: X = USV T ;

[0087] Where: U is the left singular vector matrix; S is a diagonal matrix containing singular values; V is the right singular vector matrix; X is the matrix to be decomposed; T is the matrix transpose operation.

[0088] By selecting the principal component matrix V k The first k principal components map the original data to a new low-dimensional space X. reduced =X·V k This reduces the dimensionality of the data and retains the most important features.

[0089] PCA can map a high-dimensional feature space to a low-dimensional space, maximizing the variability of the data and effectively reducing redundant information in the data, highlighting the main features related to defects.

[0090] Unlike the linear dimensionality reduction of the PCA unit, the t-SNE (t-distributed random neighborhood embedding) unit further optimizes the low-dimensional representation of the data through nonlinear dimensionality reduction techniques. t-SNE is mainly used to preserve the local structure between high-dimensional data points, so that similar data points can be better clustered together after dimensionality reduction.

[0091] The core objective of t-SNE is to minimize the Kullback-Leibler divergence between the similarity between data points in the high-dimensional space and the similarity between data points in the low-dimensional space, as shown in the following formula:

[0092] Where: p ij It represents the similarity between data points i and j in a high-dimensional space, typically measured using a Gaussian distribution; q ij It is the similarity between data points i and j in the low-dimensional space; KL(P||Q) is the KL divergence between the high-dimensional data point distribution and the low-dimensional data point distribution; It represents the information loss or deviation between the two distributions and is used to measure the ratio of the similarity of data points in high-dimensional space and low-dimensional space.

[0093] By minimizing this divergence, t-SNE can preserve the local structure in the high-dimensional space in the low-dimensional space, avoiding the problem of losing local data relationships in traditional linear dimensionality reduction methods. For defect recognition tasks, t-SNE helps to reveal details and patterns hidden in high-dimensional data, ensuring that key defect features in the image remain prominent in the dimensionality-reduced data.

[0094] t-SNE reduces the dimensionality of data to two or three dimensions, allowing subsequent defect identification tasks to be further optimized through visualization, thereby enhancing the model's ability to identify complex defects.

[0095] In the multimodal sensor module, multiple sensors provide different types of image data. Each sensor's data has different characteristics and advantages. Therefore, after data extraction and dimensionality reduction, the data fusion unit is responsible for effectively combining the data from different sensors to form a unified feature representation.

[0096] Data fusion can be performed using a weighted average. For example, if the features extracted from the visible light sensor, infrared sensor, and hyperspectral sensor are F1, F2, and F3, respectively, then the fused feature F... final This can be expressed by the following formula:

[0097] Wherein: F final It is the final characteristic after fusion; F i It is the extracted feature from the i-th sensor; w i It is the weight of each sensor feature, which is usually determined by the sensor's accuracy and importance.

[0098] Through weighted fusion, the advantages of different sensors can be complemented, and the system can fully integrate the information from all sensors during the defect detection process, ensuring sufficient robustness and accuracy when dealing with complex surfaces or hard-to-detect defects.

[0099] The data fusion unit can also optimize the fused features using deep learning methods, making the final features more suitable for subsequent defect detection tasks.

[0100] This application introduces high-dimensional data analysis methods, including PCA and t-SNE algorithms, to reduce and optimize feature data, significantly reducing the interference of redundant information. Compared with traditional feature extraction methods, this application not only improves the prominence of defect features but also reduces subsequent computation, ensuring that the system can operate efficiently in complex data environments and avoiding the problem of excessive reliance on computing resources in traditional methods.

[0101] The deep learning detection module is used to classify, identify, and locate defects based on the feature information output by the feature extraction and high-dimensional data analysis module.

[0102] Specifically, after the feature extraction and high-dimensional data analysis module completes the optimization and dimensionality reduction of the feature data, the final data will be sent to the deep learning detection module for defect classification, identification and localization. This module is the core part of the system and can automatically identify surface defects based on the extracted feature information. It also has high-precision classification and localization capabilities. The deep learning detection module processes the input features through a convolutional neural network (CNN) and provides stable and reliable defect detection capabilities by continuously optimizing the model.

[0103] The deep learning detection module is based on a convolutional neural network (CNN) structure. Through hierarchical feature learning, it transforms low-level features in images into high-level features, thereby enabling accurate defect classification and localization. The deep learning network extracts deep features of the image between each layer through multiple iterations of training and optimization, enabling the system to effectively identify minute defects on the surface of complex products, such as cracks, bubbles, and scratches. The implementation of this module provides powerful automated processing capabilities for subsequent defect identification and can improve the system's efficiency and accuracy in identifying different types of surface defects.

[0104] Convolutional Neural Network (CNN) Unit: Convolutional Neural Networks (CNNs) are widely used in image processing tasks, especially for automated defect detection. CNNs can extract image features step by step through convolution operations and perform deep feature classification through pooling layers and fully connected layers. In this application, the CNN structure is used to learn and identify defects on the product surface from the extracted high-dimensional features.

[0105] CNNs consist of multiple convolutional layers, pooling layers, and fully connected layers. In the convolutional layers, local features are extracted by performing convolution operations on the input image. For example, input data X and convolutional kernel K are convolved to obtain the output feature map F. conv The calculation formula is as follows: F conv =Conv(X,K)+b;

[0106] Wherein: F conv is the output feature map of the convolutional layer; X is the input image data; K is the convolutional kernel; b is the bias term.

[0107] Convolutional operations can effectively extract local features in images, such as edges and textures. These features are important for defect detection. As the network deepens, the features extracted by the convolutional layers gradually become more abstract, which helps to identify complex defect patterns.

[0108] In the pooling layer, the CNN reduces the dimensionality of the feature map through max pooling or average pooling operations, thereby reducing computational cost and enhancing the salient features of the image. The calculation formula for the pooling operation is as follows: F pool =max(F conv );

[0109] Wherein: F pool This is the feature map after pooling; `max` indicates selecting the maximum value within the pooling region. This process reduces the size of the input data while retaining the most important feature information.

[0110] Deep Learning Model Training and Optimization: The CNN network in the deep learning detection module is trained with a large amount of labeled data, enabling the network to accurately classify various surface defects. During training, the network updates its parameters using the backpropagation algorithm, gradually improving classification accuracy. The basic formula for backpropagation is:

[0111] Where: θ t It represents the current network weights; θ t-1 It represents the network weights from the previous step; η is the learning rate; v t It is the square momentum term of the gradient; m t ε is the momentum term of the gradient; ε is a constant to avoid division by zero.

[0112] This adaptive optimization mechanism can dynamically adjust the weights, ensuring convergence during training. By updating the network weights through backpropagation, CNNs can continuously improve their feature extraction capabilities, ultimately achieving efficient and accurate defect detection.

[0113] To improve detection accuracy, CNNs can also adopt multi-level, multi-channel structures. By processing image features from different sensors simultaneously, they can enhance the robustness of defect detection. For example, image data from visible light, infrared, and hyperspectral sensors can be processed independently through different convolution channels and finally fused into a unified feature representation. This structure can maximize the information provided by different sensors and improve the ability to identify complex defects.

[0114] Model Adaptive Optimization: The deep learning detection module uses adaptive optimization algorithms to gradually improve the model's ability to identify different types of surface defects. Typically, the model is trained on a large-scale labeled dataset and accuracy is improved by gradually adjusting the network structure and hyperparameters. For example, in the early stages of training, CNN may use larger step sizes to accelerate the training process, while smaller step sizes are used in the later stages to avoid overfitting.

[0115] The Adam optimization algorithm was used for training. In each iteration step, the Adam optimizer dynamically adjusts the learning rate based on the mean and variance of the current gradient. The Adam optimizer can improve the stability of the training process in most cases and reduce the cumbersome operation of learning rate adjustment in the traditional gradient descent method.

[0116] Output and Localization: Through the analysis of image features by CNN, the deep learning detection module can output the defect probability of each image region and determine the category and location of the defect based on these probabilities. For example, the network will output a probability distribution with multiple category labels based on the feature map in the input data, representing the probability of different defect categories. Finally, the model achieves accurate defect localization by calculating the position coordinates of the defect region in the output image.

[0117] The deep learning detection module can also process images using image segmentation technology to accurately segment defect areas and further label information such as the size and shape of defects. With this information, the subsequent process control system can adjust production parameters based on the detection results to reduce the occurrence of defects.

[0118] This application achieves automated defect classification, identification, and localization through a deep learning detection module, and uses a convolutional neural network (CNN) to accurately process surface defects. Compared with traditional manual visual inspection or rule-based detection methods, the deep learning method of this application has higher flexibility and accuracy, can automatically adapt to different types of defects, and the detection accuracy continuously improves with system training, greatly reducing human error and inefficiency.

[0119] The high-precision AOI detection method for surface defects of complex industrial products described below can be referred to in conjunction with the high-precision AOI detection system for surface defects of complex industrial products described above.

[0120] Please refer to Figure 6 for a high-precision AOI inspection method for surface defects in complex industrial products, which includes the following steps:

[0121] S1. Use a multimodal sensor module to collect multi-dimensional image data of the product surface;

[0122] S2. The multi-dimensional image data is optimized, geometrically corrected, and illuminated by the image processing and data fusion module.

[0123] S3. Extract key information about defects through feature extraction and high-dimensional data analysis modules, and perform dimensionality reduction on the data;

[0124] S4. Use the deep learning detection module to classify, identify and locate defects based on the extracted feature information.

[0125] Specifically, the system first uses a variety of sensors (such as visible light sensors, infrared imaging sensors, hyperspectral sensors, and 3D depth sensors) to collect comprehensive image data of the product surface. Each of these sensors has different advantages and can capture different types of defects under various conditions. For example, infrared sensors can identify cracks or bubbles caused by thermal differences, while 3D depth sensors can obtain precise shape information of the product surface. By combining the data from these different sensors, the system can provide multi-angle and multi-dimensional information for subsequent processing and analysis, ensuring that surface defects are fully captured.

[0126] Based on the data collected by the multimodal sensor module, the image processing and data fusion module is used to optimize, geometrically correct, and correct the illumination of the image data to improve image quality and accuracy. First, the system optimizes the camera settings and light source angles through an optical information maximization algorithm to maximize the defect information in the image. Then, the geometric correction and illumination correction unit corrects the image distortion caused by the complex shape of the product surface, material reflection differences, or illumination changes. This process ensures that the defect information in the image is not affected by illumination conditions and surface geometry changes, providing high-quality input data.

[0127] After image processing and data fusion are completed, the system enters the feature extraction and high-dimensional data analysis stage. First, the feature extraction unit extracts key features related to defects from the optimized image data. These features may include surface texture, color changes, temperature distribution, etc. Then, high-dimensional data analysis methods (such as principal component analysis PCA and t-SNE) are used to reduce the dimensionality of the extracted features, remove redundant information, and retain key information that is crucial for defect detection. This process helps reduce the computational burden of subsequent deep learning models, while improving the expressiveness and accuracy of the features.

[0128] Finally, the data after feature extraction and dimensionality reduction will be input into the deep learning detection module. This module uses deep learning algorithms such as convolutional neural networks (CNN) to classify, identify and locate defects in the image. By training a large amount of labeled data, the deep learning model can automatically identify various types of defects from the feature data, such as cracks, bubbles, scratches, etc., and accurately locate them according to their shape and position. The output of the model includes the defect type and location coordinates, which can be used for subsequent quality control and production optimization.

[0129] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A high-precision AOI inspection system for surface defects in complex industrial products, characterized in that, include: A multimodal sensor module is used to acquire multi-dimensional image data of the product surface; The image processing and data fusion module is used to perform information optimization, geometric correction, and illumination correction on the multi-dimensional image data acquired by the multimodal sensor module. The feature extraction and high-dimensional data analysis module is used to perform feature extraction and dimensionality reduction analysis on the data output by the image processing and data fusion module, and retain key information related to defect detection as feature information. as well as The deep learning detection module is used to classify, identify, and locate defects based on the feature information output by the feature extraction and high-dimensional data analysis module.

2. The high-precision AOI inspection system for surface defects of complex industrial products according to claim 1, characterized in that, The multimodal sensor module includes: Visible light sensors are used to acquire regular images of product surfaces; Infrared imaging sensors are used to capture information about surface defects caused by temperature changes; Hyperspectral imaging sensors are used to acquire reflection information at different wavelengths; and 3D depth sensors are used to collect three-dimensional information about the surface of products with complex geometries.

3. The high-precision AOI inspection system for surface defects of complex industrial products according to claim 1, characterized in that, The image processing and data fusion module includes: The optical information maximization unit is used to optimize camera parameters, light source position, and imaging angle based on Shannon's information theory to maximize the mutual information of defect information; and The geometry and illumination correction unit is used to perform geometric correction and illumination optimization on the acquired image data based on reflection geometry theory and the radiative transfer equation (RTE).

4. The high-precision AOI inspection system for surface defects of complex industrial products according to claim 3, characterized in that, The optical information maximization unit calculates the mutual information based on the following formula: I(X,Y)=H(X)+H(Y)-H(X,Y); Where: I(X,Y) represents the mutual information between random variables X and Y; H(X) and H(Y) represent the entropy of random variables X and Y, respectively; H(X,Y) represents the joint entropy of X and Y.

5. The high-precision AOI inspection system for surface defects of complex industrial products according to claim 3, characterized in that, The geometry and illumination correction unit performs illumination correction based on the following reflection model: Lambertian reflection model: R L (θ)=ρ·cos(θ); Where: R L (θ) represents the reflectivity of the surface at angle θ; ρ is the reflection coefficient; θ is the angle between the incident ray and the surface normal; Phong reflection model: R P (θ,φ)=k s ·(cos(θ)·cos(φ)) n ; Where: R P (θ,φ) represents the specular reflectivity of the surface at the incident angle θ and the reflection angle φ; k s is the specular reflection coefficient; n is the specular reflection index, which controls the smoothness of the reflected light.

6. The high-precision AOI inspection system for surface defects of complex industrial products according to claim 1, characterized in that, The feature extraction and high-dimensional data analysis module includes: Principal Component Analysis (PCA) unit is used to reduce the dimensionality of multimodal sensor data and extract the main feature orientations; t-distributed random neighborhood embedding t-SNE units are used to preserve the local structure between high-dimensional data points; and The data fusion unit is used to perform a weighted average of features from different sensors.

7. The high-precision AOI inspection system for surface defects of complex industrial products according to claim 6, characterized in that, The data fusion unit performs a weighted average of the feature data based on the following formula: Wherein: F final Features after fusion; F i The features extracted by the i-th sensor; w i The weights for each sensor feature.

8. The high-precision AOI inspection system for surface defects of complex industrial products according to claim 1, characterized in that, The deep learning detection module includes: Convolutional Neural Network (CNN) units are used for automatic defect classification and localization from extracted features; and The adaptive optimization unit dynamically adjusts the weights of the CNN model based on the Adam optimizer.

9. A high-precision AOI inspection method for surface defects of complex industrial products, applied to the high-precision AOI inspection system for surface defects of complex industrial products as described in any one of claims 1-8, characterized in that, Includes the following steps: S1. Use a multimodal sensor module to collect multi-dimensional image data of the product surface; S2. The multi-dimensional image data is optimized, geometrically corrected, and illuminated by the image processing and data fusion module. S3. Extract key information about defects as feature information through the feature extraction and high-dimensional data analysis module, and perform dimensionality reduction processing on the data; as well as S4. Use the deep learning detection module to classify, identify and locate defects based on the extracted feature information.

10. The high-precision AOI detection method for surface defects of complex industrial products according to claim 9, characterized in that, The optimization, geometric correction, and illumination correction in step S2 include: By optimizing camera parameters, light source position, and imaging angle through the optical information maximization unit, the mutual information of defect information is maximized; and Illumination correction is performed using a geometry and illumination correction unit based on the reflection geometry model and the radiative transfer equation (RTE).

Images