High-precision AOI detection system and method for surface defects of complex industrial products

Abstract

The invention relates to the technical field of automatic detection, and discloses a high-precision AOI detection system and method for complex industrial product surface defects, and the high-precision AOI detection system for complex industrial product surface defects comprises a multi-modal sensor module, an image processing and data fusion module, and a feature extraction and high-dimensional data analysis module. A deep learning detection module; the high-precision AOI detection method for the surface defects of the complex industrial product comprises the following steps: acquiring multi-dimensional image data of the surface of the product by using a multi-modal sensor module, and carrying out optimization, geometric correction and illumination correction on the multi-dimensional image data through an image processing and data fusion module. According to the method, a multi-mode sensor fusion technology is adopted, image data of visible light, infrared light, hyper-spectrum and a 3D depth sensor are comprehensively collected, surface defects of complex industrial products are comprehensively sensed, and the accuracy and robustness of defect detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of automated detection technology, and in particular to a high-precision AOI detection system and method for surface defects of complex industrial products. Background Art

[0002] With the continuous progress of modern industry, especially the increasingly sophisticated production processes of complex industrial products, the detection of surface defects has become particularly important in quality control; surface defects not only affect the appearance quality of the product, but may also affect its functionality and reliability; therefore, how to improve the accuracy, stability and efficiency of surface defect detection has become an urgent need in industrial production. At present, surface defect detection mainly relies on automatic optical inspection (AOI) technology, which obtains image information of the product surface and analyzes it to detect potential defects; although the existing technology has achieved certain results in some specific applications, when facing complex industrial products, the existing AOI systems and methods still have many shortcomings.

[0003] In existing defect detection technologies, most systems rely on a single type of sensor for image acquisition; for example, visible light sensors or infrared imaging sensors are used to obtain product surface images. These single sensors can capture some surface defects under specific conditions, but when faced with complex industrial products, a single sensor is often unable to fully capture all possible defects; for example, infrared sensors can detect cracks caused by thermal differences, but may not be able to capture some small surface cracks. Similarly, visible light sensors can clearly capture macroscopic defects on the surface, but cannot identify deep or subtle surface changes. Due to the lack of the ability to fully perceive different types of defects, existing technologies often encounter difficulties in detecting products with complex materials and irregular shapes, resulting in false detections or missed detections. In addition, image processing technology usually relies on traditional algorithms to identify defects; however, the limitation of traditional image processing methods is that they cannot effectively deal with the impact of factors such as lighting changes and surface irregularities on image quality; during the production process, the surface may be obscured or distorted due to factors such as uneven lighting, material differences or irregular surface morphology. Traditional image processing methods are usually unable to automatically adjust these effects, so the appearance of defects in the image may be unclear or difficult to detect; especially for the detection of complex surfaces or multiple materials, existing methods are difficult to maintain high image quality, thus affecting the subsequent accurate identification of defects. For this reason, those skilled in the art have proposed a high-precision AOI detection system and method for surface defects of complex industrial products to solve the above problems. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a high-precision AOI detection system and method for surface defects of complex industrial products, aiming to solve the problems of insufficient multi-sensor data fusion, low image processing accuracy, and low feature extraction efficiency in the existing defect detection technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a high-precision AOI detection system for surface defects of complex industrial products, comprising: Multimodal sensor module, used to collect multi-dimensional image data of product surface; An image processing and data fusion module, used to perform information optimization, geometric correction and illumination correction on the original image data collected by the multimodal sensor module to ensure image quality; A feature extraction and high-dimensional data analysis module, used to perform feature extraction and dimensionality reduction analysis on the data output by the image processing and data fusion module, and retain key information related to defect detection; A deep learning detection module is used to extract and analyze feature information based on the feature information output by the feature extraction and high-dimensional data analysis module.

[0006] Preferably, the multimodal sensor module comprises: Visible light sensor, used to collect regular images of product surfaces; Infrared imaging sensors to capture information about surface defects caused by temperature changes; Hyperspectral imaging sensors, which collect reflectance information at different wavelengths to help reveal defects that are difficult to detect with traditional optical inspection; 3D depth sensors are used to collect three-dimensional information on the surface of products with complex geometric shapes.

[0007] Preferably, the image processing and data fusion module includes: Optical information maximization unit, used to optimize camera parameters, light source position and imaging angle according to Shannon information theory to maximize the mutual information of defect information; The geometry and illumination correction unit is used to perform geometry correction and illumination optimization on the collected image data based on reflection geometry theory and radiation transfer equation RTE.

[0008] Preferably, the optical information maximization unit calculates the mutual information based on the following formula: ; in: Represents a random variable and The mutual information between and Denote random variables and Entropy of express and The joint entropy of .

[0009] Preferably, the geometry and illumination correction unit performs illumination correction based on the following reflection model: Lambertian reflection model: ; in: Indicates the surface at an angle Reflectivity under is the reflection coefficient; is the angle between the incident light and the surface normal; Phong reflection model: ; in: The surface is at an incident angle and reflection angle Specular reflectivity under ; is the specular reflection coefficient; The specular reflection index controls the smoothness of the reflected light.

[0010] Preferably, the feature extraction and high-dimensional data analysis module includes: Principal component analysis (PCA) unit, used to reduce the dimension of multimodal sensor data and extract the main feature directions; t-Distributed Stochastic Neighbor Embedding (t-SNE), which is used to preserve the local structure between high-dimensional data points in order to optimize defect information extraction; The data fusion unit is used to perform weighted averaging of features from different sensors to extract key information of defects.

[0011] Preferably, the data fusion unit performs weighted averaging on the feature data based on the following formula: ; in: is the fused feature; For the Features extracted by sensors; is the weight of each sensor feature.

[0012] Preferably, the deep learning detection module includes: Convolutional neural network (CNN) unit, used to automatically classify and locate defects from the extracted features; The adaptive optimization unit dynamically adjusts the weights of the CNN model based on the Adam optimizer to improve the accuracy of defect detection.

[0013] The high-precision AOI detection method for surface defects of complex industrial products includes the following steps: S1. Collect multi-dimensional image data of the product surface using a multimodal sensor module; S2, optimizing, geometrically correcting and lighting correcting the multi-dimensional image data through an image processing and data fusion module; S3. Extract key information of defects through feature extraction and high-dimensional data analysis modules, and perform dimensionality reduction on the data; S4. Use the deep learning detection module to classify, identify and locate defects based on the extracted feature information.

[0014] Preferably, the specific steps of the S2 step optimization, geometric correction and illumination correction are as follows: The optical information maximization unit optimizes the camera parameters, light source position and imaging angle to maximize the mutual information of defect information and ensure image quality; The geometry and illumination correction unit performs illumination correction based on the reflection geometry model and the radiation transfer equation (RTE) to reduce the interference caused by illumination.

[0015] The present invention provides a high-precision AOI detection system and method for complex industrial product surface defects. It has the following beneficial effects: 1. The present invention adopts multimodal sensor fusion technology to comprehensively collect image data from visible light, infrared, hyperspectral and 3D depth sensors, thereby achieving comprehensive perception of surface defects of complex industrial products. Compared with traditional single sensor technology, existing technologies are often unable to fully capture defect information when processing products with variable materials and complex shapes. The present invention significantly improves the accuracy and robustness of defect detection through multi-dimensional sensor data fusion.

[0016] 2. The present invention optimizes image quality through image processing and data fusion modules, and adopts optical information maximization and geometric correction technology to successfully eliminate interference caused by uneven lighting and irregular surface shape. Compared with the common image processing methods in the prior art, traditional methods often ignore the errors caused by lighting changes and complex surfaces, and the technical solution of the present invention greatly improves the accuracy of image data and provides more reliable data support for subsequent defect identification.

[0017] 3. The present invention introduces high-dimensional data analysis methods, including PCA and t-SNE algorithms, which reduce the dimension and optimize the feature data, significantly reducing the interference of redundant information; compared with traditional feature extraction methods, the present invention not only improves the prominence of defect features, but also reduces the amount of subsequent calculations, ensuring that the system can operate efficiently in a complex data environment, and avoiding the problem of excessive reliance on computing resources in traditional methods.

[0018] 4. The present invention realizes automatic defect classification, recognition and positioning through a deep learning detection module, and uses a convolutional neural network (CNN) to accurately process surface defects. Compared with traditional manual visual inspection or rule-based detection methods, the deep learning method of the present invention has higher flexibility and accuracy, can automatically adapt to different types of defects, and with the training of the system, the detection accuracy continues to improve, greatly reducing human errors and inefficiency problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the system architecture of the present invention; Figure 2 It is a schematic diagram of the multimodal sensor module architecture of the present invention; Figure 3 This is a schematic diagram of the image processing and data fusion module architecture of the present invention; Figure 4 It is a schematic diagram of the feature extraction and high-dimensional data analysis module architecture of the present invention; Figure 5 This is a schematic diagram of the deep learning detection module architecture of the present invention; Figure 6 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Please see attached Figure 1 -Attached Figure 5 The embodiment of the present invention provides a high-precision AOI detection system for surface defects of complex industrial products, including: Multimodal sensor module, used to collect multi-dimensional image data of product surface; Specifically, the multimodal sensor module is a key component for collecting surface data of complex industrial products. By integrating multiple sensor technologies, the module provides comprehensive surface information to ensure accurate and comprehensive identification of product defects in complex production environments. By collecting data from multiple dimensions such as visible light, infrared, hyperspectral and three-dimensional depth, the module can accurately detect surfaces at different angles and under different spectral conditions.

[0022] The multimodal sensor module uses multiple complementary sensor types to obtain product surface information from different perspectives at the same time. The sensors include: visible light sensors, infrared imaging sensors, hyperspectral imaging sensors and 3D depth sensors. Each sensor has its own unique working principle and advantages. The combination of multimodal sensors can minimize the information loss caused by a single sensor type and ensure the comprehensiveness and accuracy of defect detection.

[0023] Visible light sensor: Visible light sensor is used to capture regular images of product surfaces. Its main function is to provide high-resolution visual data, which can reveal most surface defects, such as scratches, stains, etc. These defects are usually more conspicuous on the surface and can usually be detected using standard optical imaging technology. The quality of visible light images directly affects the accuracy of defect detection, so reasonable light source settings and camera exposure control are particularly important.

[0024] The visible light sensor can be equipped with a high-resolution imaging system to capture subtle surface features, an optical zoom lens to enhance the long-distance shooting capability of the imaging system, and can automatically adjust the focus according to the surface conditions to obtain the best imaging effect.

[0025] Infrared imaging sensor: The main function of infrared imaging sensor is to identify heat-related defects by capturing changes in surface temperature. In particular, when detecting defects such as thermal cracks, bubbles, and separation of inner and outer layers, infrared imaging sensors provide additional information that traditional visible light sensors cannot provide. Infrared sensors can accurately detect these thermal defects based on temperature differences. Therefore, they play an important role in product quality inspection.

[0026] Infrared sensors detect the distribution of material surface temperature and use heat conduction models to infer possible internal defects. For example, thermal cracks caused by the material or manufacturing process will cause uneven temperature on the surface. Infrared sensors can effectively capture these tiny temperature differences, thereby helping the system to detect hidden surface defects.

[0027] The infrared imaging sensor can combine temperature data with image data to mark possible thermal cracks or bubbles in the image, thereby improving detection accuracy.

[0028] Hyperspectral imaging sensor: Hyperspectral imaging sensors identify microscopic changes and defects in surface materials by capturing information from multiple wavelengths. Unlike traditional visible light images, hyperspectral sensors can collect data in multiple spectral bands (including visible light, infrared light and ultraviolet light), thereby providing more information for the detection of tiny defects on the product surface, which is especially important for defect detection on complex material surfaces.

[0029] Through multi-band analysis of reflected light, the hyperspectral imaging sensor can effectively distinguish subtle differences in surface materials and textures, and detect defects such as small cracks, bubbles, surface contamination, etc. that are difficult to detect with traditional optical sensors. The hyperspectral sensor can generate spectral data for each pixel, and after analysis, it can identify whether there are tiny defects in the material through changes in the reflected spectrum.

[0030] 3D depth sensor: 3D depth sensor uses laser scanning or structured light technology to obtain three-dimensional shape data of the object surface by measuring the time difference or change when light is reflected on the surface. This technology can accurately capture the geometric shape of complex products and identify defects caused by geometric deformation of the surface. For example, when detecting complex metal surfaces or curved products, 3D depth sensors can identify defects caused by surface deformation, scratches, etc., and display the specific location and shape of the defects through stereo images.

[0031] The 3D depth sensor uses laser scanning to scan the product surface and capture the depth information of irregular surface areas in real time. When cracks or dents appear on the product surface, the 3D depth sensor can provide detailed depth data to help subsequent image processing and defect recognition modules accurately identify and locate defects.

[0032] The combination of 3D depth sensors and visible light image sensors further improves the detection accuracy of surface defects of complex shapes by combining depth information and surface images through stereo image matching technology.

[0033] An image processing and data fusion module, used to perform information optimization, geometric correction and illumination correction on the original image data collected by the multimodal sensor module to ensure image quality; Specifically, after the multimodal sensor module completes data acquisition, the collected image data will be sent to the image processing and data fusion module for further optimization and processing. The core task of this module is to optimize the collected image data through information maximization, geometry and illumination correction and other technologies to ensure that it can be accurately processed in the subsequent feature extraction and deep learning detection modules. The image processing and data fusion module can effectively reduce the interference caused by factors such as lighting conditions and irregular surface shapes through the coordination of multiple algorithms, thereby improving the accuracy of defect detection.

[0034] The image processing and data fusion module consists of an optical information maximization unit and a geometry and illumination correction unit. The two work closely together to ensure the improvement of image quality and provide high-quality data support for subsequent feature extraction and deep learning detection. This module reasonably adjusts the optical imaging conditions, eliminates unnecessary background noise, and corrects errors caused by changes in surface geometry or illumination conditions.

[0035] Optical Information Maximization Unit: When performing defect detection, the quality of the image directly affects the effect of subsequent processing. The core goal of the Optical Information Maximization Unit is to maximize the defect information in the image, reduce background noise, and ensure the effectiveness of image information. In order to achieve this goal, this unit optimizes the optical imaging process based on the mutual information principle in Shannon information theory.

[0036] Mutual Information Two sets of data can be described and The degree of correlation between Represents the raw data of the image, Represents defect information.

[0037] The operation of maximizing optical information is achieved by optimizing imaging conditions, such as adjusting the position of the light source, the exposure time of the camera, the imaging angle, etc., in order to maximize the representation of defect information in the image. The calculation formula of the mutual information is as follows: ; in: is a random variable and The mutual information between and Respectively and Entropy of yes and The joint entropy of .

[0038] The Optical Information Maximization Unit dynamically adjusts imaging parameters based on real-time image quality feedback. In this way, the system can automatically adjust imaging settings for different product surfaces and lighting conditions, ensuring optimal image quality in every working environment.

[0039] The Optical Information Maximization Unit can also use machine learning algorithms to automatically optimize parameter settings based on historical data to improve the intelligence level of the detection process. For example, the system learns and records the best lighting configuration under different environmental conditions, thereby realizing automated lighting adjustment.

[0040] Geometry and illumination correction unit: The main task of the geometry and illumination correction unit is to eliminate errors caused by complex surface shapes, uneven illumination and other factors. Irregular surface shapes and changes in reflection characteristics will affect the collected image data, especially when processing products with protrusions, depressions or complex curved surfaces. Changes in illumination conditions will also lead to inconsistent representation of defects in the image, further increasing the difficulty of detection.

[0041] The geometry and illumination correction unit performs correction based on the reflection geometry theory and the radiation transfer equation (RTE). First, the reflection geometry theory is based on the reflection law of light. The reflection characteristics of the object are determined by measuring the surface reflection coefficient. Different types of surfaces are processed using different reflection models. For example, the Lambertian reflection model is used for plane surfaces, and the Phong reflection model is used for surfaces with mirror gloss.

[0042] The Lambertian reflection model is used to describe the lighting behavior of diffuse reflective surfaces. The formula is as follows: ; in: Is the surface at an angle Reflectivity under is the reflectance of the surface, reflecting the light absorption characteristics of the surface; is the angle between the incident ray and the surface normal.

[0043] For surfaces with specular gloss, the Phong reflection model is more applicable, and the formula is as follows: ; in: is the surface at an incident angle and reflection angle Specular reflectivity under ; is the specular reflection coefficient, which indicates the specular glossiness; is the specular exponent, which controls the intensity and diffusion of the reflection.

[0044] In addition, the radiation transfer equation (RTE) is used to describe the process of light propagation on the surface of an object. It accurately simulates the reflection, scattering and other behaviors of light through mathematical modeling. The formula of RTE is as follows: ; in: is the radiation intensity; is the absorption coefficient, which describes the material's absorption characteristics of light; is the scattering coefficient, which describes the light scattering ability of the material; is the scattering term, describing the scattering direction of light; is the direction of the incident light; is the direction of the outgoing light; is an integral term, which represents the propagation and energy transfer of light during the scattering process.

[0045] A feature extraction and high-dimensional data analysis module, used to perform feature extraction and dimensionality reduction analysis on the data output by the image processing and data fusion module, and retain key information related to defect detection; Specifically, after the image processing and data fusion module completes image data optimization and correction, the next key step is to perform feature extraction and high-dimensional data analysis on the optimized data. This module aims to extract key information related to defects from complex data sets and convert it into a data format that is easy for deep learning models to process. The feature extraction and high-dimensional data analysis module not only helps to reduce redundant data, but also improves the efficiency and accuracy of subsequent processing through dimensionality reduction and data fusion technology.

[0046] The feature extraction and high-dimensional data analysis module includes two key units: the principal component analysis (PCA) unit and the t-SNE (t-distributed stochastic neighbor embedding) unit. Through the collaborative work of these two units, the system can effectively reduce the amount of data while ensuring data fidelity, thereby improving the speed and accuracy of subsequent detection. The module's workflow includes extracting and fusing features from the data obtained from the multimodal sensor module, and performing dimensionality reduction processing on the data to make the defect information more prominent and reduce noise and redundant information.

[0047] Principal component analysis (PCA) unit: The data input to the feature extraction module is usually high-dimensional, especially the data from multimodal sensors, such as visible light images, infrared images, hyperspectral images and 3D depth data. These data usually contain a lot of redundant information. Therefore, the main task of the PCA unit is to reduce the dimensionality of these data, extract the most important feature information, and retain the main components that are highly relevant to defect detection.

[0048] The PCA unit reduces the dimensionality of the data in the following way. Assuming that the input data matrix is , whose dimensions are (in is the sample size, is the number of features for each sample), by Performing singular value decomposition (SVD), we can get the following decomposition form: ; in: is the left singular vector matrix; is a diagonal matrix containing singular values; is the right singular vector matrix; is the matrix to be decomposed; is the matrix transpose operation.

[0049] By choosing the principal component matrix Center front principal components, mapping the original data into a new low-dimensional space , thereby reducing the dimensionality of the data and retaining the most important features.

[0050] PCA can map a high-dimensional feature space to a low-dimensional space, maximizing the variability of the data, and can effectively reduce the redundant information in the data and highlight the main features related to the defects.

[0051] t-SNE (t-Distributed Stochastic Neighbor Embedding) unit: Different from the linear dimensionality reduction of the PCA unit, the t-SNE unit further optimizes the low-dimensional representation of the data through nonlinear dimensionality reduction technology. t-SNE is mainly used to maintain the local structure between high-dimensional data points, so that similar data points can be better clustered together after dimensionality reduction.

[0052] The core goal of t-SNE is to minimize the KL divergence (Kullback-Leibler divergence) between the similarity between data points in high-dimensional space and the similarity between data points in low-dimensional space. The formula is as follows: ; in: It is a data point and Similarity in high-dimensional space is usually measured using Gaussian distribution; It is a data point and Similarity in low-dimensional space; It is the KL divergence between the distribution of high-dimensional data points and the distribution of low-dimensional data points; It represents the information loss or deviation between the two distributions and is used to measure the ratio of the similarity of data points in high-dimensional space and low-dimensional space.

[0053] By minimizing this divergence, t-SNE can retain the local structure in the high-dimensional space in the low-dimensional space, avoiding the problem of losing local data relationships in traditional linear dimensionality reduction methods. For defect identification tasks, t-SNE helps to reveal the details and patterns hidden in high-dimensional data and ensure that the key defect features in the image are still prominent in the data after dimensionality reduction.

[0054] t-SNE reduces the data dimension to two-dimensional or three-dimensional space, so that subsequent defect identification tasks can be further optimized through visualization, enhancing the model's ability to identify complex defects.

[0055] Data fusion unit: In the multimodal sensor module, multiple sensors provide different types of image data. The data of each sensor has different characteristics and advantages. Therefore, after data extraction and dimensionality reduction, the data fusion unit is responsible for effectively combining the data from different sensors to form a unified feature representation.

[0056] Data fusion can be performed by weighted averaging. For example, if the features extracted from the visible light sensor, infrared sensor, and hyperspectral sensor are , , , then the fused features It can be expressed by the following formula: ; in: It is the final feature after fusion; It is from Extraction features of various sensors; is the weight of each sensor feature, which is usually determined by the accuracy and importance of the sensor.

[0057] Through weighted fusion, the advantages of different sensors are complemented, and the system can fully integrate the information of all sensors during the defect detection process, ensuring sufficient robustness and accuracy when dealing with complex surfaces or hard-to-capture defects.

[0058] The data fusion unit can also optimize the fusion features through deep learning methods, making the final features more suitable for subsequent defect detection tasks.

[0059] A deep learning detection module is used to extract and analyze feature information based on the feature information output by the feature extraction and high-dimensional data analysis module.

[0060] Specifically, after the feature extraction and high-dimensional data analysis module completes the optimization and dimensionality reduction of feature data, the final data will be transmitted to the deep learning detection module for defect classification, identification and positioning. This module is the core part of this system. It can automatically identify surface defects based on the extracted feature information and has high-precision classification and positioning capabilities. The deep learning detection module processes the input features through a convolutional neural network (CNN) and provides stable and reliable defect detection capabilities by continuously optimizing the model.

[0061] The deep learning detection module is based on the convolutional neural network (CNN) structure. Through hierarchical feature learning, it converts low-level features in the image into high-level features, thereby accurately classifying and locating defects. The deep learning network extracts deep features of the image between layers through multiple iterative training and optimization, enabling the system to effectively identify tiny defects on the surface of complex products, such as cracks, bubbles, scratches, etc. The implementation of this module provides powerful automated processing capabilities for subsequent defect identification, and can improve the system's recognition efficiency and accuracy for different types of surface defects.

[0062] Convolutional Neural Network (CNN) Unit: Convolutional Neural Network (CNN) is widely used in image processing tasks, especially for automated defect detection. CNN can gradually extract image features through convolution operations and perform deep feature classification through pooling layers and fully connected layers. In the present invention, the CNN structure is used to learn and identify product surface defects from the extracted high-dimensional features.

[0063] CNN consists of multiple convolutional layers, pooling layers, and fully connected layers. In the convolutional layer, local features are extracted by performing convolution operations on the input image. For example, the input data and convolution kernel The output feature map is obtained through convolution operation , the calculation formula is as follows: ; in: is the output feature map of the convolutional layer; is the input image data; is the convolution kernel; is the bias term.

[0064] The convolution operation can effectively extract local features in the image, such as edges, textures, etc. These features are of great significance for defect detection. As the network deepens, the features extracted by the convolution layer gradually become more abstract, which helps to identify complex defect patterns.

[0065] In the pooling layer, CNN reduces the dimension of the feature map through maximum pooling or average pooling operations, reduces the amount of calculation and enhances the significant features of the image. The calculation formula of the pooling operation is as follows: ; in: It is the feature map after pooling; Indicates selecting the maximum value in the pooling area. This process can reduce the size of the input data while retaining the most important feature information.

[0066] Deep learning model training and optimization: The CNN network in the deep learning detection module is trained with a large amount of labeled data, so that the network can accurately classify various surface defects. During the training process, the network updates parameters through the back propagation algorithm to gradually improve the classification accuracy. The basic formula of back propagation is: ; in: is the current network weight; is the network weight of the previous step; is the learning rate; is the squared momentum term of the gradient; is the momentum term of the gradient; is a constant to avoid division by zero.

[0067] This adaptive optimization mechanism can dynamically adjust weights to ensure convergence during training. By back-propagating and updating network weights, CNN can continuously improve its feature extraction capabilities and ultimately achieve efficient and accurate defect detection.

[0068] In order to improve the accuracy of detection, CNN can also adopt a multi-level, multi-channel structure to enhance the robustness of defect detection by simultaneously processing image features from different sensors. For example, image data from visible light, infrared, and hyperspectral sensors can be processed independently through different convolution channels and finally fused into a unified feature representation. This structure can maximize the information provided by different sensors and improve the ability to recognize complex defects.

[0069] Model adaptive optimization: The deep learning detection module gradually improves the model's ability to recognize different types of surface defects through an adaptive optimization algorithm. Usually, the model is trained on a large-scale labeled data set, and the accuracy is improved by gradually adjusting the network structure and hyperparameters. For example, in the early stages of training, CNN may accelerate the training process by updating with a larger step size, and in the later stages, a smaller step size is used to avoid overfitting.

[0070] The Adam optimization algorithm is used for training. The Adam optimizer dynamically adjusts the learning rate according to the mean and variance of the current gradient in each iterative step. The Adam optimizer can improve the stability of the training process in most cases and reduce the tedious operation of learning rate adjustment in the traditional gradient descent method.

[0071] Result output and positioning: Through CNN's analysis of image features, the deep learning detection module can output the defect probability of each image area and determine the category and location of the defect based on these probabilities. For example, the network will output a probability distribution with multiple category labels based on the feature map in the input data, indicating the possibility of different defect categories. Finally, the model calculates the position coordinates of the defect area in the output image to achieve accurate defect positioning.

[0072] The deep learning detection module can also process images through image segmentation technology, accurately segment defective areas, and further calibrate information such as the size and shape of the defects. With this information, the subsequent process control system can adjust production parameters according to the detection results to reduce the occurrence of defects.

[0073] The high-precision AOI detection method for surface defects of complex industrial products described below and the high-precision AOI detection system for surface defects of complex industrial products described above can be referred to each other.

[0074] Please see attached Figure 6 , a high-precision AOI detection method for surface defects of complex industrial products, including the following steps: S1. Collect multi-dimensional image data of the product surface using a multimodal sensor module; S2, optimizing, geometrically correcting and lighting correcting the multi-dimensional image data through an image processing and data fusion module; S3. Extract key information of defects through feature extraction and high-dimensional data analysis modules, and perform dimensionality reduction on the data; S4. Use the deep learning detection module to classify, identify and locate defects based on the extracted feature information.

[0075] Specifically, we first use a variety of sensors (such as visible light sensors, infrared imaging sensors, hyperspectral sensors and 3D depth sensors) to collect comprehensive image data of the product surface. These sensors each have different advantages and can capture different types of defects under a variety of conditions. For example, infrared sensors can identify cracks or bubbles caused by thermal differences, and 3D depth sensors can obtain precise shape information of the product surface. By combining these data from different sensors, the system can provide multi-angle and multi-dimensional information for subsequent processing and analysis to ensure that surface defects are fully captured.

[0076] Based on the data collected by the multimodal sensor module, the image processing and data fusion module is entered. The task of this module is to optimize, geometrically correct and correct the image data to improve image quality and accuracy. First, the system will optimize the camera settings and light source angles through the optical information maximization algorithm to maximize the defect information in the image. Then, the geometric correction and lighting correction units will correct the image distortion caused by the complex shape of the product surface, material reflection differences or lighting changes. This process ensures that the defect information in the image is not affected by lighting conditions and changes in surface geometry, providing high-quality input data.

[0077] After image processing and data fusion are completed, the system enters the feature extraction and high-dimensional data analysis stage. First, the feature extraction unit extracts key features related to defects from the optimized image data. These features may include surface texture, color change, temperature distribution, etc. Then, high-dimensional data analysis methods (such as principal component analysis PCA and t-SNE) are used to reduce the dimensionality of the extracted features, remove redundant information, and retain key information that is crucial to defect detection. This process helps to reduce the computational burden of subsequent deep learning models while improving the expressiveness and accuracy of features.

[0078] Finally, the data after feature extraction and dimensionality reduction will be input into the deep learning detection module, which uses deep learning algorithms such as convolutional neural networks (CNN) to classify, identify and locate defects in the image. By training a large amount of labeled data, the deep learning model can automatically identify various types of defects from the feature data, such as cracks, bubbles, scratches, etc., and accurately locate them according to their shape and position. The output of the model includes defect type and location coordinates, which can be used for subsequent quality control and production optimization.

[0079] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. High-precision AOI detection system for surface defects of complex industrial products, characterized by: include: Multimodal sensor module, used to collect multi-dimensional image data of product surface; An image processing and data fusion module, used for performing information optimization, geometric correction and illumination correction on the raw image data collected by the multimodal sensor module; A feature extraction and high-dimensional data analysis module, used to perform feature extraction and dimensionality reduction analysis on the data output by the image processing and data fusion module, and retain key information related to defect detection; A deep learning detection module is used to extract and analyze feature information based on the feature information output by the feature extraction and high-dimensional data analysis module.

2. The high-precision AOI detection system for surface defects of complex industrial products according to claim 1, characterized in that: The multimodal sensor module comprises: Visible light sensor, used to collect regular images of product surfaces; Infrared imaging sensors to capture information about surface defects caused by temperature changes; Hyperspectral imaging sensors, used to collect reflection information at different wavelengths; 3D depth sensors are used to collect three-dimensional information on the surface of products with complex geometric shapes.

3. The high-precision AOI detection system for surface defects of complex industrial products according to claim 1, characterized in that: The image processing and data fusion module includes: Optical information maximization unit, used to optimize camera parameters, light source position and imaging angle according to Shannon information theory to maximize the mutual information of defect information; The geometry and illumination correction unit is used to perform geometry correction and illumination optimization on the collected image data based on reflection geometry theory and radiation transfer equation RTE.

4. The high-precision AOI detection system for surface defects of complex industrial products according to claim 3, characterized in that: The optical information maximization unit calculates the mutual information based on the following formula: ； in: Represents a random variable and The mutual information between and Denote random variables and Entropy of express and The joint entropy of .

5. The high-precision AOI detection system for surface defects of complex industrial products according to claim 3, characterized in that: The geometry and illumination correction unit performs illumination correction based on the following reflection model: Lambertian reflection model: ； in: Indicates the surface at an angle Reflectivity under is the reflection coefficient; is the angle between the incident light and the surface normal; Phong reflection model: ； in: The surface is at an incident angle and reflection angle Specular reflectivity under ; is the specular reflection coefficient; The specular reflection index controls the smoothness of the reflected light.

6. The high-precision AOI detection system for surface defects of complex industrial products according to claim 1, characterized in that: The feature extraction and high-dimensional data analysis module includes: Principal component analysis (PCA) unit, used to reduce the dimension of multimodal sensor data and extract the main feature directions; t-Distributed Stochastic Neighbor Embedding (t-SNE), which is used to preserve the local structure between high-dimensional data points; Data fusion unit is used to perform weighted averaging of features from different sensors.

7. The high-precision AOI detection system for surface defects of complex industrial products according to claim 6, characterized in that: The data fusion unit performs weighted averaging on the feature data based on the following formula: ； in: is the fused feature; For the Features extracted by sensors; is the weight of each sensor feature.

8. The high-precision AOI detection system for surface defects of complex industrial products according to claim 1, characterized in that: The deep learning detection module includes: Convolutional neural network (CNN) unit, used to automatically classify and locate defects from the extracted features; Adaptive optimization unit, dynamically adjusts the weights of the CNN model based on the Adam optimizer.

9. A high-precision AOI detection method for surface defects of complex industrial products, applied to a high-precision AOI detection system for surface defects of complex industrial products as claimed in any one of claims 1 to 8, characterized in that: The following steps are involved: S1. Collect multi-dimensional image data of the product surface using a multimodal sensor module; S2, optimizing, geometrically correcting and lighting correcting the multi-dimensional image data through an image processing and data fusion module; S3. Extract key information of defects through feature extraction and high-dimensional data analysis modules, and perform dimensionality reduction on the data; S4. Use the deep learning detection module to classify, identify and locate defects based on the extracted feature information.

10. The high-precision AOI detection method for surface defects of complex industrial products according to claim 9, characterized in that: The specific steps of the S2 step optimization, geometric correction and illumination correction are as follows: The optical information maximization unit optimizes the camera parameters, light source position and imaging angle to maximize the mutual information of defect information; The illumination correction is performed by the geometry and illumination correction unit based on the reflection geometry model and the radiation transfer equation RTE.

Images