Industrial product visual inspection system and method based on image processing

Abstract

The invention discloses an industrial product visual inspection system and method based on image processing, and relates to the technical field of industrial product visual inspection. According to the invention, through improving the SIFT-PSO registration algorithm and combining the hyperspectral, thermal imaging and 3D point cloud data acquired by multiple sensors cooperatively, multi-modal image registration with pixel-level precision is realized; an adaptive illumination compensation and noise suppression technology is adopted, so that the image quality is improved; by improving an LBP-TOP algorithm and combining geometric curvature features and heat distribution features, multi-dimensional information feature expression is constructed, and calculation load is reduced through a feature compression technology; a ResNet model is constructed, a network weight and a transfer learning mechanism are dynamically optimized through a genetic algorithm, and the accuracy and generalization ability of defect detection are improved; the method is suitable for product defect detection in a complex industrial environment, has the characteristics of high robustness, high precision and efficient processing, and realizes double breakthrough of detection precision and real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of industrial product visual inspection technology, and specifically relates to an industrial product visual inspection system and method based on image processing. Background Technology

[0002] With the deepening of global industrialization, the quality requirements for industrial products during production and processing are becoming increasingly stringent. Quality inspection, as a crucial link in ensuring product qualification, directly impacts a company's production costs and market competitiveness in terms of accuracy and efficiency. Among numerous quality inspection methods, image processing-based automatic defect detection technology has gradually become a research hotspot due to its advantages such as non-contact operation, high precision, and real-time monitoring. This technology acquires images of industrial products and uses computer vision algorithms to analyze these images to identify and locate product defects.

[0003] Chinese patent CN116385418B discloses an image processing method and a silicon wafer stacking detection system. The invention acquires images, locates the detection area, calculates the vertex coordinates of the detection area, generates several mask areas, bevels several wafers, performs wafer-by-wafer detection using a Foreach loop, classifies defects by sampling rules, and finally outputs the defect results.

[0004] Existing processing methods are sensitive to light and difficult to adapt to complex industrial environments; single visual modalities cannot fully capture product defect features; generalization ability is insufficient in detecting minute defects; and the real-time performance of the detection system is insufficient, making it difficult to meet the needs of high-speed production lines. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the problems in related technologies, this invention provides an image processing-based visual inspection method for industrial products. This invention solves the problems of sensitivity to light, inability of a single visual modality to fully capture product defect features, insufficient generalization ability in detecting minute defects, and insufficient real-time performance by using multi-sensor technology, image processing technology, and machine learning technology.

[0007] (II) Technical Solution

[0008] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0009] S1. Improve the SIFT-PSO registration algorithm to obtain an improved SIFT-PSO registration algorithm; use multiple sensors to collect images of industrial products to obtain multimodal images; use the improved SIFT-PSO registration algorithm to perform pixel-level alignment on the multimodal images to obtain registered multimodal images;

[0010] S2. Adaptive illumination compensation and noise suppression are performed on the registered multimodal images to obtain high-quality multimodal images;

[0011] S3. Improve the traditional feature extraction algorithm to obtain an improved traditional feature extraction algorithm; use the improved traditional feature extraction algorithm and the convolutional neural network model respectively to extract industrial product features from high-quality multimodal images to obtain traditional features and deep features; perform feature fusion on the traditional features and deep features to obtain fused industrial product features;

[0012] S4. Construct a ResNet defect detection model. Combine historical industrial product characteristics, historical industrial product defect data, and optimization algorithms to adjust the parameters of the ResNet defect detection model to obtain an optimized ResNet defect detection model. Use the optimized ResNet defect detection model and the fused industrial product characteristics to detect defects in industrial products.

[0013] Preferably, step S1 includes the following steps:

[0014] S11. Improve the SIFT-PSO registration algorithm to obtain the improved SIFT-PSO registration algorithm;

[0015] S12. Use a unified clock signal to control a hyperspectral camera, a 3D scanner, and a thermal imager to collect hyperspectral images, 3D point cloud projection images, and thermal images of industrial products to obtain multimodal images.

[0016] S13. Use the improved SIFT-PSO registration algorithm to extract feature points from the hyperspectral image, 3D point cloud projection image and thermal image of the multimodal image respectively, solve the optimal homography matrix, minimize the feature point alignment error, and obtain the registered multimodal image.

[0017] The above steps, through the improved SIFT-PSO registration algorithm combined with multi-sensor synchronous acquisition and pixel-level alignment, achieve high-precision multimodal data fusion in complex industrial scenarios. Among them, the synchronous acquisition of multi-source data with unified clock control solves the spatiotemporal misalignment problem of multimodal data in traditional methods. The solution of the optimal homography matrix ensures the pixel-level alignment accuracy of hyperspectral, 3D point cloud and thermal imaging data, laying a reliable foundation for subsequent cross-modal feature fusion.

[0018] Preferably, step S11 includes the following steps:

[0019] S111. Perform bidirectional matching on SIFT feature points and retain only bidirectionally consistent matching point pairs; set a scale difference threshold to filter out abnormal matching points with a scale difference ≥ the scale difference threshold.

[0020] S112. Set the inertial weight in the particle velocity update formula to decay with the number of iterations; the inertial parameter update formula is as follows.

[0021]

[0022] Where w(t) represents the inertia weight at the t-th iteration, w max w min T represents the maximum and minimum values ​​of the inertia weight, respectively. max The maximum number of iterations is represented by t, and the current number of iterations is represented by t.

[0023] The particle retention parameters are set based on the feature points of the industrial product image; the hybrid local search cycle is set to a times, and LM local optimization is performed on the globally optimal particle every a iterations;

[0024] S113. During the optimization process, the determinant of the homography matrix submatrix is ​​forced to be non-zero, and a geometric distortion penalty term is added to the fitness function; the fitness calculation task of the particle swarm is allocated to the GPU for parallel processing, and the feature point data is transferred to the GPU memory in blocks.

[0025] S114. The improved SIFT-PSO registration algorithm is obtained through steps S111, S112 and S113.

[0026] The above steps construct the SIFT-PSO registration algorithm through bidirectional matching and scale filtering, dynamic inertia weight optimization, and geometric constraint enhancement, significantly improving image registration performance in industrial scenarios. It solves problems such as high matching redundancy, low optimization efficiency, and susceptibility to local optima in traditional SIFT-PSO registration algorithms. Specifically, the bidirectional consistent matching strategy combined with scale difference threshold filtering reduces mismatch points and enhances feature point reliability; the dynamically decaying inertia weight design, combined with periodic LM local search, improves the algorithm's convergence speed and avoids local optima; the combined application of forced non-zero determinant constraints and geometric distortion penalty terms reduces registration errors; and the GPU parallel computing architecture provides algorithmic assurance for high-precision alignment of multimodal data.

[0027] Preferably, step S2 includes the following steps:

[0028] S21. Collect reflectivity data from the surface of industrial products to construct a surface reflectivity matrix A. Collect polarizer parameters b to construct a light intensity control model for the ring LED array. Combine this with the registered multimodal image to obtain a compensated multimodal image I. The formula for the light intensity control model is as follows.

[0029]

[0030] Where θ represents the incident angle compensation factor, and α and β both represent the model parameters of the light intensity control model;

[0031] S22. Perform a non-subsampled contourlet transform on the compensated image to obtain a high-quality multimodal image; the formula for the non-subsampled contourlet transform is as follows.

[0032]

[0033] Where H represents a high-quality multimodal image, K represents the number of scales, k represents the index variable, and w k Represents scale-adaptive weights, NCXT, NCXT -1 T represents the non-subsampled contour wave transform and its inverse transform, respectively. λ Represents the threshold function;

[0034] The above steps construct an adaptive illumination compensation system through surface reflection modeling and non-subsampling multi-scale processing, effectively solving the illumination interference problem in complex industrial scenarios. Based on the surface reflectivity matrix and polarization parameters, a ring LED light intensity control model achieves real-time compensation by dynamically adjusting the θ, α, and β parameters, reducing specular artifacts. The non-subsampling contour wave transform uses scale-adaptive weights and a threshold function in synergy, improving the noise signal-to-noise ratio while preserving detailed features. This enables multimodal images to meet industrial inspection standards in terms of texture clarity and defect contrast, providing a high-quality data foundation for subsequent micro-defect feature extraction.

[0035] Preferably, step S3 includes the following steps:

[0036] S31. Improve the LBP-TOP algorithm to obtain the improved LBP-TOP algorithm; the improved LBP-TOP algorithm, the geometric feature extraction algorithm, and the thermal feature extraction algorithm together constitute the improved traditional feature extraction algorithm; use the improved LBP-TOP algorithm to extract features from the hyperspectral map in the high-quality multimodal image to obtain texture features;

[0037] S32. Using a geometric feature extraction algorithm, feature extraction is performed on the 3D point cloud projection map in the high-quality multimodal image to obtain curvature value features; using a thermal feature extraction algorithm, feature extraction is performed on the thermal imaging map in the high-quality multimodal image to obtain temperature distribution features; texture features, curvature value features, and temperature distribution features together constitute traditional features;

[0038] S33. Use a convolutional neural network model to extract industrial product features from high-quality multimodal images to obtain deep features;

[0039] S34. Perform feature fusion between traditional features and deep features to obtain fused industrial product features;

[0040] The above steps achieve cross-dimensional feature representation for industrial defect detection through multimodal feature fusion; the improved LBP-TOP algorithm enhances texture feature resolution and, combined with the collaborative extraction of 3D curvature features and thermal distribution features, constructs a traditional feature set covering surface morphology, material properties, and thermodynamic behavior; the deep convolutional network captures the implicit correlation features of subtle defects through end-to-end learning; random projection dimensionality reduction technology is used during the fusion process to retain effective information while reducing feature dimensions, and the final fused features combine the physical interpretability of traditional algorithms with the abstract expression capabilities of deep learning, improving the detection accuracy of complex defects such as microcracks and material heterogeneity.

[0041] Preferably, the improvement of the LBP-TOP algorithm in step S31 to obtain the improved LBP-TOP algorithm includes the following steps:

[0042] S311. Based on the size of the hyperspectral image, set an adaptive neighborhood radius; adjust the sampling point positions using a Gaussian distribution to enhance the density in the central region;

[0043] S312. Fuse the gradient magnitudes of pixels and rotate the neighborhood to the main gradient direction; calculate the LBP value for the rotated neighborhood and record the main direction as an additional feature; construct a multi-scale image using a Gaussian pyramid; calculate LBP-TOP features for multiple images and fuse them to obtain high-dimensional features;

[0044] S313. High-dimensional features are compressed into binary codes through random projection to obtain binary code features. Principal component analysis is performed on the binary code features to obtain the final features.

[0045] S314. The improved LBP-TOP algorithm is obtained through steps S311, S312 and S313;

[0046] The above steps significantly improve the expressive power of industrial texture features by modifying the LBP-TOP algorithm, solving the problems of the traditional LBP-TOP algorithm, such as the inability of fixed radius to adapt to texture features of different scales, ignoring gradient direction information, single-scale features, and low computational efficiency. The adaptive neighborhood radius and Gaussian distribution sampling point design make the feature extraction resolution more accurate and improve the sampling density in the central region. The gradient direction rotation mechanism reduces the direction sensitivity error through principal direction calibration, and combined with the Gaussian pyramid multi-scale fusion strategy, it realizes multi-level feature capture of defects. The joint dimensionality reduction of random projection and principal component analysis compresses the feature dimension while retaining more effective information, thus improving computational efficiency. The improved LBP-TOP algorithm improves the detection accuracy of defects such as surface scratches and material fatigue.

[0047] Preferably, step S4 includes the following steps:

[0048] S41. Construct a ResNet defect detection model, setting the initial weights of the ResNet defect detection model to c1, the proportion of training data to d1, and the proportion of test data to d2.

[0049] S42. Collect a large number of images of historical industrial products, and perform feature extraction and fusion to obtain historical industrial product features; collect product defects corresponding to historical industrial product features to obtain historical industrial product defect data; divide historical industrial product features according to the training data ratio d1 and the test data ratio d2 to obtain historical industrial product training features and historical industrial product test features; divide historical industrial product defect data to obtain historical industrial product defect training data and historical industrial product defect test data.

[0050] S43. Set the training error threshold e1 and the maximum number of training iterations; use historical industrial product training features and historical industrial product defect training data to repeatedly train the ResNet defect detection model; after each round of training, calculate the training error e2 of the ResNet defect detection model and adjust the initial weights c1 according to the training error.

[0051] When e2≤e1 or the maximum number of iterations is reached, an intermediate ResNet defect detection model with initial weights c2 is obtained;

[0052] S44. Using historical industrial product test features and historical industrial product defect test data, test the intermediate ResNet defect detection model. After the test is completed, an optimized ResNet defect detection model is obtained.

[0053] S45. Input the integrated industrial product features into the optimized ResNet defect detection model to obtain industrial product defect data;

[0054] The above steps achieve an intelligent upgrade of industrial defect detection by constructing an optimized ResNet detection model; the transfer learning mechanism based on historical data combined with a dynamic weight adjustment strategy improves the model training convergence speed; the introduction of a genetic algorithm to optimize and overcome the local optimum limitation of traditional gradient descent improves the detection accuracy; and the fusion of industrial product features is input into the optimized ResNet defect detection model to obtain industrial product defect data.

[0055] Preferably, S44 includes the following steps:

[0056] S441. Set the test accuracy threshold f1, use historical industrial product test features and historical industrial product defect test data to test the intermediate ResNet defect detection model, and obtain the test accuracy f2.

[0057] S442. If f2≥f1, then the intermediate ResNet defect detection model is used as the optimized ResNet defect detection model; otherwise, a genetic algorithm is used to find the initial weights of the intermediate ResNet defect detection model, obtain the global optimal solution, and use the global optimal solution as the initial weights of the intermediate ResNet defect detection model to obtain the optimized ResNet defect detection model.

[0058] The above steps determine whether the intermediate ResNet defect detection model has achieved the expected performance by setting a test accuracy threshold and using historical data to test the model. If the threshold is reached, the model is directly adopted to ensure detection efficiency. If the threshold is not reached, a genetic algorithm is used to optimize the initial weights and seek the global optimal solution, thereby improving the model performance. This ensures the high accuracy and strong generalization ability of the optimized ResNet defect detection model, thus improving the defect detection capability of industrial products.

[0059] Preferably, the use of a genetic algorithm in step S442 to find the initial weights of the intermediate ResNet defect detection model and obtain the globally optimal solution includes the following steps:

[0060] S4421. Construct a chromosome population. Let the size of the chromosome population be h. Then, the chromosome population is represented as k = {k1, k2, ..., k}. i ,...,k h}, where k i Let u1 represent the i-th chromosome in the chromosome population. Each chromosome represents an initial weight of the intermediate ResNet defect detection model. The maximum number of test iterations is set to u1.

[0061] S4422. Substitute the initial weights in the chromosome population into the intermediate ResNet defect detection model, and use historical industrial product test features and historical industrial product defect test data to test the intermediate ResNet defect detection model to obtain the training accuracy. Based on the training accuracy, perform crossover and mutation operations on the chromosomes in the chromosome population to obtain the chromosome population after crossover and mutation operations.

[0062] S4423. Repeat S4422. Stop iterating when f2≥f1 or the maximum number of test iterations is reached, and obtain the global optimal solution.

[0063] The above steps utilize genetic algorithms to overcome the bottlenecks in traditional model training, achieving a breakthrough improvement in detection accuracy. The chromosome-encoded global weight search mechanism covers a larger solution space with a large population, and the crossover and mutation strategy, through elite retention and adaptive probability design, enables the model to quickly approach the global optimum. The dynamic feedback mechanism of weight optimization and test accuracy improves the defect detection accuracy and reduces the risk of local optima.

[0064] An image processing-based industrial product visual inspection system is used to implement the above-mentioned image processing-based industrial product visual inspection method, including an improved SIFT-PSO registration algorithm module, a multimodal image alignment module, an adaptive illumination compensation and noise suppression module, a multimodal feature extraction and fusion module, and a ResNet defect detection model optimization and defect recognition module.

[0065] The improved SIFT-PSO registration algorithm module is used to filter out abnormal feature points through bidirectional matching and scale difference threshold, dynamically adjust PSO inertia weight, optimize particle swarm by combining periodic LM local search, introduce geometric distortion penalty term to constrain homography matrix, and use GPU parallel acceleration of fitness calculation to obtain the improved SIFT-PSO registration algorithm.

[0066] The multimodal image alignment module is used to simultaneously acquire images from a hyperspectral camera, a 3D scanner, and a thermal imager to obtain multimodal images of industrial products, and then combine them with an improved SIFT-PSO registration algorithm module to obtain registered multimodal images.

[0067] The adaptive illumination compensation and noise suppression module is used to construct a ring LED light intensity control model based on the surface reflectivity matrix and polarization parameters, optimize image illumination through incident angle compensation factor, and perform multi-scale decomposition and threshold denoising using non-subsampled contour wave transform to obtain high-quality multimodal images.

[0068] The multimodal feature extraction and fusion module is used to improve the LBP-TOP algorithm by introducing adaptive neighborhood, gradient direction fusion and multi-scale feature compression; extract the texture features of hyperspectral images, the curvature features of 3D point clouds and the temperature distribution features of thermal images to obtain infectious features; use convolutional neural networks to extract the depth features of multimodal images; and fuse traditional features with depth features to obtain fused industrial product features.

[0069] The ResNet defect detection model optimization and defect identification module is used to initialize ResNet weights, divide historical data into training and testing sets, adjust weights through iterative training to reach the error threshold or the maximum number of iterations, and obtain an optimized ResNet defect detection model by combining genetic algorithms; the fused features are input into the optimized ResNet defect detection model, and the industrial product defect data are output.

[0070] (III) Beneficial Effects

[0071] The present invention has the following beneficial effects:

[0072] This invention significantly improves the robustness of detection in complex industrial environments by using an improved SIFT-PSO registration algorithm and multimodal fusion technology. Addressing the sensitivity of existing technologies to illumination, it employs a bidirectional matching strategy and scale difference threshold filtering to filter abnormal matching points. Combined with a GPU parallel computing architecture to accelerate the registration process and adaptive illumination compensation and noise suppression, it improves processing speed while maintaining pixel-level alignment accuracy. Hyperspectral, thermal imaging, and 3D point cloud data acquired collaboratively by multiple sensors form complementary features through pixel-level registration, enabling the complete presentation of complex features such as three-dimensional deformation, material anomalies, and thermal conduction anomalies of surface defects, thus solving the problem of insufficient coverage of single-modal features.

[0073] The feature extraction system of this invention effectively improves the detection capability of minute defects; the improved LBP-TOP algorithm enhances the texture feature resolution capability through Gaussian distribution sampling point optimization and multi-scale feature fusion, and constructs a feature representation containing spatial-spectral-thermodynamic multi-dimensional information by combining cross-modal fusion of geometric curvature features and thermal distribution features; the feature compression technology achieves intelligent reduction of feature dimensions through random projection and principal component analysis, reducing the computational load while retaining effective information, laying the foundation for efficient processing of subsequent deep learning models.

[0074] The optimized ResNet model constructed in this invention breaks through the generalization bottleneck of traditional detection methods. By introducing a genetic algorithm to dynamically optimize network weights and combining it with a transfer learning mechanism based on historical defect data, the model reduces the false detection rate while maintaining detection accuracy. The unique geometric distortion penalty term design and hybrid local search strategy enhance the model's adaptability to product shape variations, thereby improving the defect detection rate compared to traditional methods.

[0075] The overall system architecture of this invention achieves a dual breakthrough in detection accuracy and real-time performance; the combined application of the ring LED light intensity control model and non-subsampled contour wave transformation improves the response speed of illumination compensation, and the parallel processing pipeline design of multimodal data shortens the overall detection cycle of the system; effectively solving the technical contradiction of difficulty in achieving both speed and accuracy in industrial quality inspection.

[0076] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0077] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0078] Figure 1This is a flowchart illustrating an image processing-based visual inspection method for industrial products according to the present invention.

[0079] Figure 2 This is a schematic diagram of a module of an industrial product visual inspection system based on image processing according to the present invention. Detailed Implementation

[0080] The technical solutions of the embodiments of the invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the invention, and not all embodiments. Based on the embodiments of the invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the invention.

[0081] In the description of this invention, it should be understood that the terms "opening", "upper", "lower", "top", "middle", "inner", etc., which indicate orientation or positional relationship, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the invention.

[0082] Example 1:

[0083] Please see Figure 1 This invention discloses an image processing-based visual inspection method for industrial products, comprising the following steps:

[0084] S1. Improve the SIFT-PSO registration algorithm to obtain an improved SIFT-PSO registration algorithm; use multiple sensors to collect images of industrial products to obtain multimodal images; use the improved SIFT-PSO registration algorithm to perform pixel-level alignment on the multimodal images to obtain registered multimodal images;

[0085] S1 includes the following steps:

[0086] S11. Improve the SIFT-PSO registration algorithm to obtain the improved SIFT-PSO registration algorithm;

[0087] S11 includes the following steps:

[0088] S111. Perform bidirectional matching on SIFT feature points and retain only bidirectionally consistent matching point pairs; set a scale difference threshold to filter out abnormal matching points with a scale difference ≥ the scale difference threshold.

[0089] S112. Set the inertial weight in the particle velocity update formula to decay with the number of iterations; the inertial parameter update formula is as follows.

[0090]

[0091] Where w(t) represents the inertia weight at the t-th iteration, w max w min T represents the maximum and minimum values ​​of the inertia weight, respectively. max The maximum number of iterations is represented by t, and the current number of iterations is represented by t.

[0092] The particle retention parameters are set based on the feature points of the industrial product image; the hybrid local search cycle is set to a times, and LM local optimization is performed on the globally optimal particle every a iterations;

[0093] S113. During the optimization process, the determinant of the homography matrix submatrix is ​​forced to be non-zero, and a geometric distortion penalty term is added to the fitness function; the fitness calculation task of the particle swarm is allocated to the GPU for parallel processing, and the feature point data is transferred to the GPU memory in blocks.

[0094] S114. The improved SIFT-PSO registration algorithm is obtained through steps S111, S112 and S113.

[0095] S12. Use a unified clock signal to control a hyperspectral camera, a 3D scanner, and a thermal imager to collect hyperspectral images, 3D point cloud projection images, and thermal images of industrial products to obtain multimodal images.

[0096] S13. Use the improved SIFT-PSO registration algorithm to extract feature points from the hyperspectral image, 3D point cloud projection image and thermal image of the multimodal image respectively, solve the optimal homography matrix, minimize the feature point alignment error, and obtain the registered multimodal image.

[0097] S2. Adaptive illumination compensation and noise suppression are performed on the registered multimodal images to obtain high-quality multimodal images;

[0098] S2 includes the following steps:

[0099] S21. Collect reflectivity data from the surface of industrial products to construct a surface reflectivity matrix A. Collect polarizer parameters b to construct a light intensity control model for the ring LED array. Combine this with the registered multimodal image to obtain a compensated multimodal image I. The formula for the light intensity control model is as follows.

[0100]

[0101] Where θ represents the incident angle compensation factor, and α and β both represent the model parameters of the light intensity control model;

[0102] S22. Perform a non-subsampled contourlet transform on the compensated image to obtain a high-quality multimodal image; the formula for the non-subsampled contourlet transform is as follows.

[0103]

[0104] Where H represents a high-quality multimodal image, K represents the number of scales, k represents the index variable, and w k Represents scale-adaptive weights, NCXT, NCXT -1 T represents the non-subsampled contour wave transform and its inverse transform, respectively. λ Represents the threshold function;

[0105] S3. Improve the traditional feature extraction algorithm to obtain an improved traditional feature extraction algorithm; use the improved traditional feature extraction algorithm and the convolutional neural network model respectively to extract industrial product features from high-quality multimodal images to obtain traditional features and deep features; perform feature fusion on the traditional features and deep features to obtain fused industrial product features;

[0106] S3 includes the following steps:

[0107] S31. Improve the LBP-TOP algorithm to obtain the improved LBP-TOP algorithm; the improved LBP-TOP algorithm, the geometric feature extraction algorithm, and the thermal feature extraction algorithm together constitute the improved traditional feature extraction algorithm; use the improved LBP-TOP algorithm to extract features from the hyperspectral map in the high-quality multimodal image to obtain texture features;

[0108] The improved LBP-TOP algorithm described in step S31 includes the following steps:

[0109] S311. Based on the size of the hyperspectral image, set an adaptive neighborhood radius; adjust the sampling point positions using a Gaussian distribution to enhance the density in the central region;

[0110] S312. Fuse the gradient magnitudes of pixels and rotate the neighborhood to the main gradient direction; calculate the LBP value for the rotated neighborhood and record the main direction as an additional feature; construct a multi-scale image using a Gaussian pyramid; calculate LBP-TOP features for multiple images and fuse them to obtain high-dimensional features;

[0111] S313. High-dimensional features are compressed into binary codes through random projection to obtain binary code features. Principal component analysis is performed on the binary code features to obtain the final features.

[0112] S314. The improved LBP-TOP algorithm is obtained through steps S311, S312 and S313;

[0113] S32. Using a geometric feature extraction algorithm, feature extraction is performed on the 3D point cloud projection map in the high-quality multimodal image to obtain curvature value features; using a thermal feature extraction algorithm, feature extraction is performed on the thermal imaging map in the high-quality multimodal image to obtain temperature distribution features; texture features, curvature value features, and temperature distribution features together constitute traditional features;

[0114] S33. Use a convolutional neural network model to extract industrial product features from high-quality multimodal images to obtain deep features;

[0115] S34. Perform feature fusion between traditional features and deep features to obtain fused industrial product features;

[0116] S4. Construct a ResNet defect detection model, combine historical industrial product characteristics, historical industrial product defect data and optimization algorithms to adjust the parameters of the ResNet defect detection model to obtain an optimized ResNet defect detection model, and use the optimized ResNet defect detection model and the fused industrial product characteristics to detect defects in industrial products.

[0117] The S4 step is as follows:

[0118] S41. Construct a ResNet defect detection model, setting the initial weights of the ResNet defect detection model to c1, the proportion of training data to d1, and the proportion of test data to d2.

[0119] S42. Collect a large number of images of historical industrial products, and perform feature extraction and fusion to obtain historical industrial product features; collect product defects corresponding to historical industrial product features to obtain historical industrial product defect data; divide historical industrial product features according to the training data ratio d1 and the test data ratio d2 to obtain historical industrial product training features and historical industrial product test features; divide historical industrial product defect data to obtain historical industrial product defect training data and historical industrial product defect test data.

[0120] S43. Set the training error threshold e1 and the maximum number of training iterations; use historical industrial product training features and historical industrial product defect training data to repeatedly train the ResNet defect detection model; after each round of training, calculate the training error e2 of the ResNet defect detection model and adjust the initial weights c1 according to the training error.

[0121] When e2≤e1 or the maximum number of iterations is reached, an intermediate ResNet defect detection model with initial weights c2 is obtained;

[0122] S44. Using historical industrial product test features and historical industrial product defect test data, test the intermediate ResNet defect detection model. After the test is completed, an optimized ResNet defect detection model is obtained.

[0123] S44 includes the following steps:

[0124] S441. Set the test accuracy threshold f1, use historical industrial product test features and historical industrial product defect test data to test the intermediate ResNet defect detection model, and obtain the test accuracy f2.

[0125] S442. If f2≥f1, then the intermediate ResNet defect detection model is used as the optimized ResNet defect detection model; otherwise, a genetic algorithm is used to find the initial weights of the intermediate ResNet defect detection model, obtain the global optimal solution, and use the global optimal solution as the initial weights of the intermediate ResNet defect detection model to obtain the optimized ResNet defect detection model.

[0126] The S442 step uses a genetic algorithm to find the initial weights of the intermediate ResNet defect detection model and obtain the global optimal solution, including the following steps:

[0127] S4421. Construct a chromosome population. Let the size of the chromosome population be h. Then, the chromosome population is represented as k = {k1, k2, ..., k}. i ,...,k h}, where k i Let u1 represent the i-th chromosome in the chromosome population. Each chromosome represents an initial weight of the intermediate ResNet defect detection model. The maximum number of test iterations is set to u1.

[0128] S4422. Substitute the initial weights in the chromosome population into the intermediate ResNet defect detection model, and use historical industrial product test features and historical industrial product defect test data to test the intermediate ResNet defect detection model to obtain the training accuracy. Based on the training accuracy, perform crossover and mutation operations on the chromosomes in the chromosome population to obtain the chromosome population after crossover and mutation operations.

[0129] S4423. Repeat S4422. Stop iterating when f2≥f1 or the maximum number of test iterations is reached, and obtain the global optimal solution.

[0130] S45. Input the integrated industrial product features into the optimized ResNet defect detection model to obtain industrial product defect data.

[0131] Example 2:

[0132] Please see Figure 2 This invention discloses an industrial product visual inspection system based on image processing, used to implement the above-mentioned industrial product visual inspection method based on image processing, including an improved SIFT-PSO registration algorithm module, a multimodal image alignment module, an adaptive illumination compensation and noise suppression module, a multimodal feature extraction and fusion module, and a ResNet defect detection model optimization and defect recognition module;

[0133] The improved SIFT-PSO registration algorithm module is used to filter out abnormal feature points through bidirectional matching and scale difference threshold, dynamically adjust PSO inertia weight, optimize particle swarm by combining periodic LM local search, introduce geometric distortion penalty term to constrain homography matrix, and use GPU parallel acceleration of fitness calculation to obtain the improved SIFT-PSO registration algorithm.

[0134] The multimodal image alignment module is used to simultaneously acquire images from a hyperspectral camera, a 3D scanner, and a thermal imager to obtain multimodal images of industrial products, and then combine them with an improved SIFT-PSO registration algorithm module to obtain registered multimodal images.

[0135] The adaptive illumination compensation and noise suppression module is used to construct a ring LED light intensity control model based on the surface reflectivity matrix and polarization parameters, optimize image illumination through incident angle compensation factor, and perform multi-scale decomposition and threshold denoising using non-subsampled contour wave transform to obtain high-quality multimodal images.

[0136] The multimodal feature extraction and fusion module is used to improve the LBP-TOP algorithm by introducing adaptive neighborhood, gradient direction fusion and multi-scale feature compression; extract the texture features of hyperspectral images, the curvature features of 3D point clouds and the temperature distribution features of thermal images to obtain infectious features; use convolutional neural networks to extract the depth features of multimodal images; and fuse traditional features with depth features to obtain fused industrial product features.

[0137] The ResNet defect detection model optimization and defect identification module is used to initialize ResNet weights, divide historical data into training and testing sets, adjust weights through iterative training to reach the error threshold or the maximum number of iterations, and obtain an optimized ResNet defect detection model by combining genetic algorithms; the fused features are input into the optimized ResNet defect detection model, and the industrial product defect data are output.

[0138] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0139] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A visual inspection method for industrial products based on image processing, characterized in that, Includes the following steps: S1. Improve the SIFT-PSO registration algorithm to obtain an improved SIFT-PSO registration algorithm; use multiple sensors to collect images of industrial products to obtain multimodal images; use the improved SIFT-PSO registration algorithm to perform pixel-level alignment on the multimodal images to obtain registered multimodal images; S2. Adaptive illumination compensation and noise suppression are performed on the registered multimodal images to obtain high-quality multimodal images; S3. Improve the traditional feature extraction algorithm to obtain an improved traditional feature extraction algorithm; An improved traditional feature extraction algorithm and a convolutional neural network model are used to extract industrial product features from high-quality multimodal images, resulting in traditional features and deep features. Feature fusion is then performed on the traditional features and deep features to obtain fused industrial product features. S4. Construct a ResNet defect detection model. Combine historical industrial product characteristics, historical industrial product defect data, and optimization algorithms to adjust the parameters of the ResNet defect detection model to obtain an optimized ResNet defect detection model. Use the optimized ResNet defect detection model and the fused industrial product characteristics to detect defects in industrial products.

2. The image processing-based visual inspection method for industrial products according to claim 1, characterized in that, S1 includes the following steps: S11. Improve the SIFT-PSO registration algorithm to obtain the improved SIFT-PSO registration algorithm; S12. Use a unified clock signal to control a hyperspectral camera, a 3D scanner, and a thermal imager to collect hyperspectral images, 3D point cloud projection images, and thermal images of industrial products to obtain multimodal images. S13. Use the improved SIFT-PSO registration algorithm to extract feature points from the multimodal images, solve for the optimal homography matrix, minimize the feature point alignment error, and obtain the registered multimodal images.

3. The image processing-based visual inspection method for industrial products according to claim 2, characterized in that, S11 includes the following steps: S111. Perform bidirectional matching on SIFT feature points and retain only bidirectionally consistent matching point pairs; set a scale difference threshold to filter out abnormal matching points with a scale difference ≥ the scale difference threshold. S112. Set the inertial weight in the particle velocity update formula to decay with the number of iterations; the inertial parameter update formula is as follows. Where w(t) represents the inertia weight at the t-th iteration, w max w min T represents the maximum and minimum values ​​of the inertia weight, respectively. max The maximum number of iterations is represented by t, and the current number of iterations is represented by t. The particle retention parameters are set based on the feature points of the industrial product image; the hybrid local search cycle is set to a times, and LM local optimization is performed on the globally optimal particle every a iterations; S113. During the optimization process, the determinant of the homography matrix submatrix is ​​forced to be non-zero, and a geometric distortion penalty term is added to the fitness function; the fitness calculation task of the particle swarm is allocated to the GPU for parallel processing, and the feature point data is transferred to the GPU memory in blocks. S114. The improved SIFT-PSO registration algorithm is obtained through steps S111, S112 and S113.

4. The image processing-based visual inspection method for industrial products according to claim 1, characterized in that, S2 includes the following steps: S21. Collect reflectivity data from the surface of industrial products to construct a surface reflectivity matrix A. Collect polarizer parameters b to construct a light intensity control model for the ring LED array. Combine this with the registered multimodal image to obtain a compensated multimodal image I. The formula for the light intensity control model is as follows. Where θ represents the incident angle compensation factor, and α and β both represent the model parameters of the light intensity control model; S22. Perform a non-subsampled contourlet transform on the compensated image to obtain a high-quality multimodal image; the formula for the non-subsampled contourlet transform is as follows. Where H represents a high-quality multimodal image, K represents the number of scales, k represents the index variable, and w k Represents scale-adaptive weights, NCXT, NCXT -1 T represents the non-subsampled contour wave transform and its inverse transform, respectively. λ This represents the threshold function.

5. The method for visual inspection of industrial products based on image processing according to claim 1, characterized in that, S3 includes the following steps: S31. Improve the LBP-TOP algorithm to obtain the improved LBP-TOP algorithm; the improved LBP-TOP algorithm, the geometric feature extraction algorithm, and the thermal feature extraction algorithm together constitute the improved traditional feature extraction algorithm; use the improved LBP-TOP algorithm to extract features from the hyperspectral map in the high-quality multimodal image to obtain texture features; S32. Using a geometric feature extraction algorithm, feature extraction is performed on the 3D point cloud projection map in the high-quality multimodal image to obtain curvature value features; using a thermal feature extraction algorithm, feature extraction is performed on the thermal imaging map in the high-quality multimodal image to obtain temperature distribution features; texture features, curvature value features, and temperature distribution features together constitute traditional features; S33. Use a convolutional neural network model to extract industrial product features from high-quality multimodal images to obtain deep features; S34. Perform feature fusion between traditional features and deep features to obtain fused industrial product features.

6. The image processing-based visual inspection method for industrial products according to claim 5, characterized in that, The improved LBP-TOP algorithm described in step S31 includes the following steps: S311. Based on the size of the hyperspectral image, set an adaptive neighborhood radius; adjust the sampling point positions using a Gaussian distribution to enhance the density in the central region; S312. Fuse the gradient magnitudes of pixels and rotate the neighborhood to the main gradient direction; calculate the LBP value for the rotated neighborhood and record the main direction as an additional feature; construct a multi-scale image using a Gaussian pyramid; calculate LBP-TOP features for multiple images and fuse them to obtain high-dimensional features; S313. High-dimensional features are compressed into binary codes through random projection to obtain binary code features. Principal component analysis is performed on the binary code features to obtain the final features. S314. The improved LBP-TOP algorithm is obtained through steps S311, S312 and S313.

7. The image processing-based visual inspection method for industrial products according to claim 1, characterized in that, The S4 step is as follows: S41. Construct a ResNet defect detection model, setting the initial weights of the ResNet defect detection model to c1, the proportion of training data to d1, and the proportion of test data to d2. S42. Collect a large number of images of historical industrial products, and perform feature extraction and fusion to obtain the features of historical industrial products; Collect product defects corresponding to historical industrial product characteristics to obtain historical industrial product defect data; divide historical industrial product characteristics according to the training data ratio d1 and the test data ratio d2 to obtain historical industrial product training characteristics and historical industrial product test characteristics; divide historical industrial product defect data to obtain historical industrial product defect training data and historical industrial product defect test data. S43, Set the training error threshold e1 and the maximum number of training iterations; The ResNet defect detection model is repeatedly trained using historical industrial product training features and historical industrial product defect training data. After each round of training, the training error e2 of the ResNet defect detection model is calculated, and the initial weights c1 are adjusted according to the training error. When e2≤e1 or the maximum number of iterations is reached, an intermediate ResNet defect detection model with initial weights c2 is obtained; S44. Using historical industrial product test features and historical industrial product defect test data, test the intermediate ResNet defect detection model. After the test is completed, an optimized ResNet defect detection model is obtained. S45. Input the integrated industrial product features into the optimized ResNet defect detection model to obtain industrial product defect data.

8. The image processing-based visual inspection method for industrial products according to claim 7, characterized in that, S44 includes the following steps: S441. Set the test accuracy threshold f1, use historical industrial product test features and historical industrial product defect test data to test the intermediate ResNet defect detection model, and obtain the test accuracy f2. S442. If f2≥f1, then the intermediate ResNet defect detection model is used as the optimized ResNet defect detection model; otherwise, a genetic algorithm is used to find the initial weights of the intermediate ResNet defect detection model, obtain the global optimal solution, and use the global optimal solution as the initial weights of the intermediate ResNet defect detection model to obtain the optimized ResNet defect detection model.

9. The image processing-based visual inspection method for industrial products according to claim 8, characterized in that, The S442 step uses a genetic algorithm to find the initial weights of the intermediate ResNet defect detection model and obtain the global optimal solution, including the following steps: S4421. Construct a chromosome population. Let the size of the chromosome population be h. Then, the chromosome population is represented as k = {k1, k2, ..., k}. i ,...,k h }, where k i Let u1 represent the i-th chromosome in the chromosome population. Each chromosome represents an initial weight of the intermediate ResNet defect detection model. The maximum number of test iterations is set to u1. S4422. Substitute the initial weights in the chromosome population into the intermediate ResNet defect detection model, and use historical industrial product test features and historical industrial product defect test data to test the intermediate ResNet defect detection model to obtain the training accuracy. Based on the training accuracy, perform crossover and mutation operations on the chromosomes in the chromosome population to obtain the chromosome population after crossover and mutation operations. S4423. Repeat S4422. When f2≥f1 or the maximum number of test iterations is reached, stop the iteration and obtain the global optimal solution.

10. An image processing-based industrial product visual inspection system, used to implement the image processing-based industrial product visual inspection method as described in any one of claims 1-9, characterized in that: The system includes an improved SIFT-PSO registration algorithm module, a multimodal image alignment module, an adaptive illumination compensation and noise suppression module, a multimodal feature extraction and fusion module, and a ResNet defect detection model optimization and defect recognition module.

Images