Multi-field part size and appearance defect intelligent detection system

Abstract

The invention discloses a multi-field part size and appearance defect intelligent detection system, and relates to the field of image analysis. The system comprises an image acquisition module, an image preprocessing module, a feature extraction module, a defect identification and size measurement module, a data processing and analysis module, an automatic control module and a man-machine interaction module. According to the method, CNN and LBP, Hough transform and SIFT algorithms are fused, high-level semantics and bottom-level texture / geometric features are considered, 2304-dimensional fusion feature vectors are formed through feature splicing, the feature extraction integrity of parts in multiple fields is improved, texture detail features can be accurately extracted, geometric shape features can be accurately obtained, and the method is suitable for large-scale popularization and application. Meanwhile, the adaptive feature selection mechanism dynamically optimizes the feature combination according to the detection result, the problem of calculation redundancy is reduced, and the detection precision is ensured while the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to an intelligent detection system for component size and appearance defects in multiple fields. Background Art

[0002] In the field of modern industrial production, with the development of product refinement and diversification, the requirements for component dimensional accuracy and appearance quality are becoming increasingly stringent. The quality of components is directly related to the overall performance and reliability of the product. CCD vision-based inspection technology has been widely used in component inspection; However, traditional CCD vision-based detection algorithms such as single convolutional neural networks (CNN) have limitations in feature extraction. On the one hand, it is difficult to efficiently extract both texture details and geometric features at the same time. When detecting texture defects such as scratches on parts, CNN may not be able to accurately capture subtle texture changes. When measuring geometric features such as dimensional deviations, it is difficult to meet high-precision requirements. It has weak adaptability to different fields and different types of parts. On the other hand, traditional methods rely on fixed feature combinations and cannot dynamically optimize feature selection according to the differences in specific detection tasks, which can easily lead to waste of computing resources or omission of key features, reducing detection efficiency and accuracy.

[0003] Therefore, we have made improvements to this and proposed an intelligent detection system for component size and appearance defects in multiple fields. Summary of the Invention

[0004] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions: The specific application is as follows: An intelligent detection system for component size and appearance defects in multiple fields, comprising: an image acquisition module, an image preprocessing module, a feature extraction module, a defect recognition and size measurement module, a data processing and analysis module, an automatic control module, and a human-computer interaction module; Image acquisition module: Equipped with a high-resolution CCD camera, professional optical lenses, and an adjustable lighting device, the lighting device can intelligently and dynamically adjust the light intensity, wavelength, and illumination angle according to the material, shape, and inspection requirements of the parts conveyed by the conveyor belt, providing a uniform and stable lighting environment for the CCD camera, enabling the camera to clearly capture the surface features and contour information of the parts from multiple angles. It obtains visible light image and infrared image data stored in two-dimensional or three-dimensional arrays, as well as three-dimensional point cloud data of the parts collected by the lidar stored in a three-dimensional coordinate array. These data are then packaged into raw image data packets and sent to the image preprocessing module. The adjustable lighting device can dynamically adjust the light intensity, wavelength, and illumination angle according to the reflective characteristics of the parts and the inspection angle to enhance the contrast and clarity of the image acquisition. Image preprocessing module: closely connected to the image acquisition module, after receiving the original image data packet, it performs denoising, grayscale adjustment and contrast enhancement operations in sequence, uses filtering algorithms to remove image noise interference, and uses grayscale transformation and histogram equalization technology to improve image contrast. It then encapsulates the preprocessed image data packet and outputs it to the feature extraction module; Feature extraction module: includes: data input and preprocessing unit, multi-algorithm fusion feature extraction unit, multi-modal feature extraction unit, adaptive feature selection mechanism unit and feature optimization and enhancement unit. Based on the pre-processed image data output by the image pre-processing module, the CNN convolutional neural network deep learning algorithm is used. Through transfer learning technology, relying on the preset basic model, a small amount of sample data of new types of parts is used to quickly train and generate a dedicated feature extraction model, accurately extracting the dimensional features and appearance defect features of the parts. At the same time, multiple algorithms and multi-modal data are integrated for feature extraction. Among them, dimensional features include: length, width, diameter, and angle, and appearance defect features include: cracks, scratches, holes, and deformation; Among them, the data input and preprocessing unit receives the image data output by the image preprocessing module and the three-dimensional point cloud data of the image acquisition module, uses statistical filtering or radius filtering to remove outliers from the three-dimensional point cloud data, and normalizes the coordinate range to the interval [0,1]. The multi-algorithm fusion feature extraction unit integrates the CNN convolutional neural network and the LBP local binary pattern, extracts high-level semantic features through the CNN, extracts local texture features through the LBP and splices them, integrates the Hough transform and the SIFT scale-invariant feature transform, extracts geometric shape features and stable feature point descriptors and splices them. The multimodal feature extraction unit integrates visible light and infrared image features, and fuses feature vectors through algorithms such as weighted summation. The image and laser point cloud features are integrated, and corresponding features are spliced ​​through a 3D point cloud feature extraction algorithm and a feature matching algorithm. The adaptive feature selection mechanism unit uses a random forest or support vector machine to evaluate feature importance and dynamically adjusts the feature selection strategy according to the confidence level of the detection result. The feature optimization and enhancement unit uses principal component analysis (PCA) for dimensionality reduction and enhances feature diversity and robustness through affine transformation and generative adversarial network (GAN). Defect Recognition and Size Measurement Module: Connected to the feature extraction module, it has a built-in customizable and automatically updated standard feature database, supports multi-version management, and custom standard features have higher priority than automatically updated features. It compares the features output by the feature extraction module with the standard features in the database. For appearance defects, it determines the defect type, location, and severity by calculating the degree of feature difference and conducting statistical analysis. For dimensional parameters, it uses sub-pixel edge detection combined with geometric calculation methods to achieve high-precision measurement, and outputs the detection results to the data processing and analysis module and the automation control module. Data processing and analysis module: This module works in conjunction with the defect recognition and dimensional measurement module to classify and store the inspection data output by the module. It uses data mining algorithms such as association rule analysis and cluster analysis, as well as machine learning algorithms based on regression analysis models, to deeply mine the data, identify the relationship between defects and production process parameters, predict component quality trends, and generate analysis reports to provide a decision-making basis for production optimization and quality control. Automation control module: Communicates with the defect recognition and dimensional measurement module and the production line automation equipment, generates control instructions based on the detection results of the defect recognition and dimensional measurement module, controls the sorting device to classify qualified and unqualified parts, and feeds back detection information to the production line control system to achieve real-time monitoring and dynamic adjustment of the production process. The production line automation equipment includes: a robotic arm, a conveyor belt, and a sorting device; Human-computer interaction module: This module provides an intuitive and user-friendly interface, allowing operators to set system parameters, view inspection results in real time, query historical data, and remotely monitor system operation. The interface uses graphical display technology to display the inspection process progress, defect location annotation images, and dimensional measurement values ​​in real time, and supports one-click export of inspection results. The module uses a graphical interface developed using Unity3D, supporting resolutions of 1920×1080 and above, with an operation response time of ≤200ms. It supports visual configuration of inspection algorithm parameters and hardware parameters, with configuration parameters automatically saved to the cloud with a backup interval of ≤1 hour. It displays the inspection process progress, defect location annotation images, and dimensional measurement values ​​in real time, supports multi-dimensional historical data queries by time, component type, and defect category, with query result loading time of ≤3 seconds, and enables remote login via the web or mobile terminal. It supports equipment status monitoring, abnormality alarms, and remote parameter distribution, and one-click export of inspection results to Excel or PDF formats. Excel tables contain fields for inspection time, component number, and defect details, and PDF reports automatically generate a table of contents and chart indexes.

[0005] The specific working steps of the image acquisition module are as follows: SA1, start; SA2: Based on the material, shape, and inspection requirements of the parts conveyed by the conveyor belt, the built-in algorithm intelligently adjusts the lighting parameters of the lighting device to create a uniform and stable lighting environment; SA3. Collecting component surface features and contour information from multiple viewing angles using the CCD camera to obtain visible light image data and infrared image data stored in a two-dimensional array or a three-dimensional array; SA4: Synchronously trigger the laser radar to collect 3D point cloud data of components and store it in a 3D coordinate array; SA5: Encapsulate the visible light image data, infrared image data, and 3D point cloud data into a raw image data packet and send it to the image preprocessing module through a standardized interface; SA6, end.

[0006] The working steps of the image preprocessing module are as follows: SB1, start; SB2, receiving the original image data packet output by the image acquisition module through a dedicated interface, and parsing it into visible light image data and infrared image data; SB3, using Gaussian filtering algorithm to remove noise from visible light image data and infrared image data; SB4. Perform grayscale conversion on the infrared image data, map the pixel value range to [0, 255], and normalize the brightness of the infrared image data to eliminate the influence of uneven illumination; SB5, using histogram equalization algorithm to improve the entropy value of image grayscale distribution and enhance the contrast between defect features and background; SB6. Encapsulate the preprocessed visible light image data and infrared image data into a preprocessed image data packet and output it to the feature extraction module, keeping the data format consistent with the input; SB7, end.

[0007] The specific working steps of the feature extraction module are as follows: SC1, start; SC2, a data input and preprocessing unit receives the preprocessed image data packet output by the image preprocessing module; SC3, the data input and preprocessing unit, uses a statistical filtering algorithm on the 3D point cloud data in the preprocessed image data packet to calculate the average distance of the k nearest neighbors of each point. By default, k = 20. Points whose distance exceeds 2 times the standard deviation of the mean are considered outliers and removed. The point cloud coordinate range is normalized to the interval [0, 1] using the formula: Pnorm = Pmax − PminP − Pmin, where Pmin and Pmax are the extreme values ​​of the point cloud coordinates. SC4, the multi-algorithm fusion feature extraction unit extracts high-level semantic features from the preprocessed image data packet through CNN, uses the pre-trained ResNet-50 model to extract features from the preprocessed image, and outputs 2048-dimensional CNN features, which are stored in the form of multi-dimensional tensors; SC5, the multi-algorithm fusion feature extraction unit extracts local texture features from the pre-processed image data packet through LBP. For the same image, it traverses each pixel and calculates the LBP value using the LBP operator with an 8-neighborhood and a radius of 2. It then calculates the 256-dimensional LBP histogram features and stores them in the form of a one-dimensional vector. SC6, the multi-algorithm fusion feature extraction unit performs a splicing operation on high-level semantic features and local texture features, flattens the CNN features into a one-dimensional vector, and then splices them with the LBP features in terms of dimension to form a 2304-dimensional fusion feature; SC7, the multi-algorithm fusion feature extraction unit extracts geometric shape features from the pre-processed image data packet through Hough transform, performs Hough circle / line detection on the image, extracts the circle center coordinates, radius r or line slope k, intercept b parameters, and forms an n×3-dimensional geometric feature array, where n is the number of detected geometric shapes; SC8, the multi-algorithm fusion feature extraction unit extracts stable feature point descriptors from the pre-processed image data packet using the SIFT algorithm, detects key points using the SIFT algorithm, and calculates 128-dimensional descriptors to form an m×128-dimensional feature matrix, where m is the number of key points; SC9, the multi-algorithm fusion feature extraction unit performs a splicing operation on the geometric shape features and the stable feature point descriptors, splicing the geometric feature array and the SIFT descriptor along the dimension to form a (n×3+m×128)-dimensional geometric-texture fusion feature; SC10, the multimodal feature extraction unit extracts feature vectors and fuses them using a weighted summation algorithm: Ffusion = αFvis + (1 − α)Fir, where the weight factor α∈[0,1] can be adaptively adjusted; SC11, the multimodal feature extraction unit extracts image features from the preprocessed image, extracts 33-dimensional point cloud features from the 3D point cloud using the PFH algorithm, establishes the correspondence between image features and point cloud features based on the FLANN matcher, and concatenates the matched features along the dimensions to form image-point cloud fusion features; SC12, the adaptive feature selection mechanism unit evaluates the importance of features, uses the random forest algorithm to train all extracted features, and calculates the Gini importance score of each feature. The score range is [0, 1]. The higher the score, the greater the impact of the feature on the detection result. SC13, the adaptive feature selection mechanism unit sorts the features in descending order of importance scores and selects the top k key features through np.argsort(feature_importance_scores)[−k:], where k is a preset threshold and k≤50% of the total number of features. The confidence level is dynamically adjusted based on the detection result: if the confidence level is <70%, 5-10 auxiliary features are added; if the confidence level is ≥90%, 10-15 redundant features are reduced to improve detection speed. SC14, the feature optimization and enhancement unit uses the PCA algorithm to reduce the dimension of the feature matrix, calculate the covariance matrix and solve the eigenvalues, select the eigenvectors corresponding to the first m largest eigenvalues, and project the original features into the subspace to form a low-dimensional feature vector; SC15: Perform an affine transformation on the feature vector, randomly generate transformation parameters, expand feature diversity, use GAN to generate adversarial networks to train feature generation models, input real features to generate similar virtual features, and after screening by the discriminator, merge them with the real features to form an enhanced feature dataset; SC16, output the optimized and enhanced features in one-dimensional vector format to the defect recognition and size measurement module; SC17, end.

[0008] The specific working steps of the defect identification and size measurement module are as follows: SD1, start; SD2. Calculate the cosine similarity between the extracted features and the defect templates in the database. If the similarity threshold is ≤ 0.6, it is determined to be a defect. The defect type, location, and severity are determined through statistical analysis. SD3 uses sub-pixel edge detection to locate feature edges and combines geometric calculations to measure length, angle, and diameter parameters; SD4: Integrate defect information and dimension data into structured data, support XML and JSON formats, and output to the data processing and analysis module and the automation control module; SD5, end.

[0009] The specific working steps of the data processing and analysis module are as follows: SE1, start; SE2, receives defect information and dimension data, integrates them into structured data, classifies the inspection data by component type, inspection time, and defect category, and stores them in a distributed database; SE3: Use the Apriori algorithm to mine the relationship between defects and production process parameters, and automatically mark rules with a confidence level of ≥80%; SE4. Use K-means algorithm to cluster defect distribution and generate defect hotspot area map; SE5. Build a quality prediction model based on the regression analysis model, input historical inspection data, and predict the yield rate for the next 30 days. The regression analysis model uses random forest regression. SE6. Generate a visual analysis report based on the mining results, including a defect distribution histogram, process parameter correlation matrix, and quality trend curve. The report update frequency is ≤ 10 minutes / time; SE7, end.

[0010] The specific working steps of the automation control module are as follows: SF1, start; SF2, through OPCUA or ModbusTCP standardized interface, communicates with defect recognition and dimension measurement modules and production line automation equipment in real time; SF3 generates three types of control instructions based on the test results: sorting instructions: generate rejection signals for unqualified parts and control the sorting device to complete sorting within 100ms; production line feedback instructions: feed back dimensional tolerance and defect rate abnormality information to the production line PLC control system to trigger process parameter adjustments; status monitoring instructions: regularly collect equipment operating status data at 30-second intervals and generate equipment health reports; SF4, dynamically adjust the detection rhythm through closed-loop control algorithm to ensure synchronization with the production line speed; SF5, the end.

[0011] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention integrates CNN with LBP, Hough transform and SIFT algorithms, taking into account both high-level semantics and underlying texture / geometric features. By splicing features, a 2304-dimensional fused feature vector is formed, which improves the integrity of feature extraction for parts in multiple fields. It can accurately extract texture detail features and accurately obtain geometric shape features, significantly improving the system's adaptability to parts in multiple fields. At the same time, the adaptive feature selection mechanism dynamically optimizes feature combinations according to the detection results, reduces computational redundancy, improves detection efficiency, and ensures detection accuracy.

[0012] 2. The present invention evaluates feature importance based on random forests and dynamically selects the top k key features based on the confidence of the detection results. When the confidence is low, auxiliary features are added, and when the confidence is high, redundant features are reduced. This improves the detection speed while maintaining high accuracy. Through PCA dimensionality reduction and GAN data enhancement, a dedicated model can be quickly generated with only a small number of samples, shortening the model training cycle.

[0013] 3. By integrating visible light, infrared images, and laser point cloud data acquisition, the system can obtain more comprehensive component information. Infrared images can effectively detect defects in dark-surface components, while laser point cloud data enables high-precision three-dimensional dimension measurement, making up for the shortcomings of traditional visible light image detection. It greatly improves the detection capabilities of complex materials and hidden defects, ensuring the accuracy and reliability of the test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a schematic diagram of the system operation of this application; Figure 2 This is a flow chart of the image acquisition module of this application; Figure 3 This is a flow chart of the image preprocessing module of this application; Figure 4This is a flow chart of the feature extraction module of this application; Figure 5 This is a flow chart of the defect identification and size measurement module of this application; Figure 6 This is a flow chart of the defect identification and size measurement module of this application; Figure 7 This is a flow chart of the automation control module and the human-computer interaction module of this application. DETAILED DESCRIPTION

[0015] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0016] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents some embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0017] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions therein may be combined with each other.

[0018] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0019] In the description of the present invention, it should be noted that the terms "upper" and "lower" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or the orientations or positional relationships in which the inventive product is typically placed when in use, or the orientations or positional relationships commonly understood by those skilled in the art. Such terms are intended solely to facilitate the description of the present invention and simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" and the like are used solely for distinction and should not be construed as indicating or implying relative importance.

[0020] The present invention provides the following technical solutions: Please refer to Figure 1-7 , an intelligent detection system for component size and appearance defects in multiple fields, including: image acquisition module, image preprocessing module, feature extraction module, defect recognition and size measurement module, data processing and analysis module, automatic control module and human-computer interaction module; Image acquisition module: Equipped with a high-resolution CCD camera, professional optical lenses, and an adjustable lighting device, the lighting device can intelligently and dynamically adjust the light intensity, wavelength, and illumination angle according to the material, shape, and inspection requirements of the parts conveyed by the conveyor belt, providing a uniform and stable lighting environment for the CCD camera, enabling the camera to clearly capture the surface features and contour information of the parts from multiple angles. It obtains visible light image and infrared image data stored in two-dimensional or three-dimensional arrays, as well as three-dimensional point cloud data of the parts collected by the lidar stored in a three-dimensional coordinate array. These data are then packaged into raw image data packets and sent to the image preprocessing module. The adjustable lighting device can dynamically adjust the light intensity, wavelength, and illumination angle according to the reflective characteristics of the parts and the inspection angle to enhance the contrast and clarity of the image acquisition. Image preprocessing module: closely connected to the image acquisition module, after receiving the original image data packet, it performs denoising, grayscale adjustment and contrast enhancement operations in sequence, uses filtering algorithms to remove image noise interference, and uses grayscale transformation and histogram equalization technology to improve image contrast. It then encapsulates the preprocessed image data packet and outputs it to the feature extraction module; Feature extraction module: includes: data input and preprocessing unit, multi-algorithm fusion feature extraction unit, multi-modal feature extraction unit, adaptive feature selection mechanism unit and feature optimization and enhancement unit. Based on the pre-processed image data output by the image pre-processing module, the CNN convolutional neural network deep learning algorithm is used. Through transfer learning technology, relying on the preset basic model, a small amount of sample data of new types of parts is used to quickly train and generate a dedicated feature extraction model, accurately extracting the dimensional features and appearance defect features of the parts. At the same time, multiple algorithms and multi-modal data are integrated for feature extraction. Among them, dimensional features include: length, width, diameter, and angle, and appearance defect features include: cracks, scratches, holes, and deformation; Among them, the data input and preprocessing unit receives the image data output by the image preprocessing module and the three-dimensional point cloud data of the image acquisition module, uses statistical filtering or radius filtering to remove outliers from the three-dimensional point cloud data, and normalizes the coordinate range to the interval [0,1]. The multi-algorithm fusion feature extraction unit integrates the CNN convolutional neural network and the LBP local binary pattern, extracts high-level semantic features through the CNN, extracts local texture features through the LBP and splices them, integrates the Hough transform and the SIFT scale-invariant feature transform, extracts geometric shape features and stable feature point descriptors and splices them. The multimodal feature extraction unit integrates visible light and infrared image features, and fuses feature vectors through algorithms such as weighted summation. The image and laser point cloud features are integrated, and corresponding features are spliced ​​through a 3D point cloud feature extraction algorithm and a feature matching algorithm. The adaptive feature selection mechanism unit uses a random forest or support vector machine to evaluate feature importance and dynamically adjusts the feature selection strategy according to the confidence level of the detection result. The feature optimization and enhancement unit uses principal component analysis (PCA) for dimensionality reduction and enhances feature diversity and robustness through affine transformation and generative adversarial network (GAN). Defect Recognition and Size Measurement Module: Connected to the feature extraction module, it has a built-in customizable and automatically updated standard feature database, supports multi-version management, and custom standard features have higher priority than automatically updated features. It compares the features output by the feature extraction module with the standard features in the database. For appearance defects, it determines the defect type, location, and severity by calculating the degree of feature difference and conducting statistical analysis. For dimensional parameters, it uses sub-pixel edge detection combined with geometric calculation methods to achieve high-precision measurement, and outputs the detection results to the data processing and analysis module and the automation control module. Data processing and analysis module: This module works in conjunction with the defect recognition and dimensional measurement module to classify and store the inspection data output by the module. It uses data mining algorithms such as association rule analysis and cluster analysis, as well as machine learning algorithms based on regression analysis models, to deeply mine the data, identify the relationship between defects and production process parameters, predict component quality trends, and generate analysis reports to provide a decision-making basis for production optimization and quality control. Automation control module: Communicates with the defect recognition and dimensional measurement module and the production line automation equipment, generates control instructions based on the detection results of the defect recognition and dimensional measurement module, controls the sorting device to classify qualified and unqualified parts, and feeds back detection information to the production line control system to achieve real-time monitoring and dynamic adjustment of the production process. The production line automation equipment includes: a robotic arm, a conveyor belt, and a sorting device; Human-computer interaction module: This module provides an intuitive and user-friendly interface, allowing operators to set system parameters, view inspection results in real time, query historical data, and remotely monitor system operation. The interface uses graphical display technology to display the inspection process progress, defect location annotation images, and dimensional measurement values ​​in real time, and supports one-click export of inspection results. The module uses a graphical interface developed using Unity3D, supporting resolutions of 1920×1080 and above, with an operation response time of ≤200ms. It supports visual configuration of inspection algorithm parameters and hardware parameters, with configuration parameters automatically saved to the cloud with a backup interval of ≤1 hour. It displays the inspection process progress, defect location annotation images, and dimensional measurement values ​​in real time, supports multi-dimensional historical data queries by time, component type, and defect category, with query result loading time of ≤3 seconds, and enables remote login via the web or mobile terminal. It supports equipment status monitoring, abnormality alarms, and remote parameter distribution, and one-click export of inspection results to Excel or PDF formats. Excel tables contain fields for inspection time, component number, and defect details, and PDF reports automatically generate a table of contents and chart indexes.

[0021] The specific working steps of the image acquisition module are as follows: SA1, start; SA2: Based on the material, shape, and inspection requirements of the parts conveyed by the conveyor belt, the built-in algorithm intelligently adjusts the lighting parameters of the lighting device to create a uniform and stable lighting environment; SA3. Collecting component surface features and contour information from multiple viewing angles using the CCD camera to obtain visible light image data and infrared image data stored in a two-dimensional array or a three-dimensional array; SA4: Synchronously trigger the laser radar to collect 3D point cloud data of components and store it in a 3D coordinate array; SA5: Encapsulate the visible light image data, infrared image data, and 3D point cloud data into a raw image data packet and send it to the image preprocessing module through a standardized interface; SA6, end.

[0022] The working steps of the image preprocessing module are as follows: SB1, start; SB2, receiving the original image data packet output by the image acquisition module through a dedicated interface, and parsing it into visible light image data and infrared image data; SB3, using Gaussian filtering algorithm to remove noise from visible light image data and infrared image data; SB4. Perform grayscale conversion on the infrared image data, map the pixel value range to [0, 255], and normalize the brightness of the infrared image data to eliminate the influence of uneven illumination; SB5, using histogram equalization algorithm to improve the entropy value of image grayscale distribution and enhance the contrast between defect features and background; SB6. Encapsulate the preprocessed visible light image data and infrared image data into a preprocessed image data packet and output it to the feature extraction module, keeping the data format consistent with the input; SB7, end.

[0023] The specific working steps of the feature extraction module are as follows: SC1, start; SC2, a data input and preprocessing unit receives the preprocessed image data packet output by the image preprocessing module; SC3, the data input and preprocessing unit, uses a statistical filtering algorithm on the 3D point cloud data in the preprocessed image data packet to calculate the average distance of the k nearest neighbors of each point. By default, k = 20. Points whose distance exceeds 2 times the standard deviation of the mean are considered outliers and removed. The point cloud coordinate range is normalized to the interval [0, 1] using the formula: Pnorm = Pmax − PminP − Pmin, where Pmin and Pmax are the extreme values ​​of the point cloud coordinates. SC4, the multi-algorithm fusion feature extraction unit extracts high-level semantic features from the preprocessed image data packet through CNN, uses the pre-trained ResNet-50 model to extract features from the preprocessed image, and outputs 2048-dimensional CNN features, which are stored in the form of multi-dimensional tensors; SC5, the multi-algorithm fusion feature extraction unit extracts local texture features from the pre-processed image data packet through LBP. For the same image, it traverses each pixel and calculates the LBP value using the LBP operator with an 8-neighborhood and a radius of 2. It then calculates the 256-dimensional LBP histogram features and stores them in the form of a one-dimensional vector. SC6, the multi-algorithm fusion feature extraction unit performs a splicing operation on high-level semantic features and local texture features, flattens the CNN features into a one-dimensional vector, and then splices them with the LBP features in terms of dimension to form a 2304-dimensional fusion feature; SC7, the multi-algorithm fusion feature extraction unit extracts geometric shape features from the pre-processed image data packet through Hough transform, performs Hough circle / line detection on the image, extracts the circle center coordinates, radius r or line slope k, intercept b parameters, and forms an n×3-dimensional geometric feature array, where n is the number of detected geometric shapes; SC8, the multi-algorithm fusion feature extraction unit extracts stable feature point descriptors from the pre-processed image data packet using the SIFT algorithm, detects key points using the SIFT algorithm, and calculates 128-dimensional descriptors to form an m×128-dimensional feature matrix, where m is the number of key points; SC9, the multi-algorithm fusion feature extraction unit performs a splicing operation on the geometric shape features and the stable feature point descriptors, splicing the geometric feature array and the SIFT descriptor along the dimension to form a (n×3+m×128)-dimensional geometric-texture fusion feature; SC10, the multimodal feature extraction unit extracts feature vectors and fuses them using a weighted summation algorithm: Ffusion = αFvis + (1 − α)Fir, where the weight factor α∈[0,1] can be adaptively adjusted; SC11, the multimodal feature extraction unit extracts image features from the preprocessed image, extracts 33-dimensional point cloud features from the 3D point cloud using the PFH algorithm, establishes the correspondence between image features and point cloud features based on the FLANN matcher, and concatenates the matched features along the dimensions to form image-point cloud fusion features; SC12, the adaptive feature selection mechanism unit evaluates the importance of features, uses the random forest algorithm to train all extracted features, and calculates the Gini importance score of each feature. The score range is [0, 1]. The higher the score, the greater the impact of the feature on the detection result. SC13, the adaptive feature selection mechanism unit sorts the features in descending order of importance scores and selects the top k key features through np.argsort(feature_importance_scores)[−k:], where k is a preset threshold and k≤50% of the total number of features. The confidence level is dynamically adjusted based on the detection result: if the confidence level is <70%, 5-10 auxiliary features are added; if the confidence level is ≥90%, 10-15 redundant features are reduced to improve detection speed. SC14, the feature optimization and enhancement unit uses the PCA algorithm to reduce the dimension of the feature matrix, calculate the covariance matrix and solve the eigenvalues, select the eigenvectors corresponding to the first m largest eigenvalues, and project the original features into the subspace to form a low-dimensional feature vector; SC15: Perform an affine transformation on the feature vector, randomly generate transformation parameters, expand feature diversity, use GAN to generate adversarial networks to train feature generation models, input real features to generate similar virtual features, and after screening by the discriminator, merge them with the real features to form an enhanced feature dataset; SC16, output the optimized and enhanced features in one-dimensional vector format to the defect recognition and size measurement module; SC17, end.

[0024] The specific working steps of the defect identification and size measurement module are as follows: SD1, start; SD2. Calculate the cosine similarity between the extracted features and the defect templates in the database. If the similarity threshold is ≤ 0.6, it is determined to be a defect. The defect type, location, and severity are determined through statistical analysis. SD3, uses sub-pixel edge detection to locate feature edges and combines geometric calculations to measure length, angle, and diameter parameters; SD4 integrates defect information and dimension data into structured data, supports XML and JSON formats, and outputs them to the data processing and analysis module and the automation control module; SD5, end.

[0025] The specific working steps of the data processing and analysis module are as follows: SE1, start; SE2, receives defect information and dimension data, integrates them into structured data, classifies the inspection data by component type, inspection time, and defect category, and stores them in a distributed database; SE3, uses the Apriori algorithm to mine the correlation between defects and production process parameters, and automatically marks rules with a confidence level ≥ 80%; SE4. Use K-means algorithm to cluster defect distribution and generate defect hotspot area map; SE5. Build a quality prediction model based on the regression analysis model, input historical inspection data, and predict the yield rate for the next 30 days. The regression analysis model uses random forest regression. SE6. Generate a visual analysis report based on the mining results, including a defect distribution histogram, process parameter correlation matrix, and quality trend curve. The report update frequency is ≤ 10 minutes / time; SE7, end.

[0026] The specific working steps of the automation control module are as follows: SF1, start; SF2, through OPCUA or ModbusTCP standardized interface, communicates with defect recognition and dimension measurement modules and production line automation equipment in real time; SF3 generates three types of control instructions based on the test results: sorting instructions: generate rejection signals for unqualified parts and control the sorting device to complete sorting within 100ms; production line feedback instructions: feed back dimensional tolerance and defect rate abnormality information to the production line PLC control system to trigger process parameter adjustments; status monitoring instructions: regularly collect equipment operating status data at 30-second intervals and generate equipment health reports; SF4, dynamically adjust the detection rhythm through closed-loop control algorithm to ensure synchronization with the production line speed; SF5, the end.

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0028] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions therein may be combined with each other.

[0029] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0030] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although this specification has described the present invention in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

Claims

1. An intelligent detection system for component size and appearance defects in multiple fields, characterized by: include: Image acquisition module, image preprocessing module, feature extraction module, defect recognition and dimension measurement module, data processing and analysis module, automation control module and human-computer interaction module; Image acquisition module: Equipped with a high-resolution CCD camera, professional optical lenses, and adjustable lighting devices, the lighting device can intelligently and dynamically adjust the light intensity, wavelength, and illumination angle according to the material, shape, and inspection requirements of the parts transported by the conveyor belt, providing a uniform and stable lighting environment for the CCD camera, enabling the camera to clearly capture the surface features and contour information of the parts from multiple angles. It obtains visible light image and infrared image data stored in two-dimensional or three-dimensional arrays, as well as three-dimensional point cloud data of parts collected by lidar stored in a three-dimensional coordinate array. The data is then encapsulated as a raw image data packet and sent to the image preprocessing module; Image preprocessing module: closely connected to the image acquisition module, after receiving the original image data packet, it performs denoising, grayscale adjustment and contrast enhancement operations in sequence, uses filtering algorithms to remove image noise interference, and uses grayscale transformation and histogram equalization technology to improve image contrast. It then encapsulates the preprocessed image data packet and outputs it to the feature extraction module; Feature extraction module: includes: data input and preprocessing unit, multi-algorithm fusion feature extraction unit, multi-modal feature extraction unit, adaptive feature selection mechanism unit and feature optimization and enhancement unit. Based on the pre-processed image data output by the image pre-processing module, the CNN convolutional neural network deep learning algorithm is used. Through transfer learning technology, relying on the preset basic model, a small amount of sample data of new types of parts is used to quickly train and generate a dedicated feature extraction model, accurately extracting the dimensional features and appearance defect features of the parts. At the same time, multiple algorithms and multi-modal data are integrated for feature extraction. Among them, dimensional features include: length, width, diameter, and angle, and appearance defect features include: cracks, scratches, holes, and deformation; Among them, the data input and preprocessing unit receives the image data output by the image preprocessing module and the three-dimensional point cloud data of the image acquisition module, uses statistical filtering or radius filtering to remove outliers from the three-dimensional point cloud data, and normalizes the coordinate range to the interval [0,1]. The multi-algorithm fusion feature extraction unit integrates the CNN convolutional neural network and the LBP local binary pattern, extracts high-level semantic features through the CNN, extracts local texture features through the LBP and splices them, integrates the Hough transform and the SIFT scale-invariant feature transform, extracts geometric shape features and stable feature point descriptors and splices them. The multimodal feature extraction unit integrates visible light and infrared image features, and fuses feature vectors through algorithms such as weighted summation. The image and laser point cloud features are integrated, and corresponding features are spliced ​​through a 3D point cloud feature extraction algorithm and a feature matching algorithm. The adaptive feature selection mechanism unit uses a random forest or support vector machine to evaluate feature importance and dynamically adjusts the feature selection strategy according to the confidence level of the detection result. The feature optimization and enhancement unit uses principal component analysis (PCA) for dimensionality reduction and enhances feature diversity and robustness through affine transformation and generative adversarial network (GAN). Defect Recognition and Size Measurement Module: Connected to the feature extraction module, it has a built-in customizable and automatically updated standard feature database, supports multi-version management, and custom standard features have higher priority than automatically updated features. It compares the features output by the feature extraction module with the standard features in the database. For appearance defects, it determines the defect type, location, and severity by calculating the degree of feature difference and conducting statistical analysis. For dimensional parameters, it uses sub-pixel edge detection combined with geometric calculation methods to achieve high-precision measurement, and outputs the detection results to the data processing and analysis module and the automation control module. Data processing and analysis module: This module works in conjunction with the defect recognition and dimensional measurement module to classify and store the inspection data output by the module. It uses data mining algorithms such as association rule analysis and cluster analysis, as well as machine learning algorithms based on regression analysis models, to deeply mine the data, identify the relationship between defects and production process parameters, predict component quality trends, and generate analysis reports to provide a decision-making basis for production optimization and quality control. Automation control module: Communicates with the defect recognition and dimensional measurement module and the production line automation equipment, generates control instructions based on the detection results of the defect recognition and dimensional measurement module, controls the sorting device to classify qualified and unqualified parts, and feeds back detection information to the production line control system to achieve real-time monitoring and dynamic adjustment of the production process. The production line automation equipment includes: a robotic arm, a conveyor belt, and a sorting device; Human-computer interaction module: It is used to provide an intuitive and friendly user interface, supporting operators to set system parameters, view test results in real time, query historical data, and remotely monitor system operation. The interface uses graphical display technology to display the progress of the inspection process, defect location annotation images and dimensional measurement values ​​in real time, and supports one-click export of test results.

2. The multi-domain component size and appearance defect intelligent detection system according to claim 1 is characterized in that: The specific working steps of the image acquisition module are as follows: SA1, start; SA2: Based on the material, shape, and inspection requirements of the parts conveyed by the conveyor belt, the built-in algorithm intelligently adjusts the lighting parameters of the lighting device to create a uniform and stable lighting environment; SA3. Collecting component surface features and contour information from multiple viewing angles using the CCD camera to obtain visible light image data and infrared image data stored in a two-dimensional array or a three-dimensional array; SA4: Synchronously trigger the laser radar to collect 3D point cloud data of components and store it in a 3D coordinate array; SA5: Encapsulate the visible light image data, infrared image data, and 3D point cloud data into a raw image data packet and send it to the image preprocessing module through a standardized interface; SA6, end.

3. The multi-domain component size and appearance defect intelligent detection system according to claim 1, characterized in that: The working steps of the image preprocessing module are as follows: SB1, start; SB2, receiving the original image data packet output by the image acquisition module through a dedicated interface, and parsing it into visible light image data and infrared image data; SB3, using Gaussian filtering algorithm to remove noise from visible light image data and infrared image data; SB4. Perform grayscale conversion on the infrared image data, map the pixel value range to [0, 255], and normalize the brightness of the infrared image data to eliminate the influence of uneven illumination; SB5, using histogram equalization algorithm to improve the entropy value of image grayscale distribution and enhance the contrast between defect features and background; SB6. Encapsulate the preprocessed visible light image data and infrared image data into a preprocessed image data packet and output it to the feature extraction module, keeping the data format consistent with the input; SB7, end.

4. The multi-domain component size and appearance defect intelligent detection system according to claim 1, characterized in that: The specific working steps of the feature extraction module are as follows: SC1, start; SC2, a data input and preprocessing unit receives the preprocessed image data packet output by the image preprocessing module; SC3, the data input and preprocessing unit, uses a statistical filtering algorithm on the 3D point cloud data in the preprocessed image data packet to calculate the average distance of the k nearest neighbors of each point. By default, k = 20. Points whose distance exceeds 2 times the standard deviation of the mean are considered outliers and removed. The point cloud coordinate range is normalized to the interval [0, 1] using the formula: Pnorm = Pmax − PminP − Pmin, where Pmin and Pmax are the extreme values ​​of the point cloud coordinates. SC4, the multi-algorithm fusion feature extraction unit extracts high-level semantic features from the preprocessed image data packet through CNN, uses the pre-trained ResNet-50 model to extract features from the preprocessed image, and outputs 2048-dimensional CNN features, which are stored in the form of multi-dimensional tensors; SC5, the multi-algorithm fusion feature extraction unit extracts local texture features from the pre-processed image data packet through LBP. For the same image, it traverses each pixel and calculates the LBP value using the LBP operator with an 8-neighborhood and a radius of 2. It then calculates the 256-dimensional LBP histogram features and stores them in the form of a one-dimensional vector. SC6, the multi-algorithm fusion feature extraction unit performs a splicing operation on high-level semantic features and local texture features, flattens the CNN features into a one-dimensional vector, and then splices them with the LBP features in terms of dimension to form a 2304-dimensional fusion feature; SC7, the multi-algorithm fusion feature extraction unit extracts geometric shape features from the pre-processed image data packet through Hough transform, performs Hough circle / line detection on the image, extracts the circle center coordinates, radius r or line slope k, intercept b parameters, and forms an n×3-dimensional geometric feature array, where n is the number of detected geometric shapes; SC8, the multi-algorithm fusion feature extraction unit extracts stable feature point descriptors from the pre-processed image data packet using the SIFT algorithm, detects key points using the SIFT algorithm, and calculates 128-dimensional descriptors to form an m×128-dimensional feature matrix, where m is the number of key points; SC9, the multi-algorithm fusion feature extraction unit performs a splicing operation on the geometric shape features and the stable feature point descriptors, splicing the geometric feature array and the SIFT descriptor along the dimension to form a (n×3+m×128)-dimensional geometric-texture fusion feature; SC10, the multimodal feature extraction unit extracts feature vectors and fuses them using a weighted summation algorithm: Ffusion = αFvis + (1 − α)Fir, where the weight factor α∈[0,1] can be adaptively adjusted; SC11, the multimodal feature extraction unit extracts image features from the preprocessed image, extracts 33-dimensional point cloud features from the 3D point cloud using the PFH algorithm, establishes the correspondence between image features and point cloud features based on the FLANN matcher, and concatenates the matched features along the dimensions to form image-point cloud fusion features; SC12, the adaptive feature selection mechanism unit evaluates the importance of features, uses the random forest algorithm to train all extracted features, and calculates the Gini importance score of each feature. The score range is [0, 1]. The higher the score, the greater the impact of the feature on the detection result. SC13, the adaptive feature selection mechanism unit sorts the features in descending order of importance scores and selects the top k key features through np.argsort(feature_importance_scores)[−k:], where k is a preset threshold and k≤50% of the total number of features. The confidence level is dynamically adjusted based on the detection result: if the confidence level is <70%, 5-10 auxiliary features are added; if the confidence level is ≥90%, 10-15 redundant features are reduced to improve detection speed. SC14, the feature optimization and enhancement unit uses the PCA algorithm to reduce the dimension of the feature matrix, calculate the covariance matrix and solve the eigenvalues, select the eigenvectors corresponding to the first m largest eigenvalues, and project the original features into the subspace to form a low-dimensional feature vector; SC15: Perform an affine transformation on the feature vector, randomly generate transformation parameters, expand feature diversity, use GAN to generate adversarial networks to train feature generation models, input real features to generate similar virtual features, and after screening by the discriminator, merge them with the real features to form an enhanced feature dataset; SC16, output the optimized and enhanced features in one-dimensional vector format to the defect recognition and size measurement module; SC17, end.

5. The multi-domain component size and appearance defect intelligent detection system according to claim 1, characterized in that: The specific working steps of the defect identification and size measurement module are as follows: SD1, start; SD2 receives the one-dimensional feature vector output by the feature extraction module, loads the standard feature database, and first calls the custom standard features manually imported by the operator, followed by the automatically updated features. The extracted features are then compared with the defect templates in the database for cosine similarity calculation. When the similarity threshold is ≤0.6, it is determined to be a defect. The defect type, location, and severity are determined through statistical analysis. SD3 uses sub-pixel edge detection to locate feature edges and combines geometric calculations to measure length, angle, and diameter parameters; SD4: Integrate defect information and dimension data into structured data, support XML and JSON formats, and output to the data processing and analysis module and the automation control module; SD5, end.

6. The multi-domain component size and appearance defect intelligent detection system according to claim 1, characterized in that: The specific working steps of the data processing and analysis module are as follows: SE1, start; SE2, receives defect information and dimension data, integrates them into structured data, classifies the inspection data by component type, inspection time, and defect category, and stores them in a distributed database; SE3: Use the Apriori algorithm to mine the relationship between defects and production process parameters, and automatically mark rules with a confidence level of ≥80%; SE4. Use K-means algorithm to cluster defect distribution and generate defect hotspot area map; SE5. Build a quality prediction model based on the regression analysis model, input historical inspection data, and predict the yield rate for the next 30 days. The regression analysis model uses random forest regression. SE6. Generate a visual analysis report based on the mining results, including a defect distribution histogram, process parameter correlation matrix, and quality trend curve. The report update frequency is ≤ 10 minutes / time; SE7, end.

7. The multi-domain component size and appearance defect intelligent detection system according to claim 1, characterized in that: The specific working steps of the automation control module are as follows: SF1, start; SF2, through OPCUA or ModbusTCP standardized interface, communicates with defect recognition and dimension measurement modules and production line automation equipment in real time; SF3 generates three types of control instructions based on the test results: sorting instructions: generate rejection signals for unqualified parts and control the sorting device to complete sorting within 100ms; production line feedback instructions: feed back dimensional tolerance and defect rate abnormality information to the production line PLC control system to trigger process parameter adjustments; status monitoring instructions: regularly collect equipment operating status data at 30-second intervals and generate equipment health reports; SF4, dynamically adjust the detection rhythm through closed-loop control algorithm to ensure synchronization with the production line speed; SF5, the end.

8. The multi-domain component size and appearance defect intelligent detection system according to claim 1, characterized in that: The human-computer interaction module is based on the Unity3D development graphical interface, with a resolution of 1920×1080 and above, an operation response time of ≤200ms, and supports visual configuration of detection algorithm parameters and hardware parameters. The configuration parameters are automatically saved to the cloud, with a backup interval of ≤1 hour. It displays the progress of the detection process, defect location annotation images and dimensional measurement values ​​in real time, supports multi-dimensional query of historical data by time, component type, and defect category, and the query result loading time is ≤3 seconds. Remote login is achieved through the web or mobile terminal, supports equipment status monitoring, abnormal alarms and remote parameter distribution, and supports one-click export of detection results to Excel or PDF format. The Excel table contains the detection time field, component number field and defect details field, and the PDF report automatically generates a directory and chart index.

Images