Visual detection method and system capable of adapting to multiple brands

Abstract

The method comprises the following steps that two-dimensional image data and three-dimensional point cloud data of a to-be-detected product are synchronously collected through a multi-mode sensor, and the multi-mode sensor comprises a high-resolution line scanning camera, a TOF sensor and a tunable annular light source; and matching a multi-brand knowledge base based on the product identifier or the appearance feature, and calling a feature template and a defect judgment standard of a corresponding brand. According to the visual detection method and system capable of self-adapting to the multiple brands, through the transfer learning framework and the multi-brand knowledge base, new brand products can be automatically identified, the pre-training model can be loaded, high initial detection accuracy can be achieved without extra annotation data, and by combining the self-supervised generative adversarial network and the federal learning mechanism, the detection accuracy is improved. And when the defect sample size is small, the relatively high defect detection rate can still be realized, and the false alarm rate is relatively low.

Description

Technical Field

[0001] The present invention relates to the field of machine vision and industrial inspection technology, and in particular to a visual inspection method and system that can adapt to multiple brands. Background Art

[0002] The core challenges currently facing industrial visual inspection include:

[0003] Poor multi-brand adaptability: The appearance characteristics, materials, and defect standards of products from different brands vary significantly. Traditional detection systems require customized algorithms and hardware for a single brand, which lacks flexibility.

[0004] Sensitive to environmental interference: Industrial sites may have problems such as light fluctuations, electromagnetic interference, and mechanical vibration, which may lead to reduced detection accuracy.

[0005] Defect data is scarce: Small sample defects make it difficult to support deep learning model training, and the defect type and location are highly random.

[0006] Existing solutions, such as the FasterR-CNN-based target detection algorithm, can improve detection speed, but its fixed parameter template is difficult to adapt to the dynamic needs of multiple brands; although the CCD-based hardware system can achieve high-precision imaging, it lacks adaptive light source adjustment and feature extraction mechanisms. Summary of the Invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides a visual inspection method and system that can adapt to multiple brands, the purpose of which is to solve the problems raised in the above-mentioned background technology.

[0008] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0009] A visual inspection method capable of adapting to multiple brands includes the following steps:

[0010] S1: Synchronously collect 2D image data and 3D point cloud data of the product to be inspected using a multimodal sensor, wherein the multimodal sensor includes a high-resolution line scan camera, a TOF sensor, and a tunable ring light source;

[0011] S2: Match the multi-brand knowledge base based on product identification or appearance features, and call the corresponding brand's feature template and defect judgment criteria;

[0012] S3: The data collected in S1 is input into an adaptive feature extraction network. The network uses transfer learning to initialize model parameters and updates weights in real time through online learning. Specifically, the following steps are performed:

[0013] Cross-brand feature transfer module: uses the pre-trained ResNet-50 backbone network to extract common features;

[0014] Small Sample Defect Learning Module: Expanding defect samples through self-supervised generative adversarial networks (SS-GAN)

[0015] S4: Dynamically optimizes imaging parameters based on a reinforcement learning algorithm, including light source intensity, camera exposure time, and image filter threshold, with the goal of maximizing the signal-to-noise ratio in the defect area.

[0016] S5: Multimodal data is fused at the feature level to output a test report containing defect type, location, and confidence level, and the test results are fed back to the knowledge base for incremental learning.

[0017] Furthermore, the multimodal data acquisition in step S1 eliminates image distortion caused by mechanical vibration through a Kalman filter algorithm.

[0018] Furthermore, the transfer learning in step S3 is implemented as follows:

[0019] For new brand products, freeze the first three convolutional layers of the backbone network and only fine-tune the parameters of the fully connected layer;

[0020] The cosine similarity is used to measure the differences in feature distribution across brands and dynamically adjust the transfer learning weight coefficient.

[0021] Furthermore, the reinforcement learning in step S4 adopts the deep deterministic policy gradient (DDPG) algorithm, and its reward function is designed as:

[0022] R = α·b defect contrast + β·image uniformity - γ·hardware energy consumption.

[0023] A multi-brand adaptive visual inspection system, comprising:

[0024] Data acquisition module: consists of a high-resolution line scan camera, a TOF sensor, and a tunable ring light source;

[0025] Embedded processing unit: equipped with NVIDIA Jetson AGX Xavier processor, built-in adaptive feature extraction network and dynamic parameter optimization engine;

[0026] Multi-brand knowledge base: uses a distributed graph database to store the characteristic topological relationships, defect history data, and optimization strategies of each brand's products;

[0027] Anti-interference actuator: includes electromagnetic shielding cover, pneumatic shock absorption platform and redundant imaging channel.

[0028] Furthermore, the tunable annular light source includes 8 groups of independently controlled LED arrays, and the adjustable wavelength range of each group is 400nm-1000nm.

[0029] Furthermore, the adaptive feature extraction network achieves lightweight deployment through model pruning and quantization technology.

[0030] The beneficial effects of the present invention are:

[0031] The present invention provides a multi-brand adaptive visual inspection method and system, which can automatically identify new brand products and load pre-trained models through a transfer learning framework and a multi-brand knowledge base, and achieve a high initial detection accuracy without the need for additional labeled data. By combining a self-supervised generative adversarial network with a federated learning mechanism, a high defect detection rate and a low false alarm rate can be achieved even when the number of defect samples is small. The online learning module supports dynamic updating of model weights, and the inclusion cycle of new defect types is greatly shortened compared to traditional solutions. The DDPG reinforcement learning algorithm is used to dynamically adjust the light source and camera parameters, adaptively improve imaging quality, and use redundant imaging channels to work in conjunction with the Kalman filter to enhance anti-interference capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a diagram of the overall system architecture of the present invention;

[0033] Figure 2 This is a flow chart of the adaptive feature extraction network of the present invention;

[0034] Figure 3 Schematic diagram of multimodal data acquisition of the present invention;

[0035] Figure 4 This is a workflow diagram of the dynamic parameter optimization engine of the present invention;

[0036] Figure 5 This is a topological relationship diagram of the multi-brand knowledge base of the present invention. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present invention are described clearly and completely below with reference to the accompanying drawings.

[0038] Example 1:

[0039] like Figures 1 to 5 As shown, a visual inspection method and system that can adapt to multiple brands of automotive parts. The hardware configuration is:

[0040] Data acquisition module:

[0041] High-resolution line scan camera: BaslerraL2048-48gm, with a resolution of 20 μm, a line frequency of 48 kHz, and support for 12-bit grayscale output;

[0042] TOF sensor: SICKOD3400, measurement accuracy ±0.05mm, sampling rate 1000Hz;

[0043] Tunable ring light source: CCSLDR2-65SW, 8 groups of LEDs are independently controlled, the wavelength range is 400-1000nm, and it supports polarizer switching.

[0044] Processing unit: NVIDIA Jetson AGX Xavier, built-in TensorRT acceleration library, model inference delay ≤ 50ms.

[0045] Implementation steps

[0046] Step 1: Synchronous acquisition of multimodal data

[0047] The trigger signal controls the line scan camera and the TOF sensor to start synchronously, and aligns them through hardware timestamps;

[0048] Dynamic adjustment of light source: According to the preset brand ID, such as brand A, the initial parameters in the knowledge base are loaded, the color temperature is 4500K, the wavelength is 850nm, and the polarized light mode is activated to suppress the reflection on the metal surface.

[0049] Step 2: Migrate cross-brand features

[0050] Model loading: Call the pre-trained model of brand A from the knowledge base, based on the ResNet-50 backbone network, and freeze the weights of the first three convolutional layers;

[0051] Transfer Learning:

[0052] Input the collected 2D image and 3D point cloud data, and generate joint features through the feature fusion layer and concatenate operation;

[0053] The cosine similarity is used to calculate the matching degree between the current feature and the knowledge base template. The similarity threshold is ≥ 0.85. If the matching fails, the online learning module is triggered.

[0054] Step 3: Dynamic parameter optimization

[0055] Reinforcement learning engine: using the DDPG algorithm, with reward function parameters α = 0.6, β = 0.3, γ = 0.1;

[0056] Real-time adjustments:

[0057] When uneven lighting is detected and the standard deviation is >15 gray levels, the brightness of LED groups 2, 4, and 6 will be automatically increased in steps of 10%, and the exposure time will be reduced to 80μs;

[0058] Kalman filtering is used to eliminate the point cloud position drift caused by mechanical vibration, and the vibration frequency is <200Hz.

[0059] Step 4: Defect determination and feedback

[0060] Multimodal fusion analysis:

[0061] 2D image detection of surface cracks, threshold: length ≥ 0.1 mm, width ≥ 5 μm;

[0062] 3D point cloud deformation detection, threshold: local curvature change ≥ 0.05mm -1 ;

[0063] Joint judgment: If the spatial overlap between 2D and 3D inspection results is ≥70%, it is judged as a real defect.

[0064] Knowledge base update: New defect types, such as plating bubbles unique to brand A, are annotated and stored in the knowledge base, triggering incremental learning. The model update cycle is 20 minutes.

[0065] Implementation Effect

[0066]

[0067] Example 2: A visual inspection method and system capable of adaptively and flexibly inspecting multiple brands of household appliances, suitable for high-robustness scenarios:

[0068] Hardware Configuration

[0069] Anti-interference design:

[0070] Redundant imaging channels: visible light (500-600nm) and infrared (850nm) cameras collect data in parallel;

[0071] Electromagnetic shielding cover: double-layer aluminum-magnesium alloy shell, shielding effectiveness ≥60dB (frequency band 10MHz-1GHz);

[0072] Pneumatic shock-absorbing platform: vibration isolation efficiency ≥ 90% (frequency ≤ 500Hz).

[0073] Implementation steps

[0074] Step 1: Automatic brand recognition

[0075] Barcode recognition mode: Scan the product QR code and call the detection template of brand B in the knowledge base;

[0076] Contour matching mode (no barcode):

[0077] Extract the hash value of the product outline (SIFT feature + LSH encoding) and match it with 5000+ brand outlines in the knowledge base, with a recognition accuracy of ≥99.9%.

[0078] Step 2: Few-shot defect learning

[0079] Data augmentation: SS-GAN is used to generate defect samples, such as scratches on plastic shells, with an image resolution of 2048×2048 and a PSNR ≥ 38dB.

[0080] Semi-supervised training:

[0081] Initial training: 10 real defect images + 200 generated images, cross entropy loss function, learning rate 1e-4;

[0082] Online optimization: For every 1,000 products inspected, suspected defects with a confidence level >95% are automatically screened and added to the training set after manual review.

[0083] Step 3: Anti-interference execution

[0084] Electromagnetic interference response:

[0085] When electromagnetic noise > 50V / m is detected, the system switches to the infrared imaging channel, reducing the interference by 80%;

[0086] Enable image denoising algorithm, non-local mean filtering + wavelet transform, retain defect details, and edge retention rate ≥ 90%.

[0087] Step 4: Multi-factory data sharing

[0088] Federated learning mechanism:

[0089] Each branch trains the model locally and only uploads the model gradient to the central knowledge base;

[0090] The global model aggregation period is 24 hours, the data leakage risk is close to zero, and the differential privacy parameter ε = 0.1.

[0091] Implementation Effect

[0092]

[0093] The present invention provides a multi-brand adaptive visual inspection method and system, which can automatically identify new brand products and load pre-trained models through a transfer learning framework and a multi-brand knowledge base, and achieve a high initial detection accuracy without the need for additional labeled data. By combining a self-supervised generative adversarial network with a federated learning mechanism, a high defect detection rate and a low false alarm rate can be achieved even when the number of defect samples is small. The online learning module supports dynamic updating of model weights, and the inclusion cycle of new defect types is greatly shortened compared to traditional solutions. The DDPG reinforcement learning algorithm is used to dynamically adjust the light source and camera parameters, adaptively improve imaging quality, and use redundant imaging channels to work in conjunction with the Kalman filter to enhance anti-interference capabilities.

[0094] Example 3: A self-adaptable multi-brand mixed-line visual inspection method and system for micro-electronic components is suitable for micron-level high-precision scenarios:

[0095] Application Scenario

[0096] Test objects: mobile phone connectors of different brands, gold-plated contact oxidation, plastic shell micro-cracks, pin coplanarity deviation;

[0097] Hardware Configuration

[0098] Optical imaging module:

[0099] Dual-channel imaging system:

[0100] Channel 1: Telecentric lens (OptoEngineeringTC23M036) with a 5-megapixel CMOS camera (120 fps), 3.5 μm / pixel resolution, for 2D surface inspection.

[0101] Channel 2: Laser confocal sensor (Keyence LJ-V7000), with a Z-axis resolution of 0.1 μm, for 3D topography and coplanarity measurement;

[0102] Multispectral light source:

[0103] Coaxial white light (color temperature 5500K) + ultraviolet band (365nm) combined lighting is used to stimulate the fluorescence effect of plastic materials;

[0104] Circularly polarized light source (extinction ratio 100:1) suppresses metal reflection interference.

[0105] Processing Unit:

[0106] Embedded FPGA (Xilinx Zynq UltraScale+MPSoC) implements hardware-level image preprocessing (ROI extraction, non-uniformity correction), with processing delay less than 5ms.

[0107] Implementation steps

[0108] Step 1: Brand Adaptive Loading

[0109] Automatic identification: Match the connector pinout pattern (5×4 matrix for brand A, 6×3 matrix for brand B) with the knowledge base and call the corresponding detection template;

[0110] Model hot switching: retain the backbone network (ResNet-50) weights and only replace the fully connected layer parameters (switching time 0.2 seconds).

[0111] Step 2: Joint detection across materials

[0112] 2D channel detection:

[0113] Under UV excitation, microcracks (≥10 μm) in the plastic shell appear as dark stripes due to fluorescence quenching and are located using the U-Net segmentation network;

[0114] Metal contact oxidation spots are detected using multispectral colorimetry, and an R / G / B channel difference ratio greater than 2.5 is considered abnormal.

[0115] 3D channel detection:

[0116] Laser confocal scanning of pin height, calculation of coplanarity, maximum height difference ≤ 8μm;

[0117] Dynamic compensation: Based on the thermal expansion coefficient and the material CTE value stored in the knowledge base, temperature drift is corrected in real time, with an error of <0.3μm within ±5℃.

[0118] Step 3: Few-shot defect learning

[0119] Zero-sample startup: To address the unique defects of new brand C, such as plating bubbles, the following process is implemented:

[0120] Using self-supervised learning, the SimCLR framework extracts contrastive features from unlabeled data;

[0121] Manually annotate 5 defect samples and generate an initial detection model through the Prototypical Network.

[0122] Online learning phase: For every 100 products inspected, suspected defects are automatically clustered and the model is updated after manual confirmation. It only takes 4 hours to increase mAP to 90%.

[0123] Step 4: High-speed pipeline optimization

[0124] Parallel computing architecture:

[0125] FPGA preprocessing, denoising, and ROI cropping are executed in parallel with GPU model inference and a lightweight version of YOLOv5s, resulting in a total inspection time of 1.8 seconds for a single piece.

[0126] Hardware synchronization control:

[0127] The robot gripping, light source triggering and camera exposure are synchronized via the EtherCAT bus, with a timing error of <0.1ms.

[0128] Implementation Effect

[0129]

[0130] The present invention provides a multi-brand adaptive visual inspection method and system, which can automatically identify new brand products and load pre-trained models through a transfer learning framework and a multi-brand knowledge base, and achieve a high initial detection accuracy without the need for additional labeled data. By combining a self-supervised generative adversarial network with a federated learning mechanism, a high defect detection rate and a low false alarm rate can be achieved even when the number of defect samples is small. The online learning module supports dynamic updating of model weights, and the inclusion cycle of new defect types is greatly shortened compared to traditional solutions. The DDPG reinforcement learning algorithm is used to dynamically adjust the light source and camera parameters, adaptively improve imaging quality, and use redundant imaging channels to work in conjunction with the Kalman filter to enhance anti-interference capabilities.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A visual inspection method that is adaptive to multiple brands, characterized in that: The following steps are involved: S1: Synchronously collect 2D image data and 3D point cloud data of the product to be inspected using a multimodal sensor, wherein the multimodal sensor includes a high-resolution line scan camera, a TOF sensor, and a tunable ring light source; S2: Match the multi-brand knowledge base based on product identification or appearance features, and call the corresponding brand's feature template and defect judgment criteria; S3: The data collected in S1 is input into an adaptive feature extraction network. The network uses transfer learning to initialize model parameters and updates weights in real time through online learning. Specifically, the following steps are performed: Cross-brand feature transfer module: uses the pre-trained ResNet-50 backbone network to extract common features; Small Sample Defect Learning Module: Expanding defect samples through self-supervised generative adversarial networks (SS-GAN) S4: Dynamically optimizes imaging parameters based on a reinforcement learning algorithm, including light source intensity, camera exposure time, and image filter threshold, with the goal of maximizing the signal-to-noise ratio in the defect area. S5: Multimodal data is fused at the feature level to output a test report containing defect type, location, and confidence level, and the test results are fed back to the knowledge base for incremental learning.

2. The adaptive multi-brand visual inspection method according to claim 1, characterized in that: In step S1, the multimodal data acquisition eliminates image distortion caused by mechanical vibration through the Kalman filter algorithm.

3. The adaptive multi-brand visual inspection method according to claim 1, characterized in that: The implementation method of transfer learning in step S3 is: For new brand products, freeze the first three convolutional layers of the backbone network and only fine-tune the parameters of the fully connected layer; The cosine similarity is used to measure the differences in feature distribution across brands and dynamically adjust the transfer learning weight coefficient.

4. The adaptive multi-brand visual inspection method according to claim 1, characterized in that: In step S4, the reinforcement learning adopts the deep deterministic policy gradient (DDPG) algorithm, and its reward function is designed as: R = α·defect contrast + β·image uniformity - γ·hardware energy consumption.

5. A multi-brand adaptive visual inspection system, comprising the multi-brand adaptive visual inspection method according to any one of claims 1 to 4, characterized in that: include: Data acquisition module: consists of a high-resolution line scan camera, a TOF sensor, and a tunable ring light source; Embedded processing unit: equipped with NVIDIA Jetson AGX Xavier processor, built-in adaptive feature extraction network and dynamic parameter optimization engine; Multi-brand knowledge base: uses a distributed graph database to store the characteristic topological relationships, defect history data, and optimization strategies of each brand's products; Anti-interference actuator: includes electromagnetic shielding cover, pneumatic shock absorption platform and redundant imaging channel.

6. The multi-brand adaptive visual inspection system according to claim 5, characterized in that: The tunable annular light source comprises 8 groups of independently controlled LED arrays, and the adjustable wavelength range of each group is 400nm-1000nm.

7. The multi-brand adaptive visual inspection system according to claim 5, characterized in that: The adaptive feature extraction network achieves lightweight deployment through model pruning and quantization technology.

Images