Industrial camera system with integrated self-calibration, self-detection, self-cleaning functions

Abstract

The application discloses an industrial camera system integrated with self-calibration, self-detection and self-cleaning functions and belongs to the technical field of image acquisition.The industrial camera system comprises an online automatic focusing module, a dust identification module based on image features and a pneumatic cleaning system.The online automatic focusing module is used for real-time evaluation of image definition and dynamic adjustment of lens group position.The dust identification module based on image features is used for identification of particle pollution on the surface of a lens or protective glass based on a dust identification model, positioning of a pollution area and determination of an actual pollution physical area, a pollution grade and a defect shape type.The pneumatic cleaning system is used for determination of a cleaning strategy based on the pollution area, the pollution grade and the defect shape type, non-contact compressed air blowing on the surface of the lens according to the cleaning strategy and removal of pollution.The application realizes five-in-one collaborative operation of online automatic focusing, dust self-detection, pneumatic self-cleaning, optical self-calibration and source pollution prevention, and can complete lens pollution discrimination, self-adaptive cleaning, distortion correction and thermal drift compensation without manual intervention.

Description

Technical Field

[0001] This invention relates to the field of image acquisition technology, and more specifically to an industrial camera system integrating self-calibration, self-detection, and self-cleaning functions. Background Technology

[0002] With the continuous improvement of modern industrial automation, visual inspection systems have become an indispensable key component in the field of intelligent manufacturing. Especially in scenarios such as semiconductor wafer inspection, food packaging defect identification, and electronic component solder joint quality control, industrial camera systems undertake high-precision, high-speed image acquisition tasks, and their stability and continuous operation capability directly determine the overall production line's operational efficiency. However, in actual industrial environments, due to the long-term accumulation of pollutants such as dust, oil, and moisture, the surface of industrial camera lenses is easily contaminated, leading to problems such as image blurring, edge distortion, and decreased contrast. In severe cases, this can even result in incorrect defect identification or missed detection. Furthermore, some industries experience problems such as optical axis misalignment and sensor thermal drift caused by high-frequency vibration or thermal expansion in their production sites, causing initial calibration data to gradually become invalid over time, resulting in image distortion and increased dimensional measurement errors. These problems are particularly serious in unattended automated production lines, because once the inspection system experiences accuracy degradation, line shutdown and manual maintenance are required, increasing labor costs and potentially leading to quality accidents.

[0003] Traditional solutions mainly include periodic manual cleaning of lenses, replacement of filters, or recalibration of the camera system. However, the periodic manual intervention required makes it difficult for the entire inspection system to achieve true long-term self-stability. Especially in scenarios with high cleanliness requirements and where frequent manual operation is not feasible (such as wafer cleanrooms and pharmaceutical packaging lines), traditional maintenance methods are no longer suitable for the high availability and efficiency demands of modern intelligent manufacturing. Therefore, achieving the "self-cleaning, self-calibrating, and self-maintaining" capabilities of industrial cameras in complex working environments has become one of the cutting-edge directions in current vision system research.

[0004] Based on this, the present invention designs an industrial camera system integrating self-calibration, self-detection, and self-cleaning functions to solve the above problems. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the existing technology, the present invention provides an industrial camera system that integrates self-calibration, self-detection and self-cleaning functions.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An industrial camera system integrating self-calibration, self-testing, and self-cleaning functions, including:

[0008] Online autofocus module: used to assess image sharpness in real time and dynamically adjust the position of the lens group;

[0009] Image feature-based dust recognition module: Based on the dust recognition model, it identifies particulate contamination on the surface of the lens or protective glass, locates the contaminated area, and determines the actual physical area of ​​contamination, contamination level, and defect shape type;

[0010] Pneumatic cleaning system: Based on the contaminated area, contamination level and defect shape type, a cleaning strategy is determined, and non-contact compressed air is blown onto the lens surface according to the cleaning strategy to remove contamination;

[0011] Optical self-calibration module: used to periodically correct lens distortion, optical axis misalignment and sensor thermal drift, maintaining image geometric accuracy and measurement accuracy;

[0012] Anti-pollution design structure: used to prevent dust, oil, and moisture from adhering to the lens surface at the source.

[0013] Furthermore, the online autofocus module operates as follows:

[0014] Step 1.1 The camera acquires the current focal length. Image ;

[0015] Step 1.2. Calculate the gradient magnitude of all pixels in the image. ;

[0016] Step 1.3. By gradient magnitude Calculate normalized probability ;

[0017] Step 1.4. Based on the normalized probability Calculate the gradient contrast transformation entropy To obtain the current clear score;

[0018] Step 1.5. Iteratively adjust the focal length using Newton's iteration or quasi-Newton BFGS optimization methods. ,renew ;

[0019] After steps 1.6.3 to 5 iterations, lock. The largest It completes focusing and enables dynamic adjustment of the lens group position.

[0020] Furthermore, step 1.2. calculates the gradient magnitude of all pixels in the image. The specific steps are as follows:

[0021] Step 1.2.1: Define a 3×3 Sobel convolution kernel, and use two matrices to calculate the horizontal gradient kernel. Vertical gradient kernel ;

[0022] Step 1.2.2: Convolution calculation of horizontal gradient kernel Vertical gradient kernel ;

[0023] Step 1.2.3: Through the horizontal gradient kernel Vertical gradient kernel Synthetic gradient magnitude .

[0024] Furthermore, the dust recognition module based on image features operates as follows:

[0025] Step 2.1. Based on binary contamination mask First, pixel-level area statistics are performed in the airspace to determine the pixel contamination area. Then, the actual contamination physical area is converted through camera calibration, and the contamination level is determined.

[0026] Step 2.2 Training and optimization of the dust recognition model;

[0027] Step 2.3. Based on binary contamination mask Determine the type of defect shape.

[0028] Furthermore, step 2.1. Based on a binary contamination mask First, pixel-level area statistics are performed in the airspace to determine the pixel contamination area. Then, the actual contamination physical area is converted through camera calibration, and the contamination level is determined. The specific operation is as follows:

[0029] Step 2.1.1 Calculation of pixel-level contamination area;

[0030] Step 2.1.2 Calculate the actual physical area of ​​pollution and the pollution level based on the pixel pollution area.

[0031] Furthermore, step 2.2 involves training and optimizing the dust recognition model;

[0032] Step 2.2.1 Training and Multi-Scene Robust Optimization of the Dust Recognition Model;

[0033] Step 2.2.2 Spectral feature space regularization and model optimization;

[0034] Step 2.2.3 Model dynamic self-learning update.

[0035] Furthermore, step 2.3. Based on a binary contamination mask Determine the type of defect shape;

[0036] Step 2.3.1 Applying a binary contamination mask Perform morphological opening operations (denoising and filling small holes) to obtain a clean binary contaminated region mask. To avoid noise interfering with frequency domain analysis;

[0037] Step 2.3.2 Pre-treated binary contamination mask Perform 2D-FFT processing to obtain the frequency domain amplitude spectrum. :

[0038] Step 2.3.3 Extraction of spectral domain shape features.

[0039] Furthermore, the specific structure of the dust recognition model includes an image input layer, a spatial contamination segmentation branch, a frequency domain shape representation branch, a feature regularization fusion layer, a decision output layer, and a model optimization and self-learning layer;

[0040] Image input layer: The input consists of N consecutive frames of grayscale images captured by the camera. , serving as the global feature data source for the model;

[0041] The spatial contamination segmentation branch consists of a multi-frame cumulative difference unit, a binary mask generation unit, and a morphological filtering unit. It completes the localization and pixel-level segmentation of contaminated areas and outputs a clean binary contamination mask. This corresponds to the model segmentation and inference task;

[0042] Frequency domain shape characterization branch: Composed of 2D-FFT transform unit, low frequency contour reconstruction unit, Fourier coefficient extraction unit, and geometric feature calculation unit, it completes the standardization and shape quantization of pollution contours and outputs roundness, eccentricity, aspect ratio, compactness and spectral distribution characteristics.

[0043] Feature regularization fusion layer: Apply feature similarity constraints and feature interval constraints to the spectral / grayscale features of the polluted area, construct regularization loss, suppress noise interference and enhance the boundary distinction of different polluted areas;

[0044] Decision output layer: Outputs pollution level based on pollution physical area, outputs defect shape type based on spectrum energy distribution, and finally outputs four types of decision results: pollution area, area, pollution level, and defect shape type.

[0045] Model optimization and self-learning layer: It consists of multi-scene generalization training unit, composite loss calculation unit and online self-supervised update unit. It optimizes the model by minimizing the joint loss and composite loss of multiple scenes, and realizes long-term online iterative update of model parameters through weakly labeled data.

[0046] Furthermore, the pneumatic cleaning system operates as follows:

[0047] Step 3.1 Determine if it is lightly contaminated. If it is, proceed to step 3.2. If it is not, proceed to step 3.4.

[0048] Step 3.2 Determine whether it is a defect shape type in the low frequency domain. If the determination is yes, execute cleaning strategy 1 and then execute step 3.9. If the determination is no, execute step 3.3.

[0049] Step 3.3 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 2 and then execute step 3.9. If it is not, execute cleaning strategy 3 and then execute step 3.9.

[0050] Step 3.4 Determine if it is moderately contaminated. If it is, proceed to step 3.5. If it is not, proceed to step 3.7.

[0051] Step 3.5 Determine whether it is a defect shape type in the low frequency domain. If it is, execute cleaning strategy 4 and then execute step 3.9. If it is not, execute step 3.6.

[0052] Step 3.6 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 5 and then execute step 3.9. If it is not, execute cleaning strategy 6 and then execute step 3.9.

[0053] Step 3.7 Determine whether it is a defect shape type in the low frequency domain. If it is, execute cleaning strategy 7 and then execute step 3.9. If it is not, execute step 3.8.

[0054] Step 3.8 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 8 and then execute step 3.9. If it is not, execute cleaning strategy 9 and then execute step 3.9.

[0055] Step 3.9 The image feature-based dust recognition module continues to identify the actual physical area of ​​contamination and determines whether the actual physical area of ​​contamination is less than the cleanliness threshold. If the determination is yes, cleaning is stopped; if the determination is no, the contamination level and defect shape type are calculated, and then step 3.1 is executed.

[0056] Beneficial effects: This invention achieves five-in-one coordinated operation of online autofocus, dust self-detection, pneumatic self-cleaning, optical self-calibration, and source pollution prevention. It can complete lens contamination identification, adaptive cleaning, distortion correction, and thermal drift compensation without manual intervention, completely solving the problems of frequent manual maintenance and high downtime costs in industrial sites. At the same time, it adopts a dust recognition model based on spatial domain segmentation and frequency domain shape representation, combined with robust training in multiple scenarios and feature regularization optimization, which can accurately identify the area, level, and shape type of contamination, providing a refined strategy basis for pneumatic cleaning, and significantly improving cleaning efficiency and recognition accuracy. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0058] Figure 1 This is a block diagram of an industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0060] The present invention will be further described below with reference to embodiments.

[0061] Example 1: An industrial camera system integrating self-calibration, self-testing, and self-cleaning functions, comprising:

[0062] Online autofocus module: used to assess image sharpness in real time and dynamically adjust the position of the lens group;

[0063] To ensure that the image is always in optimal sharpness;

[0064] Image feature-based dust recognition module: Based on the dust recognition model, it identifies particulate contamination on the surface of the lens or protective glass, locates the contaminated area, and determines the actual physical area of ​​contamination, contamination level, and defect shape type;

[0065] Pneumatic cleaning system: Based on the contaminated area, contamination level and defect shape type, a cleaning strategy is determined, and non-contact compressed air is blown onto the lens surface according to the cleaning strategy to remove contamination;

[0066] Optical self-calibration module: used to periodically correct lens distortion, optical axis misalignment and sensor thermal drift, maintaining image geometric accuracy and measurement accuracy;

[0067] Anti-pollution design structure: used to prevent dust, oil, and moisture from adhering to the lens surface at the source;

[0068] Reduce the probability of pollution occurring;

[0069] The specific operation of the online autofocus module is as follows:

[0070] Step 1.1 The camera acquires the current focal length. Image ;

[0071] Step 1.2. Calculate the gradient magnitude of all pixels in the image. ;

[0072] Step 1.2.1: Define a 3×3 Sobel convolution kernel, and use two matrices to calculate the horizontal gradient kernel. Vertical gradient kernel ;

[0073] Horizontal gradient kernel Detect left-right changes:

[0074]

[0075] Vertical gradient kernel Detect vertical changes:

[0076]

[0077] Step 1.2.2: Convolution calculation of horizontal gradient kernel Vertical gradient kernel ;

[0078] In pixels Taking the center, take the gray values ​​of the surrounding 3×3 pixels, multiply them with the corresponding kernel values, and then sum them to obtain the horizontal / vertical gradient of that point;

[0079]

[0080]

[0081] Step 1.2.3: Through the horizontal gradient kernel Vertical gradient kernel Synthetic gradient magnitude

[0082]

[0083] Step 1.3. By gradient magnitude Calculate normalized probability ;

[0084]

[0085] Step 1.4. Based on the normalized probability Calculate the gradient contrast transformation entropy To obtain the current clear score;

[0086]

[0087] Step 1.5. Iteratively adjust the focal length using Newton's iteration or quasi-Newton BFGS optimization methods. ,renew ;

[0088] After steps 1.6.3 to 5 iterations, lock. The largest It completes focusing and enables dynamic adjustment of the lens group position.

[0089]

[0090] The dust recognition module based on image features operates as follows:

[0091] Step 2.1. Based on binary contamination mask First, pixel-level area statistics are performed in the airspace to determine the pixel contamination area. Then, the actual contamination physical area is converted through camera calibration, and the contamination level is determined.

[0092] Step 2.1.1 Calculation of pixel-level contamination area

[0093] The total number of pixels with a value of 1 in the mask represents the pixel contamination area of ​​the contaminated region. :

[0094]

[0095]

[0096]

[0097] Image width, Image height, For the first Frame image in coordinates The pixel grayscale value at that location, For the first Frame image in coordinates The pixel grayscale value at that location, Let (x, y) be the mean of the cumulative differences across multiple frames. The total number of consecutive image frames used to calculate the cumulative difference. , The pollution response threshold;

[0098] Step 2.1.2 Calculate the actual physical area and pollution level of the contaminated area based on the pixel contamination area;

[0099] Calculate the actual physical area of ​​pollution by combining the physical size s of a single camera pixel (unit: mm / pixel, obtained from camera calibration). :

[0100]

[0101] Percentage of polluted area and the percentage of polluted area Classify pollution levels;

[0102]

[0103] ;

[0104] Step 2.2 Training and optimization of the dust recognition model;

[0105] Step 2.2.1 Dust Recognition Model Training and Multi-Scene Robust Optimization

[0106] To improve the stability of pollution identification under different working conditions, a multi-scene imaging dataset was constructed. The dust recognition model was trained using this dataset, with the training process aiming to minimize the joint loss across multiple scenes.

[0107] The multi-scene imaging dataset is as follows:

[0108]

[0109] This represents an image sample acquired under specific imaging conditions;

[0110] This includes pollution images, focus images, and calibration plate images under different dust concentrations, light levels, and temperatures.

[0111] Minimize the joint loss across multiple scenarios :

[0112]

[0113] Confidence weighting coefficients for each scenario For the dust recognition model in the first The basic loss on each scenario sample set is used to improve the model's generalization ability across working conditions.

[0114] Step 2.2.2 Spectral Feature Space Regularization and Model Optimization

[0115] By incorporating similarity and interval constraints into pollution identification, the fine-grained accuracy of pollution area shape classification and area calculation is enhanced, making cleaning strategy judgments more accurate.

[0116] Set band and The corresponding predicted values ​​of the pollution area characteristics are as follows: Loss of similarity in characteristics within the same polluted area ;

[0117]

[0118] It is the same contaminated area;

[0119] Introducing interval loss to different polluted areas Strengthen boundary distinction:

[0120]

[0121] For different polluted areas, The feature interval threshold;

[0122] Constructing a composite loss function Optimize the dust recognition model:

[0123]

[0124]

[0125]

[0126]

[0127] , For loss weighting coefficients, This is a pixel-level realistic contamination mask (manually labeled / ground value, 1 = contamination, 0 = clean). Output a predicted contamination mask for the dust recognition model. : Pollution shape true category, Output the predicted shape category for the dust recognition model. For classification loss weights;

[0128] Defect shape type in low frequency domain ;

[0129] Defect shape type in the mid-frequency domain ;

[0130] Defect shape type in the high-frequency domain ;

[0131] Step 2.2.3 Model Dynamic Self-Learning Update

[0132] Spectral entropy Evaluation band prediction results The credibility; when the spectral entropy is greater than a preset threshold When this happens, the sample will be automatically added to the weakly labeled data pool. Every update cycle We utilize a weakly labeled data pool to perform self-supervised fine-tuning of the dust recognition model and update the model parameters according to the gradient descent rule.

[0133]

[0134] For the current parameters of the model, For learning rate, The base loss for contamination identification calculated on the weakly labeled data pool, This represents the gradient of the base loss with respect to the model parameters on the weakly labeled data pool.

[0135] Step 2.3. Based on binary contamination mask Determine the type of defect shape;

[0136] Step 2.3.1 Applying a binary contamination mask Perform morphological opening operations (denoising and filling small holes) to obtain a clean binary contaminated region mask. To avoid noise interfering with frequency domain analysis.

[0137] Step 2.3.2 Pre-treated binary contamination mask Perform 2D-FFT processing to obtain the frequency domain amplitude spectrum. :

[0138]

[0139] For two-dimensional frequency domain horizontal coordinates, Two-dimensional frequency domain vertical coordinates;

[0140] Step 2.3.3 Extraction of Spectral Domain Shape Features

[0141] Step 2.3.3.1 Determine the frequency domain amplitude spectrum low-frequency principal components Then, perform inverse 2D-IFFT processing to filter out high-frequency noise and obtain the standard contour of the polluted area. :

[0142]

[0143] Step 2.3.3.2 Calculate the standard profile of the contaminated area Contour centroid, normalized contour coordinates, before extraction The principal Fourier coefficients describe the core features of the shape; then, the roundness, eccentricity, aspect ratio, and compactness are calculated to quantify the regularity of the shape.

[0144] Step 2.3.3.3 Based on the frequency domain amplitude spectrum Calculate the Euclidean distance from any point in the frequency domain to the center. Then convert the Euclidean distance With upper limit value and lower limit value The spectral energy distribution is determined by comparison, and then the defect shape type is determined based on the spectral energy distribution;

[0145] Euclidean distance The calculation is as follows:

[0146]

[0147] The coordinates of the frequency domain center are... , ;

[0148]

[0149]

[0150]

[0151] Defect shape types in the low-frequency domain: dot-like, circular, small block-like;

[0152] Defect shape types in the mid-frequency domain: strip-shaped, sheet-shaped, diffuse;

[0153] Defect shape types in the high-frequency domain: sharp edges, debris-like contamination.

[0154] The specific structure of the dust recognition model includes an image input layer, a spatial domain contamination segmentation branch, a frequency domain shape representation branch, a feature regularization fusion layer, a decision output layer, and a model optimization and self-learning layer.

[0155] Image input layer: The input consists of N consecutive frames of grayscale images captured by the camera. , serving as the global feature data source for the model.

[0156] The spatial contamination segmentation branch consists of a multi-frame cumulative difference unit, a binary mask generation unit, and a morphological filtering unit. It completes the localization and pixel-level segmentation of contaminated areas and outputs a clean binary contamination mask. This corresponds to the model segmentation and inference task.

[0157] Frequency domain shape characterization branch: Composed of 2D-FFT transform unit, low frequency contour reconstruction unit, Fourier coefficient extraction unit, and geometric feature calculation unit, it completes the standardization and shape quantization of pollution contours and outputs roundness, eccentricity, aspect ratio, compactness and spectral distribution characteristics.

[0158] Feature regularization fusion layer: Apply feature similarity constraints and feature interval constraints to the spectral / grayscale features of the polluted area, construct regularization loss, suppress noise interference and enhance the boundary distinction of different polluted areas.

[0159] Decision output layer: Outputs pollution level based on the physical area of ​​pollution, and outputs defect shape type (point / circular / small block, strip / sheet / diffuse, sharp edge / fragmented pollution) based on the spectral energy distribution. Finally, it outputs four types of decision results: pollution area, area, pollution level, and defect shape type.

[0160] Model optimization and self-learning layer: It consists of multi-scene generalization training unit, composite loss calculation unit and online self-supervised update unit. It optimizes the model by minimizing the joint loss and composite loss of multiple scenes, and realizes long-term online iterative update of model parameters through weakly labeled data.

[0161] The specific operation of the pneumatic cleaning system is as follows:

[0162] Step 3.1 Determine if it is lightly contaminated. If it is, proceed to step 3.2. If it is not, proceed to step 3.4.

[0163] Step 3.2 Determine whether it is a defect shape type in the low frequency domain. If the determination is yes, execute cleaning strategy 1 and then execute step 3.9. If the determination is no, execute step 3.3.

[0164] Step 3.3 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 2 and then execute step 3.9. If it is not, execute cleaning strategy 3 and then execute step 3.9.

[0165] Step 3.4 Determine if it is moderately contaminated. If it is, proceed to step 3.5. If it is not, proceed to step 3.7.

[0166] Step 3.5 Determine whether it is a defect shape type in the low frequency domain. If it is, execute cleaning strategy 4 and then execute step 3.9. If it is not, execute step 3.6.

[0167] Step 3.6 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 5 and then execute step 3.9. If it is not, execute cleaning strategy 6 and then execute step 3.9.

[0168] Step 3.7 Determine whether it is a defect shape type in the low frequency domain. If it is, execute cleaning strategy 7 and then execute step 3.9. If it is not, execute step 3.8.

[0169] Step 3.8 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 8 and then execute step 3.9. If it is not, execute cleaning strategy 9 and then execute step 3.9.

[0170] Step 3.9 The dust recognition module based on image features continues to identify the actual physical area of ​​contamination and determines whether the actual physical area of ​​contamination is less than the cleanliness threshold. If the determination is yes, cleaning is stopped. If the determination is no, the contamination level and defect shape type are calculated again, and then step 3.1 is executed.

[0171] The cleanliness threshold is set at ≤0.1% of the contaminated area.

[0172] The specific cleaning strategies are as follows:

[0173] Cleaning strategy types Specific operations Cleaning Strategy 1 Low pressure + short duration + 1 center-point pulse jet Cleaning Strategy 2 Low pressure + medium duration + 1 linear sweep jet Cleaning Strategy 3 Low pressure + long duration + 1 full-area light spray Cleaning Strategy 4 Medium pressure + short duration + 2 center-point jetting Cleaning Strategy 5 Medium pressure + medium duration + 2 linear reciprocating jets Cleaning Strategy 6 Medium pressure + long duration + 2 full-area sweep jets Cleaning Strategy 7 High pressure + medium duration + 2-3 times of enhanced center jetting Cleaning Strategy 8 High pressure + long duration + 3 linear full-coverage jetting Cleaning Strategy 9 High pressure + long duration + 3 multi-point annular jet cleaning

[0174] Low pressure is 0.2~0.4 MPa, medium pressure is 0.4~0.6 MPa, and high pressure is 0.6~0.8 MPa;

[0175] Short duration: 0.2~0.5s; Medium duration: 0.5~1.0s; Long duration: 1.0~1.5s

[0176] Center-point jet cleaning: A single jet of air is directed at the center of the contaminated area;

[0177] Linear sweeping jetting: jetting at a constant speed along the long axis of the contaminated area;

[0178] Full-area sweeping spray: Uniform spraying across the entire lens surface;

[0179] Multi-point circular jetting: jetting in a circular pattern centered on the polluted area.

[0180] The specific operation of the optical self-calibration module is as follows:

[0181] Step 4.1 Distortion Modeling

[0182] A pinhole camera model and a sixth-order radial distortion model are used to jointly model and establish the mapping relationship between ideal distortion-free coordinates and actual distorted pixel positions:

[0183]

[0184]

[0185] For ideal, distortion-free coordinates, This represents the actual pixel position after distortion. Radial distortion coefficient, , This is the optical axis offset compensation amount. This represents the radial distance.

[0186] Step 4.2 Calibration Data Acquisition

[0187] Periodically trigger the camera to photograph the standard checkerboard calibration board and extract the set of corner points of the calibration board. Obtain the corner points of the ideal model .

[0188] Step 4.3 Calculation of reprojection error

[0189] Corner points of the ideal model Substituting the values ​​into the projection function and comparing them with the actual observed corner points, we construct the reprojection error:

[0190]

[0191] For the camera intrinsic parameter matrix, This is the projection transformation function.

[0192] Step 4.4 Parameter Optimization Solution

[0193] The Levenberg-Marquardt optimization algorithm is employed to jointly and iteratively optimize the camera intrinsic parameter matrix, with the goal of minimizing the reprojection error. With radial distortion coefficient .

[0194] Step 4.5 Distortion Inversion Correction

[0195] The optimized intrinsic parameter matrix With radial distortion coefficient By substituting the distortion model into the acquired image, distortion inversion correction is performed to complete online lens distortion correction.

[0196] Step 4.6 Periodic Updates and Error Compensation

[0197] The above calibration process is performed at fixed intervals to simultaneously correct measurement errors caused by optical axis misalignment and sensor thermal drift, thereby continuously maintaining image geometric accuracy and measurement accuracy.

[0198] The anti-pollution design structure includes an electrostatic precipitator and a nano-oleophobic coating;

[0199] A nano-oleophobic coating is applied to the lens glass surface, and its contact angle... Usually exceeding This significantly reduces the adhesion of liquids or particles;

[0200] Adhesion The calculation is as follows:

[0201]

[0202] The liquid-gas interfacial tension;

[0203] Oleophobic coating effectively reduces This enhances the efficiency of pollutants being carried away by airflow.

[0204] An electrostatic precipitator is integrated on the side of the lens, which is controlled by voltage. With electric field gradient Applying force to pollutant particles:

[0205]

[0206]

[0207] Adjust the electrode spacing appropriately. It can make the electrostatic force greater than the particle adhesion force. This enables active particle repulsion.

[0208] This invention achieves five-in-one coordinated operation of online autofocus, dust self-detection, pneumatic self-cleaning, optical self-calibration, and source pollution prevention. It can complete lens contamination identification, adaptive cleaning, distortion correction, and thermal drift compensation without manual intervention, completely solving the problems of frequent manual maintenance and high downtime costs in industrial sites. At the same time, it adopts a dust recognition model based on spatial domain segmentation and frequency domain shape representation, combined with robust training in multiple scenarios and feature regularization optimization, which can accurately identify the area, level, and shape type of contamination, providing a refined strategy basis for pneumatic cleaning, and significantly improving cleaning efficiency and recognition accuracy.

[0209] By periodically correcting lens distortion, optical axis misalignment, and sensor thermal drift through an optical self-calibration module, the system continuously maintains image geometric accuracy and measurement stability, meeting the long-term operational needs of high-precision visual inspection scenarios such as semiconductors, electronics, and food.

[0210] To adapt to the parallel operation of multiple machines, complex working conditions, and long-term unattended operation in industrial production lines, the system adds the following extended implementation logic:

[0211] I. Multi-device synchronization and parallel processing

[0212] A master-slave global clock synchronization is adopted to determine the global synchronization time signal of the device, and the imaging error of multiple cameras is controlled within 0.5ms. A GPU parallel processing + CPU scheduling architecture is used, with a single frame processing cycle of ≤10ms, to ensure the timing consistency of simultaneous detection, cleaning and calibration of multiple machines.

[0213] Device global synchronization time signal The calculation is as follows:

[0214]

[0215] As a global reference clock, For the first Time deviation of industrial cameras in Taiwan;

[0216] GPU parallel execution: image gradient calculation, contamination mask extraction, 2D-FFT shape analysis, distortion inversion and correction;

[0217] CPU scheduling execution: cleaning strategy judgment, spray control, calibration triggering, and anti-contamination electrostatic voltage adjustment.

[0218] The system integrates a nano-oleophobic coating and an electrostatic precipitator for dual anti-pollution structure, which inhibits the adhesion of dust, oil, and moisture at the source. With multi-device clock synchronization and GPU parallel processing architecture, it can be adapted to multi-machine parallel operation in production lines, complex working conditions, and unattended operation scenarios, making the system highly versatile and industrially practical.

[0219] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An industrial camera system integrating self-calibration, self-testing, and self-cleaning functions, characterized in that: include: Online autofocus module: used to assess image sharpness in real time and dynamically adjust the position of the lens group; Image feature-based dust recognition module: Based on the dust recognition model, it identifies particulate contamination on the surface of the lens or protective glass, locates the contaminated area, and determines the actual physical area of ​​contamination, contamination level, and defect shape type; Pneumatic cleaning system: Based on the contaminated area, contamination level and defect shape type, a cleaning strategy is determined, and non-contact compressed air is blown onto the lens surface according to the cleaning strategy to remove contamination; Optical self-calibration module: used to periodically correct lens distortion, optical axis misalignment and sensor thermal drift, maintaining image geometric accuracy and measurement accuracy; Anti-pollution design structure: used to prevent dust, oil, and moisture from adhering to the lens surface at the source.

2. The industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to claim 1, characterized in that, The specific operation of the online autofocus module is as follows: Step 1.1 The camera acquires the current focal length. Image ; Step 1.

2. Calculate the gradient magnitude of all pixels in the image. ; Step 1.

3. By gradient magnitude Calculate normalized probability ; Step 1.

4. Based on the normalized probability Calculate the gradient contrast transformation entropy To obtain the current clear score; Step 1.

5. Iteratively adjust the focal length using Newton's iteration or quasi-Newton BFGS optimization methods. ,renew ; After steps 1.6.3 to 5 iterations, lock. The largest It completes focusing and enables dynamic adjustment of the lens group position.

3. The industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to claim 2, characterized in that, Step 1.

2. Calculate the gradient magnitude of all pixels in the image. The specific steps are as follows: Step 1.2.1: Define a 3×3 Sobel convolution kernel, and use two matrices to calculate the horizontal gradient kernel. Vertical gradient kernel ; Step 1.2.2: Convolution calculation of horizontal gradient kernel Vertical gradient kernel ; Step 1.2.3: Through the horizontal gradient kernel Vertical gradient kernel Synthetic gradient magnitude .

4. The industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to claim 3, characterized in that, The dust recognition module based on image features operates as follows: Step 2.

1. Based on binary contamination mask First, pixel-level area statistics are performed in the airspace to determine the pixel contamination area. Then, the actual contamination physical area is converted through camera calibration, and the contamination level is determined. Step 2.2 Training and optimization of the dust recognition model; Step 2.

3. Based on binary contamination mask Determine the type of defect shape.

5. The industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to claim 4, characterized in that, Step 2.

1. Based on binary contamination mask First, pixel-level area statistics are performed in the airspace to determine the pixel contamination area. Then, the actual contamination physical area is converted through camera calibration, and the contamination level is determined. The specific operation is as follows: Step 2.1.1 Calculation of pixel-level contamination area; Step 2.1.2 Calculate the actual physical area of ​​pollution and the pollution level based on the pixel pollution area.

6. The industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to claim 5, characterized in that, Step 2.2 Training and optimization of the dust recognition model; Step 2.2.1 Training and Multi-Scene Robust Optimization of the Dust Recognition Model; Step 2.2.2 Spectral feature space regularization and model optimization; Step 2.2.3 Model dynamic self-learning update.

7. The industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to claim 6, characterized in that, Step 2.

3. Based on binary contamination mask Determine the defect shape type; Step 2.3.1 Applying a binary contamination mask Perform morphological opening operations (denoising and filling small holes) to obtain a clean binary contaminated region mask. To avoid noise interfering with frequency domain analysis; Step 2.3.2 Pre-treated binary contamination mask Perform 2D-FFT processing to obtain the frequency domain amplitude spectrum. : Step 2.3.3 Extraction of spectral domain shape features.

8. The industrial camera system integrating self-calibration, self-testing, and self-cleaning functions according to any one of claims 3-6, characterized in that, The specific structure of the dust recognition model includes an image input layer, a spatial domain contamination segmentation branch, a frequency domain shape representation branch, a feature regularization fusion layer, a decision output layer, and a model optimization and self-learning layer. Image input layer: The input consists of N consecutive frames of grayscale images captured by the camera. , serving as the global feature data source for the model; The spatial contamination segmentation branch consists of a multi-frame cumulative difference unit, a binary mask generation unit, and a morphological filtering unit. It completes the localization and pixel-level segmentation of contaminated areas and outputs a clean binary contamination mask. This corresponds to the model segmentation and inference task; Frequency domain shape characterization branch: Composed of 2D-FFT transform unit, low frequency contour reconstruction unit, Fourier coefficient extraction unit, and geometric feature calculation unit, it completes the standardization and shape quantization of pollution contours and outputs roundness, eccentricity, aspect ratio, compactness and spectral distribution characteristics. Feature regularization fusion layer: Apply feature similarity constraints and feature interval constraints to the spectral / grayscale features of the polluted area, construct regularization loss, suppress noise interference and enhance the boundary distinction of different polluted areas; Decision output layer: Outputs pollution level based on pollution physical area, outputs defect shape type based on spectrum energy distribution, and finally outputs four types of decision results: pollution area, area, pollution level, and defect shape type. Model optimization and self-learning layer: It consists of multi-scene generalization training unit, composite loss calculation unit and online self-supervised update unit. It optimizes the model by minimizing the joint loss and composite loss of multiple scenes, and realizes long-term online iterative update of model parameters through weakly labeled data.

9. The industrial camera system integrating self-calibration, self-detection, and self-cleaning functions according to claim 3, characterized in that, The specific operation of the pneumatic cleaning system is as follows: Step 3.1 Determine if it is lightly contaminated. If it is, proceed to step 3.

2. If it is not, proceed to step 3.

4. Step 3.2 Determine whether it is a defect shape type in the low frequency domain. If the determination is yes, execute cleaning strategy 1 and then execute step 3.

9. If the determination is no, execute step 3.

3. Step 3.3 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 2 and then execute step 3.

9. If it is not, execute cleaning strategy 3 and then execute step 3.

9. Step 3.4 Determine if it is moderately contaminated. If it is, proceed to step 3.

5. If it is not, proceed to step 3.

7. Step 3.5 Determine whether it is a defect shape type in the low frequency domain. If it is, execute cleaning strategy 4 and then execute step 3.

9. If it is not, execute step 3.

6. Step 3.6 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 5 and then execute step 3.

9. If it is not, execute cleaning strategy 6 and then execute step 3.

9. Step 3.7 Determine whether it is a defect shape type in the low frequency domain. If it is, execute cleaning strategy 7 and then execute step 3.

9. If it is not, execute step 3.

8. Step 3.8 Determine whether it is a defect shape type in the mid-frequency domain. If it is, execute cleaning strategy 8 and then execute step 3.

9. If it is not, execute cleaning strategy 9 and then execute step 3.

9. Step 3.9 The image feature-based dust recognition module continues to identify the actual physical area of ​​contamination and determines whether the actual physical area of ​​contamination is less than the cleanliness threshold. If the determination is yes, cleaning is stopped; if the determination is no, the contamination level and defect shape type are calculated, and then step 3.1 is executed.

Images