Lens defect detection method and system based on photoelectric multi-fusion perception technology

Abstract

The invention discloses a lens defect detection method and system based on a photoelectric multi-fusion perception technology, and relates to the field of product defect detection.The lens defect detection system comprises a capturing module used for collecting visible light, infrared light and laser scattering signals on the surface and in a lens and converting the collected signals into processable electric signals; the registration module is used for receiving the electric signals output by the capture module, performing denoising, normalization and spatial registration on the electric signals, and establishing signal mapping according to a spatial corresponding relation of the same detection area of the lens so as to complete preliminary association of multi-modal signals; according to the invention, through cooperative acquisition and precise processing of multi-source signals, surface and internal defect information of the lens is comprehensively captured, and based on comparison of a dynamic adaptive determination standard and efficient characteristics, defect identification accuracy and category discrimination are greatly improved, a three-dimensional position of the defect is precisely locked, reliable detection data is output and rapidly fed back, and detection precision is improved. And the lens quality and the production efficiency are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of product defect detection technology, specifically to a lens defect detection method and system based on photoelectric multi-fusion sensing technology. Background Technology

[0002] Product defect detection typically involves acquiring product images and extracting features to quickly identify defects such as surface flaws and dimensional deviations. It boasts advantages such as non-contact operation, high precision, and high speed, and is widely used in quality control in manufacturing industries such as electronics, automotive, and 3C products, enabling real-time defect screening and efficiency improvement. In the manufacturing of optical filters and other lenses, defects such as scratches, bubbles, uneven coatings, and internal microcracks on the lens surface directly affect optical indicators such as light transmittance and cutoff band performance, placing higher demands on the precision and dimensionality of defect detection.

[0003] The invention patent application with application number 202111488051.X discloses a product defect detection method and device. This application aims to solve the problem that "in the training process of deep learning application for mobile phone surface defect detection, it is necessary to consider both the imbalance of different defect sample categories and the generalization ability of the model; different mobile phone projects generally require the construction of new detection models and retraining, resulting in weak generalization ability of the model. In addition, different mobile phones have different surface textures and colors, and how to build a model with strong generalization ability faces great challenges."

[0004] However, relying solely on image recognition technologies cannot meet the diverse technical needs of lens defect detection scenarios. Especially for lenses such as optical filters, which are transparent and have multiple layers of optical thin films on their surfaces, single visible light imaging methods are insufficient to effectively detect latent defects such as internal bubbles, film defects, and microcracks. Therefore, the collaborative application of multimodal sensing technologies is urgently needed. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a lens defect detection method and system based on photoelectric multi-fusion sensing technology, which can effectively solve the problems of the existing technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions;

[0007] This invention discloses a lens defect detection system based on photoelectric multi-fusion sensing technology, comprising:

[0008] The system comprises the following modules: a capture module for acquiring visible light, infrared, and laser scattering signals from the lens surface and interior, converting these signals into processable electrical signals; a registration module for receiving the electrical signals output from the capture module, performing noise reduction, normalization, and spatial registration, and establishing signal mapping based on the spatial correspondence of the same detection area of ​​the lens to complete the initial association of multimodal signals; an extraction module for mining the grayscale, texture, and geometric features corresponding to lens defects based on the association information of multimodal signals, constructing a feature vector set containing defect differentiation information; an identification module for performing hierarchical analysis of the feature vector set based on dynamic thresholds and feature matching, identifying lens defects and marking the preliminary category attributes of the defects; an integration module for acquiring the spatial coordinate information of the multimodal signals, identifying the location of defects on the lens, and integrating defect category, location, and signal strength information to establish a defect detection data package; and a feedback module for receiving the defect detection data package output from the integration module and feeding it back to a preset production management terminal.

[0009] The capture module is interactively connected to the registration module via a wireless network. The configuration module is interactively connected to the extraction module and the recognition module via a wireless network. The recognition module is interactively connected to the integration module via a wireless network. The integration module is interactively connected to the feedback module via a wireless network.

[0010] Furthermore, the capture module integrates a visible light sensor array, an infrared focal plane detector, and a laser scattering detection unit;

[0011] The sampling frequency of the visible light sensor array is synchronized with the frame rate of the infrared focal plane detector, and the emission wavelength of the laser scattering detection unit is complementary to the characteristic absorption wavelength of the lens material.

[0012] The capture module performs synchronous calibration on the acquired multi-source raw signals using the following formula:

[0013] ;

[0014] In the formula: The synthesized electrical signal after calibration at time t; , , The original electrical signals corresponding to visible light, infrared light, and laser scattering at time t; , , These are the weighting coefficients for visible light signals, infrared signals, and laser scattering signals.

[0015] Furthermore, the electrical signal denoising in the registration module follows the following rules:

[0016] ;

[0017] In the formula: The signal at coordinates (x, y) is the denoised electrical signal. Let (x, y) be the original electrical signal at coordinates (x, y). This is half the size of the filtering window; is the adaptive weighting factor; m and p are the horizontal and vertical offsets relative to the center coordinates (x, y) within the filtering window, respectively; This represents the mean of the original electrical signal within the filtering window;

[0018] The normalization processing of the electrical signal uses the min-max normalization method to map the amplitude of the denoised electrical signal to a preset range. The spatial registration is based on the three-dimensional coordinates of the preset reference marker points of the lens, and the spatial alignment of the multimodal signals is performed through rigid transformation. The signal mapping is completed by establishing a gray-scale response correlation model of the multimodal signals under the same spatial coordinates.

[0019] Furthermore, when the extraction module mines geometric features, it includes the three-dimensional contour parameters, curvature distribution, and connected component morphology parameters of the defect;

[0020] The three-dimensional contour parameters are calculated by inverse mapping of the gray-level gradient of the multimodal signal;

[0021] The construction process of the feature vector set is as follows: after normalizing the grayscale, texture and geometric features respectively, the defect identification contribution of each feature is calculated and sorted based on the information gain criterion. The top X key features are selected to form the basic feature vector. Then, the basic feature vector is mapped to the high-dimensional feature space through kernel function mapping to form the final feature vector set containing defect differentiation information.

[0022] Furthermore, the dynamic threshold of the recognition module is updated in real time using the following formula:

[0023] ;

[0024] In the formula; The dynamic threshold for the k-th detection; The dynamic threshold for the (k-1)th detection; These are the threshold adjustment coefficients and threshold reduction coefficients; Let be the signal complexity factor of the k-th detection region; is the dispersion factor of the feature vector set detected in the kth instance; is the background uniformity factor of the k-th detection region; is the clustering factor of the feature vector set detected in the kth instance;

[0025] In the feature matching stage, feature matching is performed by calculating the fusion matching degree between the feature vector to be detected and the standard defect feature vector:

[0026] ;

[0027] In the formula: For feature vectors and The degree of fusion matching; Cosine similarity weight; for and Cosine similarity; for and Mahalanobis distance; This is the Mahalanobis distance adjustment parameter.

[0028] Furthermore, the hierarchical analysis operation in the identification module includes:

[0029] Initial screening layer: This layer compares the feature values ​​of each dimension in the feature vector with a dynamic threshold. Comparison, removal of all dimensional feature values ​​below The flawless feature vector retains at least one dimension with a feature value higher than 1. Candidate defect feature vectors;

[0030] Fine matching layer: This layer fuses and matches candidate defect feature vectors with feature vectors of each category in a pre-defined standard defect feature vector library, calculating the fusion matching degree. Feature vectors with a fusion matching degree higher than a preset matching threshold and their corresponding standard defect categories are selected.

[0031] Category Confirmation Layer: For candidate categories selected by the fine matching layer, calculate the average fusion matching degree between the feature vector to be detected and all standard feature vectors under that category. If the average value is higher than the category confirmation threshold, then the category is marked as a preliminary category attribute of the defect. If there are multiple candidate categories that meet the conditions, then select the category with the highest average fusion matching degree as the preliminary category attribute. If the difference between the highest average value and the second highest average value is lower than the preset difference threshold, then it is marked as a composite category attribute.

[0032] The process of identifying lens defects is as follows: candidate defect feature vectors are obtained through a preliminary screening layer, feature comparison is completed through a fine matching layer, and the defect identification result and corresponding preliminary category attribute are output by combining the judgment result of the category confirmation layer.

[0033] Furthermore, the integration module determines the precise three-dimensional location of the defect through a spatial coordinate calibration model based on the multimodal signal. This spatial coordinate calibration model is established based on the coordinate transformation relationship between the lens's CAD three-dimensional model and the actual inspection scene.

[0034] ;

[0035] In the formula: [X, Y, Z] are the precise three-dimensional coordinates of the defect; These are the parameters of the rotation matrix for coordinate transformation; The translation vector parameters for coordinate transformation; The initial three-dimensional coordinates of the defect detected by the visible light signal; The initial three-dimensional coordinates of the defect detected by the infrared signal; The initial three-dimensional coordinates of the defect detected by the laser scattering signal; For visible light coordinate weights, For infrared coordinate weights, The coordinate weights are for laser scattering.

[0036] Furthermore, the defect detection data packet also includes a signal confidence parameter for the defect.

[0037] On the other hand, a lens defect detection method based on photoelectric multi-fusion sensing technology includes:

[0038] The system collects raw visible light, infrared, and laser scattering signals from the lens surface and interior, converts them into processable electrical signals, and assigns signal weights based on lens material characteristics, surface reflectivity, and thermal conductivity to achieve synchronous calibration of multi-source signals. Adaptive denoising and min-max normalization are applied to the calibrated electrical signals. Based on the three-dimensional coordinates of the lens reference markers, rigid transformation is used to achieve spatial alignment of multimodal signals, establishing signal mapping associations within the same detection area. Grayscale, texture, and three-dimensional geometric features corresponding to defects are mined, and after normalization, key features are selected according to the information gain criterion. A high-dimensional feature vector set containing defect-specific information is constructed through kernel function mapping. Dynamic thresholds are updated in real-time based on signal complexity factors, feature vector set dispersion factors, background uniformity factors, and feature vector set aggregation factors. Through hierarchical analysis of initial screening, fine matching, and category confirmation, combined with cosine similarity and Mahalanobis distance, the fusion matching degree is calculated to identify defects and label preliminary category attributes. Based on the coordinate transformation relationship between the lens CAD three-dimensional model and multimodal signals, the precise three-dimensional location of the defect is determined. Defect category, location, signal strength, and reliability parameters are integrated to generate a defect detection data package. The defect detection data package is then fed back to a pre-set production management terminal.

[0039] Compared with the known prior art, the technical solution provided by this invention has the following beneficial effects:

[0040] This invention provides a lens defect detection method and system based on optoelectronic multi-fusion sensing technology. During execution, this method and system effectively captures information related to surface and internal defects of the lens through multi-source signal collaborative acquisition and precise calibration. Noise reduction, normalization, and spatial registration processing improve signal reliability, thereby mining multi-dimensional defect features and optimizing the feature vector set to enhance the ability to identify defect differences. Dynamic thresholds can adapt to the detection scenario in real time. Combined with a fusion matching algorithm, the accuracy of defect identification and category determination is improved, and the three-dimensional location of defects is accurately located. Finally, multi-dimensional detection information is integrated and data reliability is ensured, providing comprehensive and accurate defect data support for production management, reducing the production of defective lenses, and improving production quality and efficiency. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0042] Figure 1 This is a schematic diagram of a lens defect detection system based on optoelectronic multi-fusion sensing technology.

[0043] Figure 2 This is a flowchart illustrating a lens defect detection method based on optoelectronic multi-fusion sensing technology. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0045] The present invention will be further described below with reference to embodiments.

[0046] Example 1:

[0047] This embodiment presents a lens defect detection system based on photoelectric multi-fusion sensing technology, such as... Figure 1 As shown, it includes:

[0048] The capture module is used to collect visible light, infrared and laser scattering signals from the surface and interior of the lens, and convert the collected signals into processable electrical signals.

[0049] The capture module integrates a visible light sensor array, an infrared focal plane detector, and a laser scattering detection unit;

[0050] The sampling frequency of the visible light sensor array is synchronized with the frame rate of the infrared focal plane detector, and the emission wavelength of the laser scattering detection unit is complementary to the characteristic absorption wavelength of the lens material.

[0051] The capture module performs synchronous calibration on the acquired multi-source raw signals using the following formula:

[0052] ;

[0053] In the formula: The synthesized electrical signal after calibration at time t; , , The original electrical signals corresponding to visible light, infrared light, and laser scattering at time t; , , These are the weighting coefficients for visible light signals, infrared signals, and laser scattering signals;

[0054] The above formula combines the characteristics of three original signals: visible light, infrared light, and laser scattering. Based on the reflectivity of the lens surface, the internal thermal conductivity, and the refractive index of the material, the weight coefficients of the corresponding signals are adjusted respectively. Through reasonable weighted fusion, the synchronous calibration of multi-source signals is achieved, so that the synthesized electrical signal can accurately adapt to the material and surface properties of different lenses and more realistically reflect the actual state of the lens.

[0055] in, ∈ (0, 1], the higher the reflectivity of the lens surface, the smaller the value; the lower the reflectivity of the lens surface, the larger the value. ∈ (0, 1], the higher the internal thermal conductivity of the lens, the larger the value; the lower the internal thermal conductivity of the lens, the smaller the value. ∈ (0, 1], the higher the refractive index of the lens material, the larger the value; the lower the refractive index of the lens material, the smaller the value. Taking optical filters as an example, the refractive index of their substrate materials (such as quartz glass, BK7 optical glass) is usually greater than 1.5, with many coating interfaces and abundant laser scattering signals, therefore... Take a larger value (e.g., 0.6~0.8); the lens surface generally has an anti-reflective or cut-off coating system, and the actual reflectivity is affected by the design of the coating system. It is advisable to dynamically determine the specific coating spectral characteristics of the lens; the lens has a low thermal conductivity. While a relatively small value is preferable, when the coating layer has absorptive defects (pinholes, scratches), localized thermal anomalies will occur in this area under laser irradiation. The infrared focal plane detector can still respond effectively, and in this case, the value can be locally increased for the suspicious area. Dynamic weights.

[0056] The registration module is used to receive the electrical signal output by the capture module, perform noise reduction, normalization and spatial registration on the electrical signal, and establish signal mapping based on the spatial correspondence of the same detection area of ​​the lens to complete the initial association of multimodal signals.

[0057] The electrical signal denoising in the registration module follows the following rules:

[0058] ;

[0059] In the formula: The signal at coordinates (x, y) is the denoised electrical signal. Let (x, y) be the original electrical signal at coordinates (x, y). This is half the size of the filtering window; is the adaptive weighting factor; m and p are the horizontal and vertical offsets relative to the center coordinates (x, y) within the filtering window, respectively; This represents the mean of the original electrical signal within the filtering window;

[0060] The above formula addresses the noise interference problem of the original electrical signal at the coordinates by introducing the half-size of the filtering window, the adaptive weighting factor, and the horizontal and vertical offsets within the window. It is calculated in combination with the mean of the original electrical signal within the window. While effectively filtering noise, it retains the key signal features related to lens defects to the greatest extent, adapts to the noise distribution differences of signals at different locations, and avoids losing effective information during the denoising process.

[0061] The normalization of the electrical signal uses the min-max normalization method to map the amplitude of the denoised electrical signal to a preset range. Spatial registration is based on the three-dimensional coordinates of the preset reference marker points of the lens. The spatial alignment of the multimodal signals is performed through rigid transformation. The signal mapping is completed by establishing a gray-scale response correlation model of the multimodal signals under the same spatial coordinates.

[0062] The rigid change process is as follows:

[0063] ;

[0064] In the formula: The aligned target space coordinates; It is a three-dimensional rotation matrix; These are the original spatial coordinates of any mode in the multimodal signal; The three-dimensional coordinates of the preset reference marker points; It is a three-dimensional translation vector;

[0065] Based on the three-dimensional coordinates of the preset reference marks on the lens, the above formula constructs a three-dimensional rotation matrix and a three-dimensional translation vector that satisfy the orthogonality constraint. Through rigid transformation, the original spatial coordinates of each modal signal are converted into target spatial coordinates, ensuring that the multimodal signals in the same detection area are accurately aligned in space, laying the foundation for establishing signal correlation in the future.

[0066] , Represents the identity matrix. , The elements in the matrix are rotation matrix elements, whose values ​​are obtained by solving the coordinate deviation of the lens preset reference mark points under different modes, and satisfy the orthogonality constraint of the three-dimensional rotation matrix.

[0067] The extraction module is used to mine the grayscale, texture and geometric features corresponding to lens defects based on the correlation information of multimodal signals, and construct a feature vector set containing defect differentiation information;

[0068] When the extraction module mines geometric features, it includes the three-dimensional contour parameters, curvature distribution, and connected component morphology parameters of the defects.

[0069] The three-dimensional contour parameters are calculated by inverse mapping of the gray-level gradient of the multimodal signal;

[0070] For lens-type lenses, grayscale features primarily reflect local brightness anomalies caused by surface scratches and edge chipping; texture features can effectively characterize periodic texture differences in areas of uneven coating; and the curvature distribution in geometric features has high sensitivity to local deformations caused by bubbles and microcracks inside the lens. The synergy of these three types of features can achieve full-dimensional coverage of common defects in optical filters. Therefore, the construction process of the feature vector set is as follows: after normalizing the grayscale, texture, and geometric features respectively, the defect identification contribution of each feature is calculated based on the information gain criterion and ranked. The top X key features are selected to form the basic feature vector, where X is a preset feature dimension threshold and X is a positive integer. Then, the basic feature vector is mapped to a high-dimensional feature space through a kernel function mapping to form the final feature vector set containing defect differentiation information.

[0071] The contribution calculation formula and the kernel function mapping formula are as follows:

[0072] ;

[0073] In the formula: The information gain of feature f is the contribution to defect identification. The information entropy of the defect category set C; Let f be the set of all possible values ​​for feature f; The probability that feature f takes the value v; The conditional information entropy of the defect category set C when feature f takes the value v; It is the high-dimensional feature vector mapped from the basic feature vector x; The number of support vectors; These are the support vector weight coefficients; The Gaussian kernel function; , which are unit basis vectors in high-dimensional space;

[0074] The above formula selects the top X key features that contribute highly to defect identification by calculating the information gain of the features, forming a basic feature vector to eliminate redundant information and reduce computational complexity. Then, the Gaussian kernel function is used to map the basic feature vector to a high-dimensional feature space. The support vector weight coefficients are obtained through supervised learning of the defect sample set and are related to the contribution of the support vector to the defect category boundary. The kernel function bandwidth parameter is adapted to the feature vector distribution density, thereby fully mining the differential information of defects to improve the feature's ability to distinguish defects.

[0075] in, >0, obtained through supervised learning training of the defect sample set, is positively correlated with the contribution of the i-th support vector to the defect category boundary, that is, the higher the effectiveness of boundary partitioning, the larger the value, and the higher the redundancy of boundary partitioning, the smaller the value.

[0076] , This represents the kernel function bandwidth parameter, which is negatively correlated with the distribution density of the eigenvectors. This represents the i-th support vector;

[0077] The identification module is used to perform hierarchical analysis of the feature vector set based on dynamic threshold and feature matching, identify lens defects, and mark the preliminary category attributes of the defects;

[0078] The dynamic threshold of the recognition module is updated in real time using the following formula:

[0079] ;

[0080] In the formula; The dynamic threshold for the k-th detection; Let be the dynamic threshold for the (k-1)th detection, where k is a positive integer and k≥1. When k=1, The initial preset threshold for the system; These are the threshold adjustment coefficients and threshold reduction coefficients; Let be the signal complexity factor of the k-th detection region; is the dispersion factor of the feature vector set detected in the kth instance; is the background uniformity factor of the k-th detection region; is the clustering factor of the feature vector set detected in the kth instance;

[0081] The above formula refers to the threshold of the previous detection, introduces an upward adjustment coefficient related to the defect detection rate of the previous detection and a downward adjustment coefficient related to the false detection rate. At the same time, it combines the signal complexity factor reflecting the signal variance of the detection area, the dispersion factor related to the standard deviation of the background signal, the background uniformity factor of the mean Euclidean distance of the associated feature vectors, and the clustering factor related to the mean distance from the feature vectors to the cluster centers. The detection threshold is updated in real time, so that the threshold can flexibly adapt to the signal changes and feature distribution states under different detection scenarios, and reduce the missed detections or false detections caused by fixed thresholds.

[0082] In the feature matching stage, feature matching is performed by calculating the fusion matching degree between the feature vector to be detected and the standard defect feature vector:

[0083] ;

[0084] In the formula: For feature vectors (Feature vector to be detected) and The degree of fusion matching of (standard defect feature vectors); Cosine similarity weight; for and Cosine similarity; for and Mahalanobis distance; This is the Mahalanobis distance adjustment parameter;

[0085] The above formula combines the cosine similarity and Mahalanobis distance between the feature vector to be detected and the feature vector of the standard defect to measure the degree of matching between the two. The cosine similarity weight is adjusted according to the consistency of the dimensions of the two, and the Mahalanobis distance adjustment parameter is related to the trace of the covariance matrix of the feature vector of the standard defect. It fully considers the consistency of the vector direction and takes into account the differences in data distribution, making the feature matching results more comprehensive and reliable, and accurately reflecting the fit between the feature to be detected and the feature of the standard defect.

[0086] in, >0, positively correlated with the defect detection rate of the first k-1 tests; >0, positively correlated with the false positive rate of the first k-1 detections; ∈ (0, 1], and is positively correlated with the signal variance in that region; ∈ (0, 1], and is negatively correlated with the standard deviation of the background signal in this region; ∈ (0, 1], and is positively correlated with the mean Euclidean distance between feature vectors; ∈ (0, 1], and is negatively correlated with the mean distance from the feature vector to the cluster center; ∈ (0, 1), the value is larger when the dimension consistency between the feature vector to be detected and the standard defect feature vector is higher, and the value is smaller when the dimension consistency is lower; >0, the value is larger when the trace of the covariance matrix of the standard defect feature vector is larger, and the value is smaller when the trace of the covariance matrix is ​​smaller;

[0087] The hierarchical analysis operations in the identification module include:

[0088] Initial screening layer: This layer compares the feature values ​​of each dimension in the feature vector with a dynamic threshold. Comparison, removal of all dimensional feature values ​​below The flawless feature vector retains at least one dimension with a feature value higher than 1. Candidate defect feature vectors;

[0089] Fine matching layer: This layer fuses and matches candidate defect feature vectors with feature vectors of each category in a pre-defined standard defect feature vector library, calculating the fusion matching degree. Feature vectors with a fusion matching degree higher than a preset matching threshold and their corresponding standard defect categories are selected.

[0090] Category Confirmation Layer: For candidate categories selected by the fine matching layer, calculate the average fusion matching degree between the feature vector to be detected and all standard feature vectors under that category. If the average value is higher than the category confirmation threshold, then the category is marked as a preliminary category attribute of the defect. If there are multiple candidate categories that meet the conditions, then select the category with the highest average fusion matching degree as the preliminary category attribute. If the difference between the highest average value and the second highest average value is lower than the preset difference threshold, then it is marked as a composite category attribute.

[0091] The process of identifying lens defects is as follows: candidate defect feature vectors are obtained through a preliminary screening layer, feature comparison is completed through a fine matching layer, and the defect identification result and corresponding preliminary category attribute are output in combination with the judgment result of the category confirmation layer. For optical filters, the preset standard defect feature vector library should include five types of standard defect feature vectors: scratches, edge chips, bubbles, microcracks, and uneven coating. Among them, the feature vectors of scratches and uneven coating have significant differences in the texture dimension, and the feature vectors of bubbles and microcracks have significant differences in the geometric curvature dimension. This can effectively support the accurate judgment of the fine matching layer and the category confirmation layer, and output composite category attributes when the two types of defects coexist.

[0092] The integration module is used to acquire the spatial coordinate information of multimodal signals, identify the location of defects on the lens, and integrate defect type, location and signal intensity information to establish a defect detection data package;

[0093] The integration module determines the precise three-dimensional location of the defect through a spatial coordinate calibration model based on multimodal signals. This spatial coordinate calibration model is established based on the coordinate transformation relationship between the lens's CAD three-dimensional model and the actual inspection scene.

[0094] ;

[0095] In the formula: [X, Y, Z] are the precise three-dimensional coordinates of the defect; The rotation matrix parameters for coordinate transformation are determined based on the spatial pose differences between the CAD model and the detection scene; The translation vector parameters for coordinate transformation are determined based on the positional deviation between the origin of the CAD model and the reference point of the detection scene; The initial three-dimensional coordinates of the defect detected by the visible light signal; The initial three-dimensional coordinates of the defect detected by the infrared signal; The initial three-dimensional coordinates of the defect detected by the laser scattering signal; For visible light coordinate weights, For infrared coordinate weights, The weights are the coordinates for laser scattering.

[0096] The above formula is based on the coordinate transformation relationship between the lens CAD 3D model and the actual inspection scene. It determines the rotation matrix parameters and translation vector parameters of the coordinate transformation, which correspond to the spatial attitude difference between the two and the position deviation between the origin and the reference point, respectively. At the same time, it introduces the weight coefficients that are positively correlated with the spatial positioning accuracy of each modal signal, and fuses the initial 3D coordinates of the defects detected by the three signals to accurately calculate the 3D precise position of the defects, ensuring the accuracy and consistency of the defect position information.

[0097] in, It is positively correlated with the spatial positioning accuracy of visible light signals. It is positively correlated with the spatial positioning accuracy of infrared signals. It is positively correlated with the spatial positioning accuracy of the laser scattering signal, and , , The sum is 1;

[0098] The defect detection data package also includes a signal confidence parameter for the defect. The confidence parameter is calculated by the consistency of the response of multimodal signals to the same defect. Specifically, it is the ratio of the mean confidence level of each modal signal to the standard deviation of the confidence level when the defect is detected to the standard deviation of the confidence level. When the standard deviation of the confidence level is 0, the signal confidence parameter takes the preset maximum value.

[0099] The feedback module is used to receive the defect detection data packets output from the integration module and feed them back to the preset production management terminal.

[0100] The capture module is interconnected with the registration module via a wireless network. The configuration module is interconnected with the extraction module and the recognition module via a wireless network. The recognition module is interconnected with the integration module via a wireless network. The integration module is interconnected with the feedback module via a wireless network.

[0101] In this embodiment, the capture module collects visible light, infrared, and laser scattering signals from the surface and interior of the lens, converting the collected signals into processable electrical signals. The registration module then receives the electrical signals output by the capture module, performs noise reduction, normalization, and spatial registration on the signals, and establishes a signal mapping based on the spatial correspondence of the same detection area of ​​the lens to complete the initial association of multimodal signals. The extraction module further mines the grayscale, texture, and geometric features corresponding to lens defects based on the association information of the multimodal signals, constructing a feature vector set containing defect differentiation information. The recognition module then performs hierarchical analysis of the feature vector set based on dynamic thresholds and feature matching, identifying lens defects and marking their preliminary category attributes. The integration module obtains the spatial coordinate information of the multimodal signals, identifies the location of the defects on the lens, and integrates the defect category, location, and signal strength information to establish a defect detection data packet. Finally, the feedback module receives the defect detection data packet output from the integration module and feeds it back to a preset production management terminal.

[0102] In the above embodiments, the system comprehensively captures surface and internal defect information of the lens through multi-source signal collaborative acquisition and precise processing. Based on dynamic adaptation judgment criteria and efficient feature comparison, it significantly improves the accuracy of defect identification and category differentiation, accurately locks the three-dimensional position of the defect, outputs reliable detection data and provides rapid feedback, effectively improving lens quality and production efficiency.

[0103] It should be noted that the system in the above embodiments is particularly suitable for defect detection of lenses such as optical filters. Common defects in lenses include surface scratches, edge chipping, bubbles, internal microcracks, and uneven coating. The visible light sensor array is suitable for detecting surface scratches and edge chipping, while the infrared focal plane detector, under laser excitation assistance, detects localized thermal anomalies in the coating layer caused by defects (pinholes, absorbing impurities). The laser scattering detection unit is suitable for detecting internal bubbles, surface and near-surface microcracks. For bubbles located in the near-surface (subsurface) region of the lens substrate, the laser scattering detection unit can respond effectively; for deep bubbles, visible light transmission mode is used for assisted identification. The three modes of coordinated sensing can achieve comprehensive coverage of lens defects, significantly improving detection sensitivity and defect location accuracy, and meeting the high-quality control requirements of optical filter manufacturing.

[0104] Example 2:

[0105] At the implementation level, based on Example 1, this example refers to... Figure 2A further detailed description of the lens defect detection system based on photoelectric multi-fusion sensing technology in Example 1 is provided below:

[0106] A lens defect detection method based on optoelectronic multi-fusion sensing technology includes:

[0107] The system collects raw signals of visible light, infrared light, and laser scattering from the surface and interior of the lens and converts them into processable electrical signals. It then assigns signal weights based on the lens material characteristics, surface reflectivity, and thermal conductivity to complete multi-source signal synchronous calibration.

[0108] Adaptive denoising and min-max normalization are performed on the calibrated electrical signal. Based on the three-dimensional coordinates of the lens reference marker points, multimodal signal spatial alignment is achieved through rigid transformation, and signal mapping association in the same detection area is established.

[0109] The grayscale, texture, and 3D geometric features corresponding to defects are mined, and after normalization, key features are selected according to the information gain criterion. A high-dimensional feature vector set containing defect differentiation information is constructed through kernel function mapping.

[0110] The dynamic threshold is updated in real time based on the signal complexity factor, feature vector set dispersion factor, background uniformity factor, and feature vector set aggregation factor. Through hierarchical analysis of initial screening, fine matching, and category confirmation, the fusion matching degree is calculated by combining cosine similarity and Mahalanobis distance to identify defects and mark preliminary category attributes.

[0111] Based on the coordinate transformation relationship between the lens CAD 3D model and multimodal signals, the precise 3D location of the defect is determined, and the defect category, location, signal strength and confidence parameters are integrated to generate a defect detection data package;

[0112] The defect detection data packet is fed back to the preset production management terminal.

[0113] In summary, the system and method described in the above embodiments effectively capture information related to surface and internal defects of lenses through multi-source signal collaborative acquisition and precise calibration. Noise reduction, normalization, and spatial registration improve signal reliability, thereby mining multi-dimensional defect features and optimizing the feature vector set to enhance the ability to identify defect differences. Dynamic thresholds can adapt to the detection scenario in real time. Combined with a fusion matching algorithm, the accuracy of defect identification and category determination is improved, and the three-dimensional location of defects is accurately located. Finally, multi-dimensional detection information is integrated and data credibility is ensured, providing comprehensive and accurate defect data support for production management, reducing the output of defective lenses, and improving production quality and efficiency. This method and system are particularly suitable for precision defect detection scenarios of lenses such as optical filters, effectively identifying various lens-specific defects such as surface scratches, edge chipping, bubbles, microcracks, and uneven coating, providing strong technical support for quality control in optical component manufacturing.

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lens defect detection system based on photoelectric multi-fusion sensing technology, characterized in that, include: The capture module is used to collect visible light, infrared and laser scattering signals from the surface and interior of the lens, and convert the collected signals into processable electrical signals. The registration module is used to receive the electrical signal output by the capture module, perform noise reduction, normalization and spatial registration on the electrical signal, and establish signal mapping based on the spatial correspondence of the same detection area of ​​the lens to complete the initial association of multimodal signals. The extraction module is used to mine the grayscale, texture and geometric features corresponding to lens defects based on the correlation information of multimodal signals, and construct a feature vector set containing defect differentiation information; The identification module is used to perform hierarchical analysis of the feature vector set based on dynamic threshold and feature matching, identify lens defects, and mark the preliminary category attributes of the defects; The integration module is used to acquire the spatial coordinate information of multimodal signals, identify the location of defects on the lens, and integrate defect type, location and signal intensity information to establish a defect detection data package; The feedback module is used to receive the defect detection data packets output from the integration module and feed them back to the preset production management terminal.

2. The lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 1, characterized in that, The capture module is integrated with a visible light sensor array, an infrared focal plane detector, and a laser scattering detection unit. The sampling frequency of the visible light sensor array is synchronized with the frame rate of the infrared focal plane detector, and the emission wavelength of the laser scattering detection unit is complementary to the characteristic absorption wavelength of the lens material. The capture module performs synchronous calibration on the acquired multi-source raw signals using the following formula: ； In the formula: The synthesized electrical signal after calibration at time t; , , The original electrical signals corresponding to visible light, infrared light, and laser scattering at time t; , , These are the weighting coefficients for visible light signals, infrared signals, and laser scattering signals.

3. The lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 1, characterized in that, The electrical signal denoising in the registration module follows the following rules: ； In the formula: The signal at coordinates (x, y) is the denoised electrical signal. Let (x, y) be the original electrical signal at coordinates (x, y). This is half the size of the filtering window; is the adaptive weighting factor; m and p are the horizontal and vertical offsets relative to the center coordinates (x, y) within the filtering window, respectively; This represents the mean of the original electrical signal within the filtering window. The normalization processing of the electrical signal uses the min-max normalization method to map the amplitude of the denoised electrical signal to a preset range. The spatial registration is based on the three-dimensional coordinates of the preset reference marker points of the lens, and the spatial alignment of the multimodal signals is performed through rigid transformation. The signal mapping is completed by establishing a gray-scale response correlation model of the multimodal signals under the same spatial coordinates.

4. The lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 1, characterized in that, When the extraction module mines geometric features, it includes the three-dimensional contour parameters, curvature distribution, and connected component morphology parameters of the defect. The three-dimensional contour parameters are calculated by inverse mapping of the gray-level gradient of the multimodal signal; The construction process of the feature vector set is as follows: after normalizing the grayscale, texture and geometric features respectively, the defect identification contribution of each feature is calculated and sorted based on the information gain criterion. The top X key features are selected to form the basic feature vector. Then, the basic feature vector is mapped to the high-dimensional feature space through kernel function mapping to form the final feature vector set containing defect differentiation information.

5. A lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 1, characterized in that, The dynamic threshold of the recognition module is updated in real time using the following formula: ； In the formula; The dynamic threshold for the k-th detection; The dynamic threshold for the (k-1)th detection; These are the threshold adjustment coefficients and threshold reduction coefficients; Let be the signal complexity factor of the k-th detection region; is the dispersion factor of the feature vector set detected in the kth instance; is the background uniformity factor of the k-th detection region; is the clustering factor of the feature vector set detected in the kth instance; In the feature matching stage, feature matching is performed by calculating the fusion matching degree between the feature vector to be detected and the standard defect feature vector: ； In the formula: For feature vectors and The degree of fusion matching; Cosine similarity weight; for and Cosine similarity; for and Mahalanobis distance; This is the Mahalanobis distance adjustment parameter.

6. A lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 5, characterized in that, The hierarchical analysis operation in the identification module includes: Initial screening layer: This layer compares the feature values ​​of each dimension in the feature vector with a dynamic threshold. Comparison, removal of all dimensional feature values ​​below The flawless feature vector retains at least one dimension with a feature value higher than 1. Candidate defect feature vectors; Fine matching layer: This layer fuses and matches candidate defect feature vectors with feature vectors of each category in a pre-defined standard defect feature vector library, calculating the fusion matching degree. Feature vectors with a fusion matching degree higher than a preset matching threshold and their corresponding standard defect categories are selected. Category Confirmation Layer: For candidate categories selected by the fine matching layer, calculate the average fusion matching degree between the feature vector to be detected and all standard feature vectors under that category. If the average value is higher than the category confirmation threshold, then the category is marked as a preliminary category attribute of the defect. If there are multiple candidate categories that meet the conditions, then select the category with the highest average fusion matching degree as the preliminary category attribute. If the difference between the highest average value and the second highest average value is lower than the preset difference threshold, then it is marked as a composite category attribute. The process of identifying lens defects is as follows: candidate defect feature vectors are obtained through a preliminary screening layer, feature comparison is completed through a fine matching layer, and the defect identification result and corresponding preliminary category attribute are output by combining the judgment result of the category confirmation layer.

7. A lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 1, characterized in that, The integration module determines the precise three-dimensional location of the defect through a spatial coordinate calibration model based on multimodal signals. This spatial coordinate calibration model is established based on the coordinate transformation relationship between the lens's CAD three-dimensional model and the actual inspection scene. ； In the formula: [X, Y, Z] are the precise three-dimensional coordinates of the defect; These are the parameters of the rotation matrix for coordinate transformation; The translation vector parameters for coordinate transformation; The initial three-dimensional coordinates of the defect detected by the visible light signal; The initial three-dimensional coordinates of the defect detected by the infrared signal; The initial three-dimensional coordinates of the defect detected by the laser scattering signal; For visible light coordinate weights, For infrared coordinate weights, The coordinate weights are for laser scattering.

8. A lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 7, characterized in that, The defect detection data package also includes the signal confidence parameter of the defect.

9. A lens defect detection system based on photoelectric multi-fusion sensing technology according to claim 1, characterized in that, The capture module is interactively connected to the registration module via a wireless network. The configuration module is interactively connected to the extraction module and the recognition module via a wireless network. The recognition module is interactively connected to the integration module via a wireless network. The integration module is interactively connected to the feedback module via a wireless network.

10. A method for detecting lens defects based on photoelectric multi-fusion sensing technology, wherein the method is an implementation method of the lens defect detection system based on photoelectric multi-fusion sensing technology as described in any one of claims 1-9, characterized in that, include: The system collects raw signals of visible light, infrared light, and laser scattering from the surface and interior of the lens and converts them into processable electrical signals. It then assigns signal weights based on the lens material characteristics, surface reflectivity, and thermal conductivity to complete multi-source signal synchronous calibration. Adaptive denoising and min-max normalization are performed on the calibrated electrical signal. Based on the three-dimensional coordinates of the lens reference marker points, multimodal signal spatial alignment is achieved through rigid transformation, and signal mapping association in the same detection area is established. The grayscale, texture, and 3D geometric features corresponding to defects are mined, and after normalization, key features are selected according to the information gain criterion. A high-dimensional feature vector set containing defect differentiation information is constructed through kernel function mapping. The dynamic threshold is updated in real time based on the signal complexity factor, feature vector set dispersion factor, background uniformity factor, and feature vector set aggregation factor. Through hierarchical analysis of initial screening, fine matching, and category confirmation, the fusion matching degree is calculated by combining cosine similarity and Mahalanobis distance to identify defects and mark preliminary category attributes. Based on the coordinate transformation relationship between the lens CAD 3D model and multimodal signals, the precise 3D location of the defect is determined, and the defect category, location, signal strength and confidence parameters are integrated to generate a defect detection data package; The defect detection data packet is fed back to the preset production management terminal.

Images