Machine vision microdefect detection method and system based on spectrum-polarization-phase three-domain collaborative perception

Abstract

The invention provides a machine vision microdefect detection method and system based on spectrum-polarization-phase three-domain collaborative perception, and the method comprises the steps: constructing a third-order tensor through synchronously obtaining the spectrum, polarization and phase information of the surface of a to-be-detected object, and extracting features through a lightweight multi-stream network; and dynamic fusion of three-domain information is realized by adopting a cross attention mechanism, and finally defect category, position and size information is output. According to the method, the problem of insufficient single-mode information is solved, the surface defect characteristics of the object can be more comprehensively reflected by combining three-domain information, and the capability of identifying the defects of complex objects such as high-reflective surfaces and transparent materials is improved; a lightweight multi-stream network and a cross attention mechanism are adopted, the multi-domain information fusion efficiency is improved, the network parameter quantity is reduced, deployment at a production line edge calculation unit is facilitated, and the detection speed and cost requirements are met; the method can better adapt to surface form changes of complex objects such as curved surface parts and flexible materials, the defect omission ratio is reduced, and the product yield is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision detection, and in particular to a machine vision micro-defect detection method and system based on spectrum-polarization-phase three-domain collaborative perception. BACKGROUND

[0002] With the continuous improvement of product precision requirements in 3C electronics, automotive electronics and other industries, online detection of sub-micron micro-defects has become a key link to ensure product reliability. In actual industrial application scenarios, the existing technology has the following limitations:

[0003] Single modal information bottleneck:

[0004] 1.1 Gray scale / RGB image is easily affected by high light reflection (such as the surface of metal connectors), transparent material light penetration (such as cover glass), resulting in insufficient defect contrast, and it is difficult to distinguish real defects from surface texture;

[0005] 1.2 Traditional spectral detection relies on push-broom imaging, which has low frame rate and cannot match the real-time requirements of high-speed production lines (such as mobile phone motherboard assembly lines);

[0006] 1.3 Polarization or phase detection is mostly applied separately, and cannot combine with material spectral characteristics, so the recognition ability for "material difference type defects" (such as foreign matter and substrate) is limited.

[0007] Low efficiency of multi-domain fusion:

[0008] Existing multi-modal fusion methods mostly use simple feature splicing or post-fusion, without establishing an internal correlation model of spectrum-polarization-phase, and the network parameter quantity is large, which is difficult to deploy in edge computing units of production lines, and cannot meet the dual requirements of detection speed and cost of precision manufacturing enterprises.

[0009] Insufficient scene adaptability:

[0010] For complex objects such as curved parts (such as smart watch shells) and flexible materials (such as flexible screen substrates), traditional methods are difficult to adapt to changes in surface morphology, resulting in high defect omission rate and affecting product yield. SUMMARY

[0011] In view of the shortcomings of the prior art, the present application provides a machine vision micro-defect detection method and system based on spectrum-polarization-phase three-domain collaborative perception, which can realize high-speed and high-precision defect detection of precision electronic components, optical devices and other products.

[0012] The technical scheme of the present application is: a machine vision micro-defect detection method based on spectrum-polarization-phase three-domain collaborative perception, comprising the following steps:

[0013] S1), synchronously acquiring three-domain information of spectral information, polarization information and phase information of the surface of the object to be measured;

[0014] S2), pixel-level alignment of the three-domain information and construction of a three-order tensor;

[0015] S3), extraction of spectral anomaly features, polarization anomaly features and phase anomaly features by a lightweight multi-stream shared weight network;

[0016] S4), calculation of cross-attention mechanism fusion weights using the three anomaly features, fusion of the three anomaly features using the cross-attention mechanism fusion weights to obtain fusion features, and generation of a defect confidence map according to the fusion features;

[0017] S5), segmentation of the defect confidence map based on a local adaptive threshold to distinguish defects from backgrounds, output of defect categories, sub-pixel level positions and size information through a sub-pixel edge detection algorithm, and determination of defect geometric features through Zernike moment analysis.

[0018] S5), segmentation of the defect confidence map based on a local adaptive threshold to distinguish defects from backgrounds, output of defect categories, sub-pixel level positions and size information through a sub-pixel edge detection algorithm, and determination of defect geometric features through Zernike moment analysis.

[0019] Preferably, in step S1), the spectral information is synchronously acquired by combining a MEMS tunable filter with an industrial camera under a preset wavelength sequence to acquire a multi-spectral image.

[0020] Preferably, in step S1), the wavelength of the spectral information covers the characteristic absorption or reflection band of the material of the object to be measured, the wavelength range is 400-1000 nm, the channel number is 3-20, and the sampling interval is adaptively adjusted according to the spectral characteristics of the material.

[0021] Preferably, in step S1), the polarization information is acquired by a pixel-level polarization camera, a multi-direction polarization image is output by single exposure, and a Stokes vector is calculated to reflect the orientation of the surface microstructure, i.e.,

[0022] p(x,y)=[I(x,y),Q(x,y),U(x,y),V(x,y)] T ;

[0023] In the formula, p(x,y) is a polarization information vector; I(x,y) represents an intensity component; Q(x,y)=I 90 0-I 45 0-I 135 0 is a linear polarization component; V(x,y) is a circular polarization component; I 45 0, I 90 0, I 135respectively represent the light intensity values measured by 0°, 45°, 90°, 135° direction polarizer; x, y respectively represent the spatial coordinates in the image plane, wherein x is the horizontal direction coordinate and y is the vertical direction coordinate; T is the transpose operation.

[0024] As preferred, in step S1), the phase information is obtained by projecting phase shift fringes through a common-path interference structured light projector, collecting phase shift images and unwrapping to obtain a phase map, and obtaining a phase information vector φ(x, y).

[0025] As preferred, in step S1), a FPGA trigger module is used to realize the time sequence synchronization of the tunable filter switching, camera exposure and phase shifter, and the trigger signal period is dynamically adjusted according to the production line speed.

[0026] As preferred, in step S2), the third-order tensor X(x, y) is constructed as:

[0027] X(x, y) = [s(x, y), p(x, y), φ(x, y)];

[0028] In the formula, s(x, y) = [s1(x, y), s2(x, y), …, sn(x, y)] is an n-dimensional spectral information vector. n (x, y)] is an n-dimensional spectral information vector.

[0029] As preferred, in step S3), the lightweight multi-stream shared weight network includes a spectral stream, a polarization stream and a phase stream.

[0030] The spectral stream enhances the material features through 1x1 convolution and channel attention mechanism.

[0031] The polarization stream captures polarization orientation abnormalities through local convolution and Transformer module.

[0032] The phase stream extracts surface height mutation features through gradient operator and shared convolution.

[0033] As preferred, in step S4), the calculation of the cross-attention mechanism fusion weight is:

[0034] Taking the spectral abnormal feature as a query vector and the polarization and phase abnormal features as key value vectors, the cross-attention mechanism is used to dynamically allocate and fuse the weights, i.e.:

[0035]

[0036] In the formula, α p , α φ are the fusion weights of polarization and phase respectively; are the learnable weight matrices of spectrum, polarization and phase respectively; d k is the key vector dimension; fs , f p , f φ respectively are spectral anomaly feature, polarization anomaly feature, phase anomaly feature.

[0037] As preferred, in step S4), the three anomaly features are fused by using cross attention mechanism fusion weight to obtain fusion feature F as:

[0038] F=α p ⊙f s +α φ ⊙f p +f φ ;

[0039] And the fusion feature F is processed by 1*1 convolution and sigmoid activation function to obtain defect confidence map M(x,y)∈[0,1].

[0040] As preferred, the present application also provides a machine vision micro-defect detection system based on spectral-polarization-phase three-domain collaborative perception, comprising:

[0041] A three-domain information acquisition module synchronously acquires three-domain information of spectral information, polarization information and phase information of the surface of the object to be measured.

[0042] A three-order tensor construction module is used for pixel-level alignment of the three-domain information and construction of a three-order tensor.

[0043] An anomaly feature extraction module extracts spectral anomaly feature, polarization anomaly feature and phase anomaly feature through a lightweight multi-stream shared weight network.

[0044] A defect confidence map generation module calculates cross attention mechanism fusion weight by using the three anomaly features, and fuses the three anomaly features by using the cross attention mechanism fusion weight to obtain fusion feature, and generates a defect confidence map according to the fusion feature.

[0045] A segmentation module segments the defect confidence map based on local adaptive threshold to distinguish defects from background, and outputs defect category, sub-pixel level position and size information through a sub-pixel edge detection algorithm, and determines defect geometric features through Zernike moment analysis.

[0046] As preferred, the three-domain information acquisition module comprises a spectral information acquisition submodule, a polarization information acquisition submodule, a phase information acquisition submodule and a synchronous control submodule.

[0047] The present application has the following advantages:

[0048] 1. This invention overcomes the problem of insufficient information in a single modality. By combining information from three domains, it can more comprehensively reflect the surface defect characteristics of an object and improve the ability to identify defects in complex objects such as highly reflective surfaces and transparent materials.

[0049] 2. This invention employs a lightweight multi-stream network and a cross-attention mechanism, which improves the efficiency of multi-domain information fusion, reduces the number of network parameters, facilitates deployment in edge computing units on the production line, and meets the requirements of detection speed and cost.

[0050] 3. This invention enhances scene adaptability and can better adapt to the surface morphology changes of complex objects such as curved parts and flexible materials, thereby reducing the defect missed detection rate and improving product yield. Attached Figure Description

[0051] Figure 1 This is a schematic flowchart of the method of the present invention;

[0052] Figure 2 This is a schematic diagram of the process for synchronous acquisition and pixel-level alignment of three-domain information according to the present invention;

[0053] Figure 3 This is a schematic diagram of the lightweight multi-flow shared weight network structure of the present invention;

[0054] Figure 4 This is a flowchart illustrating the cross-attention dynamic fusion mechanism of the present invention. Detailed Implementation

[0055] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0056] Example 1

[0057] like Figure 1 As shown, this embodiment provides a machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative sensing, including the following steps:

[0058] S1) Simultaneously acquire the three-domain information of the spectral information, polarization information and phase information of the surface of the object under test;

[0059] In this embodiment, the spectral information is obtained by using a combination of MEMS tunable filter and industrial camera to synchronously acquire multispectral images under a preset wavelength sequence, and hardware triggering ensures that the single frame acquisition cycle matches the production line cycle.

[0060] The wavelength of the spectral information covers the characteristic absorption or reflection bands of the material being tested, with a wavelength range of 400-1000 nm, 3-20 channels, and the sampling interval is adaptively adjusted according to the spectral characteristics of the material.

[0061] The polarization information is acquired using a pixel-level polarization camera, outputting multi-directional polarization images in a single exposure, and calculating the Stokes vector to reflect the orientation of the surface microstructure, i.e.:

[0062] p(x,y)=[I(x,y),Q(x,y),U(x,y),V(x,y)] T ;

[0063] In the formula, p(x,y) is the polarization information vector; I(x,y) represents the intensity component; Q(x,y)=I0-I 90 For linearly polarized components; U(x,y)=I 45 -I 135 V(x,y) represents the linear polarization component; V(x,y) represents the circular polarization component; I0, I 45 I 90 I 135 represents the light intensity values ​​measured by polarizers at 0°, 45°, 90°, and 135°, respectively; x and y represent the spatial coordinates in the image plane, where x is the horizontal coordinate and y is the vertical coordinate; T represents the transpose operation.

[0064] The phase information is obtained by projecting phase-shifted fringes through a common-path interference structured light projector to acquire phase-shifted images and then unwrapping them to obtain a phase map, thus obtaining the phase information vector φ(x,y).

[0065] In this embodiment, as Figure 2 As shown, an FPGA trigger module is used to realize the timing synchronization of tunable filter switching, camera exposure and phase shifter. The trigger signal period is dynamically adjusted according to the production line speed, ranging from 5 to 50 ms.

[0066] S2) Align the three-domain information at the pixel level and construct a third-order tensor;

[0067] The pixel-level alignment uses a checkerboard calibration plate and sub-pixel corner matching, with the alignment error controlled within 0.3 pixels. In this embodiment, a 1mm spacing checkerboard calibration plate is used to calculate the rotation and translation matrix R between each sensor during the calibration process, and sub-pixel level correction is achieved through B-spline interpolation.

[0068] Construct the third-order tensor X(x,y) as follows:

[0069] X(x,y)=[s(x,y),p(x,y),φ(x,y)];

[0070] In the formula, s(x,y)=[s1(x,y),s2(x,y),…,s n [x,y] is an n-dimensional spectral information vector.

[0071] S3), extracting spectral anomaly features, polarization anomaly features and phase anomaly features through a lightweight multi-flow shared weight network;

[0072] As shown in Figure 3 , the lightweight multi-flow shared weight network comprises a spectral flow, a polarization flow and a phase flow.

[0073] The spectral flow enhances material features through 1x1 convolution and channel attention mechanism, and calculates channel attention weight W s =σ(W2δ(W1s)), where W2, W1 are learnable parameters; δ is a ReLU activation function; σ is a sigmoid function; s is a spectral information vector.

[0074] The polarization flow captures polarization orientation anomalies through local convolution and a Transformer module, and calculates polarization ellipticity and polarization direction angle as features, i.e.:

[0075] ε(x,y)=arctan(|V| / I);

[0076]

[0077] In the formula, ε(x,y), θ(x,y) are polarization ellipticity and polarization direction angle respectively; I represents the intensity component; Q, U are linear polarization components; V is a circular polarization component.

[0078] The phase flow extracts surface height mutation features through a gradient operator and shared convolution, and the phase gradient is:

[0079]

[0080] S4), calculating cross-attention mechanism fusion weights using the three anomaly features, and fusing the three anomaly features using the cross-attention mechanism fusion weights to obtain fusion features, and generating a defect confidence map according to the fusion features;

[0081] As shown in Figure 4 , in the embodiment, the calculation of the cross-attention mechanism fusion weight is:

[0082] Taking the spectral anomaly feature as a query vector and the polarization and phase anomaly features as key-value vectors, the fusion weight is dynamically allocated through a cross-attention mechanism, i.e.

[0083]

[0084] In the formula, α p , α φ are the fusion weights of polarization and phase respectively; are learnable weight matrices of spectrum, polarization and phase respectively; dk is the key vector dimension; f s , f p , f φ are spectrum anomaly features, polarization anomaly features, and phase anomaly features, respectively.

[0085] The three anomaly features are fused by using a cross-attention mechanism fusion weight to obtain a fusion feature F as follows:

[0086] F = a p ⊙f s + a φ ⊙f p + f φ ;

[0087] wherein, ⊙ represents element-wise multiplication, and the adaptability to different types of defects is enhanced.

[0088] The fusion feature F is processed by 1x1 convolution and a sigmoid activation function to obtain a defect confidence map M(x, y) e [0, 1].

[0089] S5), based on a local adaptive threshold T(x, y), the defect confidence map is segmented to distinguish defects from the background, and a sub-pixel edge detection algorithm is used to output defect categories, sub-pixel level positions, and size information, and a Zernike moment analysis is used to determine defect geometric features.

[0090] wherein, the adaptive threshold T(x, y) = μ(x, y) + kσ(x, y), wherein, μ(x, y) and σ(x, y) are the mean and standard deviation of the local region, respectively; and k is an empirical coefficient.

[0091] Embodiment 2

[0092] The embodiment provides a machine vision micro-defect detection system based on spectrum-polarization-phase three-domain collaborative perception, comprising:

[0093] A three-domain information acquisition module is configured to synchronously acquire spectrum information, polarization information, and phase information of a surface of an object to be measured, and comprises a spectrum information acquisition submodule, a polarization information acquisition submodule, a phase information acquisition submodule, and a synchronous control submodule.

[0094] The spectrum information acquisition submodule is combined with an industrial camera by using a MEMS tunable filter to synchronously acquire multispectral images under a preset wavelength sequence, and a hardware trigger is used to ensure that a single-frame acquisition period matches a production line beat. The wavelength of the spectrum information covers a characteristic absorption or reflection waveband of a material of the object to be measured, the wavelength range is 400-1000 nm, the channel number is 3-20, and a sampling interval is adaptively adjusted according to the spectral characteristics of the material.

[0095] The polarization information acquisition submodule adopts a pixel-level polarization camera to acquire polarization information, outputs a multi-direction polarization image in a single exposure, and calculates a Stokes vector to reflect the orientation of a surface microstructure, that is:

[0096] p(x,y)=[I(x,y),Q(x,y),U(x,y),V(x,y)] T ;

[0097] In the formula, p(x,y) is a polarization information vector; I(x,y) represents an intensity component; Q(x,y)=I0-I 90 is a linear polarization component; U(x,y)=I 45 -I 135 is a linear polarization component; V(x,y) is a circular polarization component; I0, I 45 , I 90 , I 135 respectively represent light intensity values measured through 0°, 45°, 90°, and 135° polarizing plates; x and y respectively represent spatial coordinates in an image plane, wherein x is a horizontal direction coordinate and y is a vertical direction coordinate; and T is a transposition operation.

[0098] The phase information acquisition submodule acquires a phase shift image by projecting a phase shift fringe through a common-path interference structure light projector and obtains a phase image by unwrapping to obtain a phase information vector φ(x,y).

[0099] The synchronization control submodule adopts an FPGA trigger module to realize time sequence synchronization of tunable filter switching, camera exposure, and a phase shifter, and a trigger signal period is dynamically adjusted according to a production line speed, and the range is 5-50 ms.

[0100] The third-order tensor construction module is used for pixel-level alignment of the three-domain information and construction of a third-order tensor;

[0101] The pixel-level alignment adopts a chessboard calibration plate and sub-pixel corner matching, and an alignment error is controlled within 0.3 pixels; in this embodiment, a 1mm pitch chessboard calibration plate is adopted, a rotation translation matrix R between sensors is calculated in a calibration process, and sub-pixel correction is realized through B-spline interpolation.

[0102] The third-order tensor X(x,y) is constructed as follows:

[0103] X(x,y)=[s(x,y),p(x,y),φ(x,y)];

[0104] In the formula, s(x,y)=[s1(x,y),s2(x,y),…,s n (x,y)] is an n-dimensional spectral information vector.

[0105] An abnormal feature extraction module extracts spectral abnormal features, polarization abnormal features and phase abnormal features through a lightweight multi-stream shared weight network.

[0106] The lightweight multi-stream shared weight network comprises a spectral stream, a polarization stream and a phase stream.

[0107] The spectral stream enhances material features through 1x1 convolution and channel attention mechanism, and calculates channel attention weight W s =σ(W2δ(W1s)), wherein W2 and W1 are learnable parameters, δ is a ReLU activation function, σ is a sigmoid function, and s is a spectral information vector.

[0108] The polarization stream captures polarization orientation abnormalities through local convolution and a Transformer module, and calculates polarization ellipticity and polarization direction angle as features, i.e.

[0109] ε(x,y)=arctan(|V| / I);

[0110]

[0111] In the formula, ε(x,y) and θ(x,y) are polarization ellipticity and polarization direction angle respectively, I represents an intensity component, Q and U are linear polarization components, and V is a circular polarization component.

[0112] The phase stream extracts surface height mutation features through a gradient operator and shared convolution, and the phase gradient is:

[0113]

[0114] A defect confidence map generation module calculates cross-attention mechanism fusion weights using the three abnormal features, fuses the three abnormal features using the cross-attention mechanism fusion weights to obtain fused features, and generates a defect confidence map according to the fused features.

[0115] The calculation of the cross-attention mechanism fusion weight is:

[0116] The spectral abnormal feature is used as a query vector, and the polarization and phase abnormal features are used as key-value vectors, and the cross-attention mechanism is used to dynamically assign fusion weights, i.e.

[0117]

[0118]

[0119] In the formula, α p and α φ are the fusion weights of the polarization and the phase respectively. These are the learnable weight matrices for the spectrum, polarization, and phase, respectively; d k f is the dimension of the key vector; s f p f φ These are spectral anomalies, polarization anomalies, and phase anomalies, respectively.

[0120] The three anomaly features are fused using a cross-attention mechanism to obtain the fused feature F:

[0121] F = α p ⊙f s +α φ ⊙f p +f φ ;

[0122] Here, ⊙ represents element-wise multiplication, which enhances adaptability to different types of defects;

[0123] The fused feature F is then processed by 1×1 convolution and sigmoid activation function to obtain the defect confidence map M(x,y)∈[0,1].

[0124] The segmentation module segments the defect confidence map based on a local adaptive threshold to distinguish between defects and the background. It outputs defect category, sub-pixel level location and size information through a sub-pixel edge detection algorithm and determines the geometric features of the defects through Zernike moment analysis.

[0125] Example 3

[0126] This embodiment of the method for detecting plating defects in mobile phone metal connectors includes the following steps:

[0127] S1) Simultaneously acquire the three-domain information of the spectral information, polarization information and phase information of the surface of the object under test;

[0128] Spectral information acquisition:

[0129] The 400-850nm wavelength band (covering the characteristic wavelengths of common coating materials such as copper, gold, and nickel) is selected, and a 12-channel MEMS filter (wavelength spacing of 40nm) is used to acquire data synchronously with a 200fps industrial camera. The switching timing of the filter is precisely controlled by the FPGA trigger signal to ensure that the single frame acquisition period is 5ms, which matches the pace of the mobile phone production line.

[0130] Polarization information acquisition:

[0131] Using a Sony IMX250MYR-F monochrome polarization camera, polarization images in four directions (0°, 45°, 90°, and 135°) are output in a single exposure. Stokes vectors are calculated, and the microstructure features of the coating surface are enhanced through polarization difference analysis; that is:

[0132] p(x,y)=[I(x,y),Q(x,y),U(x,y),V(x,y)] T ;

[0133] In the formula, p(x,y) is the polarization information vector; I(x,y) represents the intensity component; Q(x,y)=I0-I 90 For linearly polarized components; U(x,y)=I 45 -I 135 V(x,y) represents the linear polarization component; V(x,y) represents the circular polarization component; I0, I 45 I 90 I 135 represents the light intensity values ​​measured by polarizers at 0°, 45°, 90°, and 135°, respectively; x and y represent the spatial coordinates in the image plane, where x is the horizontal coordinate and y is the vertical coordinate; T represents the transpose operation.

[0134] Phase information acquisition: A 635nm laser projects phase-shifting fringes through a DMD micromirror array, and a four-step phase-shifting method is used to obtain a phase map to quantify the height difference of the coating surface;

[0135] Synchronization: The timing of each module is controlled by FPGA trigger signals with a period of 5ms to ensure the spatiotemporal consistency of information in the three domains; the rising edge of the trigger signal simultaneously starts the filter switching, camera exposure and phase shifter, and the falling edge completes the data acquisition.

[0136] S2) Align the three-domain information at the pixel level and construct a third-order tensor;

[0137] The pixel-level alignment is performed using a 0.8mm pitch checkerboard calibration board for system calibration. The rotation and translation matrix R between each sensor is calculated using a sub-pixel corner detection algorithm, and the alignment error is controlled within 0.25 pixels.

[0138] Construct a 17-dimensional third-order tensor, where each pixel is represented as X(x,y):

[0139] X(x,y)=[s(x,y),p(x,y),φ(x,y)];

[0140] In the formula, s(x,y)=[s1(x,y),s2(x,y),…,s n [x,y] is an n-dimensional spectral information vector.

[0141] The tensor is normalized to make the data in each domain similar in magnitude.

[0142] S3) Extract spectral anomaly features, polarization anomaly features, and phase anomaly features through a lightweight multi-stream shared weight network; in this embodiment, the lightweight multi-stream shared weight network adopts a depthwise separable convolutional structure with only 1.2M parameters, which meets the requirements of edge computing; the lightweight multi-stream shared weight network includes spectral flow, polarization flow, and phase flow;

[0143] The spectral flow enhances material features through 1×1 convolution and channel attention mechanisms, with a focus on enhancing features in the 520nm and 680nm bands; this is achieved by calculating the channel attention weights W. s =σ(W2δ(W1s)), where W2 and W1 are learnable parameters; δ is the ReLU activation function; σ is the sigmoid function; and s is the spectral information vector.

[0144] The described polarization flow captures polarization orientation anomalies through local convolution and the Transformer module; it supplements features by calculating polarization ellipticity and polarization direction angle; and it highlights the changes in polarization characteristics caused by microcracks on the coating surface; that is:

[0145] ε(x,y)=arctan(|V| / I);

[0146]

[0147] In the formula, ε(x,y) and θ(x,y) are the polarization ellipticity and polarization direction angle, respectively; I represents the intensity component; Q and U are the linear polarization components; and V is the circular polarization component.

[0148] The phase flow described above extracts surface height abrupt changes through gradient operators and shared convolution; it captures surface height abrupt changes in the coating and is sensitive to depression-type defects.

[0149] S4) Calculate the cross-attention mechanism fusion weight using the three abnormal features, and fuse the three abnormal features using the cross-attention mechanism fusion weight to obtain the fused features, and generate a defect confidence map based on the fused features.

[0150] The calculation of the fusion weights of the cross-attention mechanism is as follows:

[0151] Using spectral anomaly features as the query vector and polarization and phase anomaly features as the key vectors, the fusion weights are dynamically allocated through a cross-attention mechanism, i.e.:

[0152]

[0153] In the formula, α p α φ These are the fusion weights for polarization and phase, respectively; These are the learnable weight matrices for the spectrum, polarization, and phase, respectively; d kf is the dimension of the key vector; s f p f φ These are spectral anomalies, polarization anomalies, and phase anomalies, respectively.

[0154] The three anomaly features are fused using a cross-attention mechanism to obtain the fused feature F:

[0155] F = α p ⊙f s +α φ ⊙f p +f φ ;

[0156] Here, ⊙ represents element-wise multiplication, which enhances adaptability to different types of defects;

[0157] The fused feature F is then processed by 1×1 convolution and sigmoid activation function to obtain the defect confidence map M(x,y)∈[0,1].

[0158] S5) The defect confidence map is segmented based on the local adaptive threshold T(x,y) to distinguish between defects and background. The defect category, sub-pixel level position and size information are output through the sub-pixel edge detection algorithm, and the geometric features of the defect are determined through Zernike moment analysis.

[0159] Wherein, the adaptive threshold T(x,y) = μ(x,y) + kσ(x,y), where μ(x,y) and σ(x,y) are the mean and standard deviation of the local region, respectively; and k is an empirical coefficient.

[0160] A sub-pixel edge detection algorithm is used to determine the precise location of the defect and calculate geometric features such as the defect area A, perimeter P, and aspect ratio r.

[0161] Defect classification is achieved based on Zernike moment analysis, with the first 8 Zernike moments forming the feature vector:

[0162] z = [z1, z2, dots, z8] T ;

[0163]

[0164] In the formula, Z n (x,y) is a Zernike polynomial.

[0165] Defect information is uploaded to the production line MES system via the Profinet protocol, triggering the quality control process.

[0166] Example 4

[0167] Micro-defect detection in flexible OLED panels includes the following steps:

[0168] S1) Simultaneously acquire the three-domain information of the spectral information, polarization information and phase information of the surface of the object under test;

[0169] Spectral information acquisition:

[0170] To address the characteristics of flexible OLED panels, a 450-700nm wavelength band (covering the characteristic wavelengths of ITO transparent electrodes and organic light-emitting materials) was selected, along with a 16-channel MEMS filter, and the sampling interval was adaptively adjusted (intensified sampling in curved areas).

[0171] Polarization information acquisition:

[0172] A high-sensitivity polarization camera is employed to enhance the ability to capture polarization information in weak signal regions, with particular attention paid to the sensitivity of the circular polarization component V(x,y) to the stress of flexible materials; that is:

[0173] p(x,y)=[I(x,y),Q(x,y),U(x,y),V(x,y)] T ;

[0174] In the formula, p(x,y) is the polarization information vector; I(x,y) represents the intensity component; Q(x,y)=I0-I 90 For linearly polarized components; U(x,y)=I 45 -I 135 V(x,y) represents the linear polarization component; V(x,y) represents the circular polarization component; I0, I 45 I 90 I 135 represents the light intensity values ​​measured by polarizers at 0°, 45°, 90°, and 135°, respectively; x and y represent the spatial coordinates in the image plane, where x is the horizontal coordinate and y is the vertical coordinate; T represents the transpose operation.

[0175] Phase information acquisition: A 635nm laser projects phase-shifting fringes through a DMD micromirror array, and a three-step phase-shifting method combined with a surface correction algorithm is used to process the bending deformation of the flexible panel;

[0176] Synchronization: The timing of each module is controlled by FPGA trigger signals with a period of 8ms to ensure the spatiotemporal consistency of information in the three domains; the rising edge of the trigger signal simultaneously starts the filter switching, camera exposure and phase shifter, and the falling edge completes the data acquisition.

[0177] S2) Align the three-domain information at the pixel level and construct a third-order tensor;

[0178] Based on the phase information, a 3D surface model of the panel is reconstructed, and the local curvature κ(x,y) is calculated. The tensor is then adjusted according to the curvature to construct weights, with phase information weights added to areas of greater curvature. A third-order tensor is constructed, with each pixel represented as X(x,y).

[0179] X(x,y)=[βs(x,y),(1-β)p(x,y),φ(x,y)];

[0180]

[0181] In the formula, s(x,y)=[s1(x,y),s2(x,y),…,s n [x,y] is an n-dimensional spectral information vector, and λ is the curvature sensitivity coefficient.

[0182] S3) Extract spectral anomaly features, polarization anomaly features, and phase anomaly features through a lightweight multi-stream shared weight network; in this embodiment, the lightweight multi-stream shared weight network adopts a depthwise separable convolutional structure with only 1.2M parameters, which meets the requirements of edge computing; the lightweight multi-stream shared weight network includes spectral flow, polarization flow, and phase flow;

[0183] The spectral flow enhances material characteristics through 1×1 convolution and channel attention mechanism. For the curved surface characteristics of flexible panels, the polarization flow adds a curvature sensing module to calculate the angle between the polarization direction and the surface normal.

[0184] The described polarization flow captures polarization orientation anomalies through local convolution and the Transformer module; it supplements these features by calculating polarization ellipticity and polarization direction angle; and it highlights the changes in polarization characteristics caused by microcracks on the flexible panel surface.

[0185] ε(x,y)=arctan(|V| / I);

[0186]

[0187] In the formula, ε(x,y) and θ(x,y) are the polarization ellipticity and polarization direction angle, respectively; I represents the intensity component; Q and U are the linear polarization components; and V is the circular polarization component.

[0188] The phase flow described above extracts the height abrupt change features of the flexible panel surface through gradient operators and shared convolution; it captures the height abrupt change of the flexible panel surface and is sensitive to concave defects;

[0189] S4) Calculate the fusion weights of the cross-attention mechanism using three abnormal features, and introduce curvature information as a priori to dynamically adjust the fusion weights; then use the fusion weights of the cross-attention mechanism to fuse the three abnormal features to obtain fused features, and generate a defect confidence map based on the fused features.

[0190] The calculation of the fusion weights of the cross-attention mechanism is as follows:

[0191] Using spectral anomaly features as the query vector and polarization and phase anomaly features as the key vectors, the fusion weights are dynamically allocated through a cross-attention mechanism, i.e.:

[0192]

[0193] In the formula, α p α φ These are the fusion weights for polarization and phase, respectively; These are the learnable weight matrices for the spectrum, polarization, and phase, respectively; d k f is the dimension of the key vector; s f p f φ These are spectral anomalies, polarization anomalies, and phase anomalies, respectively.

[0194] The three anomaly features are fused using a cross-attention mechanism to obtain the fused feature F:

[0195] F = α p ⊙f s +α φ ⊙f p +f φ ;

[0196] Here, ⊙ represents element-wise multiplication, which enhances adaptability to different types of defects;

[0197] The fused feature F is then processed by 1×1 convolution and sigmoid activation function to obtain the defect confidence map M(x,y)∈[0,1].

[0198] S5) The defect confidence map is segmented based on the local adaptive threshold T(x,y) to distinguish between defects and background. The defect category, sub-pixel level position and size information are output through the sub-pixel edge detection algorithm, and the geometric features of the defect are determined through Zernike moment analysis.

[0199] Wherein, the adaptive threshold T(x,y) = μ(σ,y) + kσ(x,y), where μ(x,y) and σ(x,y) are the mean and standard deviation of the local region, respectively; and k is an empirical coefficient.

[0200] A multi-scale analysis strategy is adopted to process defects of different sizes separately. Small defects (<50μm) are analyzed based on spectral and polarization characteristics, while large defects are analyzed based on phase characteristics. The output results include the precise location and morphological description of the defects on the 3D surface, which facilitates quality traceability and process improvement.

[0201] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative sensing, characterized in that, Includes the following steps: S1) Simultaneously acquire the three-domain information of the spectral information, polarization information and phase information of the surface of the object under test; S2) Align the three-domain information at the pixel level and construct a third-order tensor; S3) Extract spectral anomaly features, polarization anomaly features, and phase anomaly features using a lightweight multi-stream shared weight network; S4) Calculate the cross-attention mechanism fusion weight using the three abnormal features, and fuse the three abnormal features using the cross-attention mechanism fusion weight to obtain the fused features, and generate a defect confidence map based on the fused features. S5) The defect confidence map is segmented based on local adaptive threshold to distinguish between defects and background. The defect category, sub-pixel level location and size information are output through sub-pixel edge detection algorithm, and the geometric features of the defect are determined through Zernike moment analysis.

2. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative sensing according to claim 1, characterized in that: In step S1), the acquisition of spectral information is achieved by using a combination of a MEMS tunable filter and an industrial camera to simultaneously acquire multispectral images under a preset wavelength sequence. The wavelengths of the spectral information collected cover the characteristic absorption or reflection bands of the material under test, with a wavelength range of 400-1000nm, 3-20 channels, and the sampling interval is adaptively adjusted according to the spectral characteristics of the material.

3. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative sensing according to claim 2, characterized in that: In step S1), the polarization information is acquired using a pixel-level polarization camera, outputting a multi-directional polarization image in a single exposure, and calculating the Stokes vector to reflect the orientation of the surface microstructure, i.e.: p(x,y)=[I(x,y),Q(x,y),U(x,y),V(x,y)] T ; In the formula, p(x,y) is the polarization information vector; I(x,y) represents the intensity component; Q(x,y)=I0-I 90 For linearly polarized components; U(x,y)=I 45 -I 135 V(x,y) represents the linear polarization component; V(x,y) represents the circular polarization component; I0, I 45 I 90 I 135 represents the light intensity values ​​measured by polarizers at 0°, 45°, 90°, and 135°, respectively; x and y represent the spatial coordinates in the image plane, where x is the horizontal coordinate and y is the vertical coordinate; T represents the transpose operation.

4. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative perception according to claim 3, characterized in that: In step S1), the phase information is obtained by projecting phase-shifted fringes through a common-path interference structured light projector to acquire a phase-shifted image and unwrapping it to obtain a phase map, thus obtaining the phase information vector φ(x,y).

5. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative perception according to claim 4, characterized in that: In step S2), the pixel-level alignment uses a checkerboard calibration board to match sub-pixel corner points, and the alignment error is controlled within 0.3 pixels.

6. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative sensing according to claim 5, characterized in that: In step S2), the third-order tensor X(x,y) is constructed as follows: X(x,y)=[s(x,y),p(x,y),φ(x,y)]; In the formula, s(x,y)=[s1(x,y),s2(x,y),…,s n [x,y] is an n-dimensional spectral information vector.

7. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative perception according to claim 6, characterized in that: In step S3), the lightweight multi-flow shared weight network includes spectral flow, polarization flow, and phase flow; The spectral flow enhances material features through 1×1 convolution and channel attention mechanisms; The polarization flow is described in this way: it captures polarization orientation anomalies through local convolution and the Transformer module. The phase flow extracts surface height abrupt change features through gradient operators and shared convolution.

8. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative perception according to claim 7, characterized in that: In step S4), the calculation of the fusion weights of the cross-attention mechanism is as follows: Using spectral anomaly features as the query vector and polarization and phase anomaly features as the key vectors, the fusion weights are dynamically allocated through a cross-attention mechanism, i.e.: In the formula, α p α φ These are the fusion weights for polarization and phase, respectively; These are the learnable weight matrices for the spectrum, polarization, and phase, respectively; d k f is the dimension of the key vector; s f p f φ These are spectral anomalies, polarization anomalies, and phase anomalies, respectively.

9. The machine vision micro-defect detection method based on spectral-polarization-phase three-domain collaborative sensing according to claim 8, characterized in that: In step S4), the three abnormal features are fused using a cross-attention mechanism to obtain the fused feature F: F=a p ⊙f s +a φ ⊙f p +f φ ; The fused feature F is then processed by 1×1 convolution and sigmoid activation function to obtain the defect confidence map M(x,y)∈[0,1], where ⊙ represents element-wise multiplication.

10. A machine vision micro-defect detection system based on spectral-polarization-phase three-domain collaborative perception, characterized in that, The system described herein utilizes the method described in any one of claims 1-9 to achieve micro-defect detection, and the system comprises: The three-domain information acquisition module is used to simultaneously acquire the spectral information, polarization information, and phase information of the surface of the object under test; it includes a spectral information acquisition submodule, a polarization information acquisition submodule, a phase information acquisition submodule, and a synchronization control submodule. The third-order tensor construction module is used to perform pixel-level alignment of the three-domain information and construct a third-order tensor. The anomaly feature extraction module extracts spectral anomalies, polarization anomalies, and phase anomalies using a lightweight multi-stream shared weight network. The defect confidence map generation module calculates the fusion weights of the cross-attention mechanism using three abnormal features, and then uses the fusion weights of the cross-attention mechanism to fuse the three abnormal features to obtain fused features, and generates a defect confidence map based on the fused features. The segmentation module segments the defect confidence map based on a local adaptive threshold to distinguish between defects and the background. It outputs defect category, sub-pixel level location and size information through a sub-pixel edge detection algorithm and determines the geometric features of the defects through Zernike moment analysis.

Images