Multi-channel self-adaptive glass component defect detection method

Abstract

The invention relates to the technical field of industrial detection, and discloses a multi-channel self-adaptive glass component defect detection method which sequentially comprises the following steps: S1, multi-channel image acquisition; performing rapid switching and imaging of three illumination modes by adopting a time-sharing stroboscopic technology to obtain a multi-channel image of the glass component; s2, multi-channel image registration is carried out; s2, registering the multi-channel images obtained in the step S1 so as to align the features of the multi-channel images, and obtaining registered multi-channel images; s3, multi-channel feature extraction and fusion; performing single-channel feature extraction on the registered multi-channel image, and combining a channel attention mechanism and a space attention mechanism to obtain a multi-channel image fusion feature map; s4, segmenting a defect region; and S5, a post-processing scoring mechanism is used for preventing over-killing. Through a multi-channel image acquisition technology, multi-scale feature extraction in deep learning, a self-adaptive attention mechanism and a post-processing scoring mechanism are combined to prevent over-killing, and more efficient and more accurate plane glass component defect detection is carried out.

Description

Technical Field

[0001] The invention relates to the technical field of industrial detection, and in particular to a multi-channel adaptive glass component defect detection method. Background Art

[0002] Glass components are widely used in high-precision fields such as optical devices, display panels, and medical devices, and their quality directly affects the performance and reliability of terminal products. Due to the particularity of glass component materials, its defect detection is very difficult. Traditional glass component defect detection methods mainly rely on single-channel image processing technology, such as visible light imaging or infrared imaging. However, single-channel detection methods often have shortcomings when dealing with complex defects of transparent materials. Visible light imaging is easily affected by surface reflection and transmitted light, making it difficult to accurately detect internal defects; infrared imaging may be insensitive to surface defects. In addition, single-channel detection is difficult to provide multi-level information, which limits the reliability and accuracy of the detection results. At present, there are some multi-channel detection systems on the market that attempt to improve detection accuracy by combining multiple imaging technologies. However, these systems usually rely on simple image superposition or merging, lack of deep fusion and effective use of multi-channel information, resulting in unsatisfactory detection results. Summary of the invention

[0003] In order to overcome or alleviate one or more of the above technical problems, the purpose of the present invention is to provide a multi-channel adaptive glass component defect detection method, which provides an efficient and accurate glass component defect detection method by introducing multi-channel image acquisition technology, combining feature extraction and fusion in deep learning, adaptive attention mechanism, and redesigning the post-processing scoring mechanism to prevent over-killing.

[0004] The present invention provides the following technical solutions:

[0005] A multi-channel adaptive glass component defect detection method comprises the following steps in sequence:

[0006] S1: multi-channel image acquisition;

[0007] The time-sharing stroboscopic technology is used to quickly switch and image three lighting modes to obtain a multi-channel image of the glass component; the three lighting modes are: the light source is set on the front and back of the glass component to be inspected for lighting, and the infrared lighting is performed inside the glass component to be inspected;

[0008] S2: multi-channel image registration;

[0009] Registering the multi-channel images obtained in step S1 to align features between the multi-channel images and obtain a registered multi-channel image;

[0010] S3: multi-channel feature extraction and fusion;

[0011] By extracting features of each channel of the registered multi-channel image, combining the channel attention mechanism and the spatial attention mechanism, information of different channels is fused to obtain a fused feature map;

[0012] S4: defect area segmentation;

[0013] The fused feature map is input into the feature pyramid network to generate a multi-scale feature map; anchor frames covering regions of different sizes and aspect ratios are generated on the multi-scale feature map; the region proposal network RPN is used to perform target classification and bounding box regression on each anchor frame, and a defect mask is generated on the aligned feature map to segment the defect from the original image;

[0014] S5: Post-processing scoring mechanism to prevent overkill;

[0015] First, confidence screening is used to preliminarily filter defects. For defect areas with low confidence, a statistical feature anomaly detection mechanism is used to further determine whether the defect belongs to this category. If the feature anomaly detection mechanism passes, the scoring mechanism is used for final determination.

[0016] According to some embodiments, in step S1, for front lighting, a high-brightness and uniform LED light source is selected, and the coaxial LED light source is aligned with the optical axis of the camera and installed to ensure that the light is irradiated onto the glass component vertically along the optical axis of the camera; for back lighting, a uniform backlight LED lamp board is installed below the glass component to be tested, so that the light irradiates the glass component from the back; for infrared lighting, an infrared LED light source with a wavelength in the range of 800-1200nm is used, and the infrared LED light source is installed on the side of the inspection station, and in conjunction with an infrared camera, the infrared light penetrates the glass component material to detect internal defects; using time-sharing stroboscopic technology, according to a preset timing, the strobe controller controls the flashing time of the coaxial light, the back light and the infrared light source in turn, and controls the three light sources to flash quickly in turn, so that each light source illuminates the glass component separately within a set extremely short time, and a multi-channel image of the glass component is obtained.

[0017] According to some embodiments, step S2 specifically comprises the following steps:

[0018] S21: feature detection;

[0019] Use ORB to detect feature points; first, use the FAST detection method to detect the feature points from the three channel images I i Extract potential feature points from the formula as follows:

[0020]

[0021] in, is the jth feature point detected in the i-th channel image; then the BRIEF descriptor is calculated for each feature point, the formula is as follows:

[0022]

[0023] is the feature point the associated binary descriptor;

[0024] S22: feature point matching;

[0025] Use the Hamming distance to match feature point descriptors and select the best matching set of glass component feature point pairs. The formula is as follows:

[0026]

[0027] M ij is the set of feature point pairs matched between the reference channel and the target channel, ‖.‖ H represents the Hamming distance;

[0028] S23: Calculate transformation matrix;

[0029] Use the RANSAC algorithm to calculate the transformation matrix H i , to ensure that images of different channels are aligned, the formula is as follows:

[0030] H i =RANSAC(M ij ) (4)

[0031] S24: Apply transformation;

[0032] Take the forward illumination image I1 as the reference image and perform image alignment. The formula is as follows:

[0033] I2′=H2I2 (5)

[0034] I3′=H3I3 (6)

[0035] Get the registered multi-channel image.

[0036] According to some embodiments, S3 comprises the following steps:

[0037] S31: single channel feature extraction;

[0038] MobileNetV3 is used as the feature extraction network, and the depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution, thereby reducing the amount of convolution calculation to 1 / 2 of the traditional convolution. times, where α is the convolution kernel size; at the same time, MobileNetV3 adds inverted residual and SE modules to enhance the model's expressiveness while extracting features. The formula is as follows:

[0039] F1 k =MobileNetV3 k (I1) (7)

[0040]

[0041] F3 k =MobileNetV3 k (I′3) (9)

[0042] MobileNetV3k represents the K-th layer of convolution operations, Represents the k-th layer feature map of the i-th channel image;

[0043] S32: Channel attention mechanism;

[0044] Through the channel attention mechanism, we can adapt to the feature differences under different lighting conditions, where W1 and W2 are learnable weight matrices, δ is the Relu activation function, and σ is the sigmoid activation function. First, we convert the multi-channel features into Concatenate and generate channel attention weight A through two layers of fully connected networks c ; Then use the channel weight to perform element-by-element multiplication on each channel feature for weighted feature fusion to obtain F fusion , the formula is as follows:

[0045]

[0046] S33: spatial attention mechanism;

[0047] Use the spatial attention mechanism to assign a weight to each spatial position to emphasize important areas and ignore unimportant parts. First, the feature map is concatenated and input into the Conv convolution operation to generate the weight A. s ; Use the generated spatial attention map to multiply the channel fused feature map element by element to obtain the final fused feature map F att , the formula is as follows:

[0048]

[0049] According to some embodiments, step S4 specifically comprises the following steps:

[0050] S41: The fused feature map F att Input into the feature pyramid network to generate multi-scale features to detect defects of different sizes. The formula is as follows:

[0051] P i =C i +UpSample(P i+1 ) (14)

[0052] C i is the fusion feature F att The extracted i-th layer feature, UpSample, i.e. P i+1 It is to upsample the high-level feature map to enhance the low-level feature map;

[0053] S42: Generate anchor boxes on the multi-scale feature map to cover areas of different sizes and aspect ratios; use the region proposal network RPN to perform target classification and bounding box regression on each anchor box; then use ROI pooling to extract and align the regions from the multi-scale feature map based on the candidate regions generated by RPN, as follows:

[0054] R=ROIAlign(P,ROIs) (15)

[0055] Each aligned candidate region is classified and predicted and the bounding box is regressed. The formula is as follows:

[0056] P = Softmax(W c R) (16)

[0057] t=W b R (17)

[0058] P is the class probability distribution, W c is the classification weight matrix, t is the regression parameter, W b is the regression weight matrix;

[0059] S43: Finally, a defect mask is generated on the aligned feature map to segment the defect from the original image. The formula is as follows:

[0060] M=σ(W m R) (18)

[0061] M is the generated binary mask, W m It is the weight matrix generated by the mask, and the defect area and the corresponding category are obtained.

[0062] According to some embodiments, step S5 specifically comprises the following steps:

[0063] S51: According to the suspected defect area, category and confidence obtained in step S4, for each segmented suspected defect area, extract the features for the category, including area, roundness, contrast, length, width, amplitude and frequency; for scratches, extract length, width and contrast; for inclusions, extract area, contrast and roundness; for bubbles, extract area, contrast and roundness; for cracks, extract length, perimeter and contrast; for ripples, extract amplitude and frequency; for burrs, extract length and width;

[0064] S52: For defective areas with low confidence, when the confidence is less than the preset confidence threshold, do not give up immediately, but enter the abnormality detection stage, that is, enter step S53; if the confidence is higher than or equal to the confidence threshold, directly enter the scoring mechanism, that is, enter step S54 for judgment;

[0065] S53: The anomaly detection stage is to perform anomaly detection based on statistical features; the statistical features are obtained by analyzing a large amount of defect data of known categories, including the mean, variance, maximum and minimum values ​​of the features; check whether the feature value of the defect area with low confidence falls within the statistical range of the category; if the feature value meets the statistical range, it is considered that the defect may belong to the category, and enter step S54; otherwise, it is directly marked as overkill;

[0066] S54: Normalize the features of the defective areas with high confidence and those that pass the anomaly detection, and standardize the feature values ​​to the [0,1] interval. The formula is as follows:

[0067]

[0068] f i is the original eigenvalue, f i,min and f i,max are the minimum and maximum values ​​of the characteristic;

[0069] S55: The defect score is calculated using a comprehensive scoring mechanism. The confidence and feature scores jointly determine the final score. The scoring formula is as follows:

[0070] S=β·Conf+(1-β)·(ω1·f1′+ω2·f2′+…ω n ·f n ′) (20)

[0071] β is the confidence weight, ω i is the weight of each feature;

[0072] S56: The scoring threshold can be set according to the following formula. If the defect score S is greater than the scoring threshold, it is determined to be a real defect; otherwise, it is determined to be an overkill. The scoring formulas for each category of defects are as follows:

[0073] S 划痕=β·Cof+(1-β)-·(0.4·f′ 长度 +0.3·f′ 宽度 +0.3·f′ 对比度 ) (twenty one)

[0074] S 夹杂 =β·Conf+(1-β)·(0.5·f′ 对比度 +0.3·f′ 圆度 +0.2·f′ 面积 ) (twenty two)

[0075] S 气泡 =β·Conf+(1-β)·(0.4·f′ 面积 +0.4·f′ 圆度 +0.2·f′ 对比度 ) (twenty three)

[0076] S 裂纹 =β·Conf+(1-β)·(0.5·f′ 长度 +0.3·f′ 周长 +0.2·f′ 对比度 ) (twenty four)

[0077] S 毛刺 =β·Conf+(1-β)·(0.6·f′ 长度 +0.4·f′ 宽度 ) (25)

[0078] The scoring threshold is dynamically adjusted according to the different tolerance levels of each product to defects.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] The present invention provides a multi-channel adaptive glass component defect detection method, which uses multi-channel image acquisition technology and combines multi-scale feature extraction, adaptive attention mechanism, and post-processing scoring mechanism in deep learning to prevent overkill, so as to perform more efficient and accurate flat glass component defect detection. The method can achieve high-precision detection in the early stages of the glass component manufacturing process, prevent defective products from entering the subsequent processing and assembly links, thereby improving overall production efficiency and product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 A schematic flow chart of a multi-channel adaptive glass component defect detection method provided in an embodiment of the present invention.

[0082] Figure 2 A flow chart of a deep learning network model provided for an embodiment of the present invention.

[0083] Figure 3 A flow chart of a post-processing scoring mechanism provided in an embodiment of the present invention.

[0084] Figure 4 This is a diagram showing the detection implementation effect provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0085] The present invention is described in detail below in conjunction with the embodiments and drawings, but it should be understood that the embodiments and drawings are only used to exemplify the present invention and do not constitute any limitation on the protection scope of the present invention. All reasonable changes and combinations within the scope of the inventive concept of the present invention fall within the protection scope of the present invention.

[0086] The present invention will be further described below in conjunction with the accompanying drawings.

[0087] Example 1

[0088] like Figure 1 The present embodiment provides a multi-channel adaptive glass component defect detection method comprising the following steps:

[0089] S1: Multi-channel image acquisition

[0090] This embodiment uses multi-channel image acquisition technology to achieve comprehensive inspection of glass components. Time-sharing stroboscopic technology is used to achieve rapid switching and imaging of three lighting modes, ensuring that the shooting interval between visible light and infrared light images is extremely short, thereby improving the effects of image registration and feature fusion.

[0091] (1) For front lighting, choose a high-brightness, uniform LED light source. Align the coaxial light source with the optical axis of the camera to ensure that the light is irradiated vertically along the optical axis of the camera onto the glass component. Coaxial light irradiates vertically along the optical axis, which can effectively reduce the reflected light and shadow on the surface of the glass component and improve the image quality.

[0092] (2) For back lighting, a uniform backlight LED light panel is installed under the glass component so that light can illuminate the glass component from the back, highlighting the edges and minor defects of the glass component and improving the detectability of defects.

[0093] (3) For infrared lighting, an infrared LED light source with a wavelength in the range of 800-1200nm is used. The infrared light source is installed on the side of the inspection station and matched with an infrared camera. The infrared light can penetrate the glass component material, detect internal defects, and improve the depth and accuracy of inspection.

[0094] By using the light source stroboscopic technology, according to the preset timing, the stroboscopic controller controls the flashing time of the coaxial light, back light and infrared light source in turn, controls the three light sources to flash quickly in turn, and each light source illuminates the glass component within 100ms to obtain a multi-channel image of the glass component.

[0095] S2: Multi-channel image registration

[0096] After obtaining the multi-channel image in step S1, the images of each channel need to be registered to ensure the alignment between features. Specifically, the following steps are included:

[0097] S21: Feature Detection

[0098] Oriented FAST and Rotated BRIEF (ORB) is used for feature point detection. ORB has rotation invariance and is suitable for various lighting environments of glass components. First, the FAST (Features from Accelerated Segment Test) detection method is used to detect feature points from the three channel images I i Extract potential feature points from the formula as follows:

[0099]

[0100] in, is the jth feature point detected in the i-th channel image. Then the BRIEF descriptor is calculated for each feature point, and the formula is as follows:

[0101]

[0102] is the feature point the associated binary descriptor;

[0103] S22: Feature point matching

[0104] Use the Hamming distance to match feature point descriptors and select the best matching set of glass component feature point pairs. The formula is as follows:

[0105]

[0106] M ij is the set of feature point pairs matched between the reference channel and the target channel,||.|| H represents the Hamming distance.

[0107] S23: Calculate the transformation matrix

[0108] Use the RANSAC algorithm to calculate the transformation matrix H i , to ensure that images of different channels are aligned, the formula is as follows:

[0109] H i =RANSAC(M ij ) (4)

[0110] S24: Apply transformation

[0111] Take the forward illumination image I1 as the reference image and perform image alignment. The formula is as follows:

[0112] I2′=H2I2 (5)

[0113] I3′=H3I3 (6)

[0114] Obtain a registered multi-channel image;

[0115] like Figure 2 , is a schematic diagram of the process of steps S3 and S4, specifically:

[0116] S3: Multi-channel feature extraction and fusion

[0117] S31: Multi-channel feature extraction

[0118] The convolution kernels in traditional convolutional neural networks usually contain a large number of parameters, and these parameters do not contribute evenly to improving model performance. A large number of redundant parameters not only increase the complexity of the model, but also increase storage and computing overhead. In this embodiment, MobileNetV3 is used as the feature extraction network. MobileNetV3 uses depthwise separable convolution to decompose the standard convolution into depthwise convolution and pointwise convolution, thereby reducing the amount of convolution calculation to 1 / 2 of the traditional convolution. times (α is the convolution kernel size); at the same time, MobileNetV3 adds inverted residual and SE modules to enhance the model's expressiveness while extracting features. The formula is as follows:

[0119] F1 k =MobileNetV3 k ( I ) (7)

[0120]

[0121] F3 k =MobileNetV3 k ((I′3) (9)

[0122] MobileNetV3k represents the k-th layer convolution operation, F i k Represents the k-th layer feature map of the i-th channel image;

[0123] S32: Channel Attention Mechanism

[0124] Through the channel attention mechanism, we can adapt to the feature differences under different lighting conditions, where W1 and W2 are learnable weight matrices, δ is the Relu activation function, and σ is the sigmoid activation function. First, we convert the multi-channel features into Concatenate and generate channel attention weight A through two layers of fully connected networks c ; Then use the channel weight to perform element-by-element multiplication on each channel feature for weighted feature fusion to obtain F fusion , the formula is as follows:

[0125]

[0126] S33: Spatial Attention Mechanism

[0127] Then, a spatial attention mechanism is used to assign a weight to each spatial position to emphasize important areas and ignore unimportant parts. First, the feature map is concatenated and input into the Conv convolution operation to generate the weight A s ; Use the generated spatial attention map to multiply the channel fused feature map element by element to obtain the final fused feature map F att , the formula is as follows:

[0128]

[0129] S4: Defective area segmentation

[0130] S41: The fused feature map F att Input into the feature pyramid network to generate multi-scale features to detect defects of different sizes. The formula is as follows:

[0131] P i =C i +UpSample(P i+1 ) (14)

[0132] C i is the fusion feature F att The extracted i-th layer features, UpSample(P i+1 ) is to perform upsampling operation on high-level feature maps to enhance low-level feature maps;

[0133] S42: Generate anchor boxes on the multi-scale feature map to cover areas of different sizes and aspect ratios; use the region proposal network RPN to perform target classification and bounding box regression on each anchor box; then use ROI pooling to extract and align the regions from the multi-scale feature map based on the candidate regions generated by RPN, as follows:

[0134] R=ROIAlign(P,ROIs) (15)

[0135] Each aligned candidate region is classified and predicted and the bounding box is regressed. The formula is as follows:

[0136] P = Softmax(W c R) (16)

[0137] t=W b R (17)

[0138] P is the class probability distribution, W c is the classification weight matrix. t is the regression parameter, W b is the regression weight matrix;

[0139] S43: Finally, a defect mask is generated on the aligned feature map to segment the defect from the original image. The formula is as follows:

[0140] M=σ(W m R) (18)

[0141] M is the generated binary mask, W m is the weight matrix generated by the mask. At this point, the defect area and the corresponding category are obtained.

[0142] S5: Post-processing scoring mechanism to prevent overkill

[0143] Common types of glass component defects include scratches, inclusions, bubbles, cracks, burrs, etc. In S4, the specific area of ​​the defect and the corresponding category are obtained. This embodiment combines the image features of three channels, and the detection accuracy is extremely high. However, different products have different tolerances for defects, so this embodiment designs a post-processing module to prevent overkill.

[0144] This step is to first use confidence screening to preliminarily filter defects. For defect areas with low confidence, we do not abandon them directly, but use the statistical feature-based anomaly detection mechanism to further determine whether the defect belongs to this category. If the anomaly detection passes, we enter the scoring mechanism for final judgment. Figure 3 , the specific steps are as follows:

[0145] S51: In S4, the suspected defect area, category and confidence are obtained. For each segmented suspected defect area, the features for the category are extracted, including area, roundness, contrast, length, width, etc. The actual meaning of each feature is shown in the following table:

[0146] feature Practical significance Area Total number of pixels in the defect area Perimeter Total number of pixels at the defect area boundary Circularity Evaluate how close a shape is to being circular Length Maximum diameter of defect area Width Average width of defect area Contrast Calculate the grayscale difference between the defect and the background

[0147] For scratches, extract length, width, and contrast; for inclusions, extract area, contrast, and roundness; for bubbles, extract area, contrast, and roundness; for cracks, extract length, perimeter, and contrast; for burrs, extract length and width;

[0148] S52: For defective areas with low confidence <0.5, do not give up immediately, but enter the abnormality detection stage and enter step S53. If the confidence is equal to or higher than the confidence threshold of 0.5, directly enter the scoring mechanism for judgment, that is, enter S54 for judgment;

[0149] S53: For defect areas with confidence lower than the set threshold, perform anomaly detection based on statistical features. Statistical features are obtained by analyzing a large amount of defect data of known categories, including the mean, variance, maximum and minimum values ​​of the features. Check whether the feature value of the defect area with low confidence falls within the statistical range of the category. If the feature value meets the statistical range, it is considered that the defect may belong to this category and enter the next scoring mechanism, that is, enter step S54; otherwise, it is directly marked as overkill;

[0150] S54: Normalize the features of the defective areas with high confidence and those that pass the anomaly detection, and standardize the feature values ​​to the [0,1] interval. The formula is as follows:

[0151]

[0152] f i is the original eigenvalue, f i，min and f i,max are the minimum and maximum values ​​of the characteristic;

[0153] S55: The defect score is calculated using a comprehensive scoring mechanism. The confidence and feature scores jointly determine the final score. The scoring formula is as follows:

[0154] S=β·Conf+(1-β)·(ω1·f1′+ω2·f2′+…ω n ·f n ′) (20)

[0155] β is the confidence weight, ω i is the weight of each feature;

[0156] S56: The scoring threshold S can be set according to the formula. For example, if S>0.4, it is judged as a real defect, otherwise it is judged as an overkill. The scoring formula for each category of defects is as follows:

[0157] S 划痕 =β·Conf+(1-β)·(0.4·f′ 长度 +0.3·f′ 宽度 +0.3·f′ 对比度 ) (twenty one)

[0158] S 夹杂 =β·Conf+(1-β)·(0.5·f′ 对比度 +0.3·f′ 圆度+0.2·f′ 面积 ) (twenty two)

[0159] S 气泡 =β·Conf+(1-β)·(0.4·f′ 面积 +0.4·f′ 圆度 +0.2·f′ 对比度 ) (twenty three)

[0160] S 裂纹 =β·Conf+(1-β)·(0.5·f′ 长度 +0.3·f′ 周长 +0.2·f′ 对比度 ) (twenty four)

[0161] S 毛刺 =β·Conf+(1-β)·(0.6·f′ 长度 +0.4·f′ 宽度 ) (25)

[0162] The above-mentioned specific parameter scoring thresholds can be dynamically adjusted according to the different tolerance levels of each product to defects. The scoring mechanism provided in this embodiment prevents overkill strategies and improves the accuracy and reliability of detection through comprehensive feature extraction and flexible rule application.

[0163] Figure 4 The following is a diagram showing the effect of the detection implementation of a case study. The method of the present invention can accurately detect a variety of defects in glass components.

[0164] The above embodiments are only preferred implementations of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A multi-channel adaptive glass component defect detection method, characterized in that: The following steps are included in sequence: S1: multi-channel image acquisition; The time-sharing stroboscopic technology is used to quickly switch and image three lighting modes to obtain a multi-channel image of the glass component; the three lighting modes are: the light source is set on the front and back of the glass component to be inspected for lighting, and the infrared lighting is performed inside the glass component to be inspected; S2: multi-channel image registration; Registering the multi-channel images obtained in step S1 to align features between the multi-channel images and obtain a registered multi-channel image; S3: multi-channel feature extraction and fusion; By extracting features of each channel of the registered multi-channel image, combining the channel attention mechanism and the spatial attention mechanism, information of different channels is fused to obtain a fused feature map; S4: defect area segmentation; The fused feature map is input into the feature pyramid network to generate a multi-scale feature map; anchor frames covering regions of different sizes and aspect ratios are generated on the multi-scale feature map; the region proposal network RPN is used to perform target classification and bounding box regression on each anchor frame, and a defect mask is generated on the aligned feature map to segment the defect from the original image; S5: Post-processing scoring mechanism to prevent overkill; First, confidence screening is used to preliminarily filter defects. For defect areas with low confidence, a statistical feature anomaly detection mechanism is used to further determine whether the defect belongs to this category. If the feature anomaly detection mechanism passes, the scoring mechanism is used for final determination.

2. The multi-channel adaptive glass component defect detection method according to claim 1, characterized in that: In step S1, for front lighting, a high-brightness and uniform LED light source is selected, and the coaxial LED light source is aligned with the optical axis of the camera and installed to ensure that the light is irradiated onto the glass component vertically along the optical axis of the camera; for back lighting, a uniform backlight LED lamp board is installed below the glass component to be tested, so that the light irradiates the glass component from the back; for infrared lighting, an infrared LED light source with a wavelength in the range of 800-1200nm is used, and the infrared LED light source is installed on the side of the inspection station, so that the infrared light penetrates the glass component material to detect internal defects; using time-sharing stroboscopic technology, according to a preset timing, the strobe controller controls the flashing time of the coaxial light, the back light and the infrared light source in turn, and controls the three light sources to flash quickly in turn, so that each light source illuminates the glass component separately within a set extremely short time, and a multi-channel image of the glass component is obtained.

3. The multi-channel adaptive glass component defect detection method according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21: feature detection; Use ORB to detect feature points; first, use the FAST detection method to detect the feature points from the three channel images I i Extract potential feature points from the formula as follows: in, is the jth feature point detected in the i-th channel image; then the BRIEF descriptor is calculated for each feature point, the formula is as follows: is the feature point the associated binary descriptor; S22: feature point matching; Use the Hamming distance to match feature point descriptors and select the best matching set of glass component feature point pairs. The formula is as follows: M ij is the set of feature point pairs matched between the reference channel and the target channel, ‖.‖ H represents the Hamming distance; S23: Calculate transformation matrix; Use the RANSAC algorithm to calculate the transformation matrix H i , to ensure that images of different channels are aligned, the formula is as follows: H i =RANSAC(M ij ) (4) S24: Apply transformation; Take the forward illumination image I1 as the reference image and perform image alignment. The formula is as follows: I2′=H2I2 (5) I3′=H3I3 (6) Get the registered multi-channel image.

4. The multi-channel adaptive glass component defect detection method according to claim 3, characterized in that: S3 includes the following steps: S31: single channel feature extraction; MobileNetV3 is used as the feature extraction network, and the depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution, thereby reducing the amount of convolution calculation to 1 / 2 of the traditional convolution. times, where a is the convolution kernel size; at the same time, MobileNetV3 adds inverted residual and SE modules to enhance the model's expressiveness while extracting features. The formula is as follows: F1 k =MobileNetV3 k (I1) (7) <h2 style=";text-align:left;direction:ltr">F2<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> MobileNetV3<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> (I′2) (8) F3 k =MobileNetV3 k (I′3) (9) MobileNetV3k represents the K-th layer of convolution operations, Represents the K-th layer feature map of the i-th channel image; S32: Channel attention mechanism; Through the channel attention mechanism, we can adapt to the feature differences under different lighting conditions. W1 and W2 are learnable weight matrices, δ is the Relu activation function, and σ is the sigmoid activation function. First, we convert the multi-channel features into Concatenate and generate channel attention weight A through two layers of fully connected networks c ; Then use the channel weight to perform element-by-element multiplication on each channel feature for weighted feature fusion to obtain F fusion , the formula is as follows: S33: spatial attention mechanism; Use the spatial attention mechanism to assign a weight to each spatial position to emphasize important areas and ignore unimportant parts. First, the feature map is concatenated and input into the Conv convolution operation to generate the weight A. s ; Use the generated spatial attention map to multiply the channel fused feature map element by element to obtain the final fused feature map F att , the formula is as follows:

5. The multi-channel adaptive glass component defect detection method according to claim 4, characterized in that: Step S4 The specific steps include: S41: The fused feature map F att Input into the feature pyramid network to generate multi-scale features to detect defects of different sizes. The formula is as follows: P i =C i +UpSample(P i+1 ) (14) C i is the fusion feature F att The extracted i-th layer feature, UpSample, i.e. P i+1 It is to upsample the high-level feature map to enhance the low-level feature map; S42: Generate anchor boxes on the multi-scale feature map to cover areas of different sizes and aspect ratios; use the region proposal network RPN to perform target classification and bounding box regression on each anchor box; then use ROI pooling to extract and align the regions from the multi-scale feature map based on the candidate regions generated by RPN, as follows: R=ROIAlign(P,ROIs) (15) Each aligned candidate region is classified and predicted and the bounding box is regressed. The formula is as follows: P=Softmax(W c R) (16) t=W b R (17) P is the class probability distribution, W c is the classification weight matrix, t is the regression parameter, W b is the regression weight matrix; S43: Finally, a defect mask is generated on the aligned feature map to segment the defect from the original image. The formula is as follows: M=σ(W m (18) M is the generated binary mask, W m It is the weight matrix generated by the mask, and the defect area and the corresponding category are obtained.

6. The multi-channel adaptive glass component defect detection method according to claim 5, characterized in that: Step S5 specifically includes the following steps: S51: According to the suspected defect area, category and confidence obtained in step S4, for each segmented suspected defect area, extract the features for the category, including area, roundness, contrast, length, width, amplitude and frequency; for scratches, extract length, width and contrast; for inclusions, extract area, contrast and roundness; for bubbles, extract area, contrast and roundness; for cracks, extract length, perimeter and contrast; for ripples, extract amplitude and frequency; for burrs, extract length and width; S52: For defective areas with low confidence, when the confidence is less than the preset confidence threshold, do not give up immediately, but enter the abnormality detection stage, that is, enter step S53; if the confidence is higher than or equal to the confidence threshold, directly enter the scoring mechanism, that is, enter step S54 for judgment; S53: The anomaly detection stage is to perform anomaly detection based on statistical features; the statistical features are obtained by analyzing a large amount of defect data of known categories, including the mean, variance, maximum and minimum values ​​of the features; check whether the feature value of the defect area with low confidence falls within the statistical range of the category; if the feature value meets the statistical range, it is considered that the defect may belong to the category, and enter step S54; otherwise, it is directly marked as overkill; S54: Normalize the features of the defective areas with high confidence and those that pass the anomaly detection, and standardize the feature values ​​to the [0,1] interval. The formula is as follows: f i is the original eigenvalue, f i，min and f i，max are the minimum and maximum values ​​of the characteristic; S55: The defect score is calculated using a comprehensive scoring mechanism. The confidence and feature scores jointly determine the final score. The scoring formula is as follows: S=β·Conf+(1-β)·(ω1·f1′+ω2·f2′+…ω n ·f n ′) (20) β is the confidence weight, ω i is the weight of each feature; S56: The scoring threshold can be set according to the following formula. If the defect score S is greater than the scoring threshold, it is determined to be a real defect; otherwise, it is determined to be an overkill. The scoring formulas for each category of defects are as follows: S 划痕 =β·Conf+(1-β)·(0.4·f′ 长度 +0.3·f′ 宽度 +0.3·f′ 对比度 ) (21) S 夹杂 =β·Conf+(1-β)·(0.5·f′ 对比度 +0.3·f′ 圆度 +0.2·f′ 面积 ) (22) S 气泡 =β·Conf+(1-β)·(0.4·f′ 面积 +0.4·f′ 圆度 +0.2·f′ 对比度 ) (23) S 裂纹 =β·Conf+(1-β)·(0.5·f′ 长度 +0.3·f′ 周长 +0.2·f′ 对比度 ) (24) S 波纹 =β·Conf+(1-β)·(0.5·f′ 幅度 +0.5 f′ 频率 ) (25) S 毛刺 =β·Conf+(1-β)·(0.6·f′ 长度 +0.4 f′ 宽度 ) (26) The scoring threshold is dynamically adjusted according to the different tolerance levels of each product to defects.

Images