Screen defect detection method and system based on energy operator and multimodal feature fusion

Abstract

The present invention discloses a screen defect detection method and system that integrates an energy operator and multimodal features, comprising the following steps: performing morphological energy calculation on the extracted defect contour, and characterizing the curvature by the energy value of the low-frequency component; calculating the standard deviation of the defect width; extracting the mean brightness of the defect area in the HSV space, wherein the mean brightness of scratches is higher than that of hairs, and calculating the grayscale gradient change along the long axis of the defect, wherein the gradient fluctuation amplitude of hairs is smaller than that of scratches; constructing a manually annotated data set, and establishing a mapping relationship between multimodal features and manual scores through a regression model; dynamically adjusting the classification threshold to ensure that the Kappa consistency coefficient between the machine judgment result and the manual re-inspection is ≥0.85. The present invention constructs a quantitative classification model by integrating the defect morphological energy operator with multimodal optical features, and realizes calibration mapping between machine vision results and human eye detection, thereby improving the defect classification accuracy and production line detection consistency.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image processing technology, and specifically to an intelligent algorithm system for detecting appearance defects on mobile phones, tablets, and computer screens. In particular, it relates to a defect classification method that integrates morphological feature energy analysis with optical properties, and a calibration technology for calibrating its consistency with human eye detection results. Background Art

[0002] Currently, screen appearance defect detection mainly relies on traditional image processing technologies (such as edge detection and threshold segmentation) or classification models based on deep learning.

[0003] The limitations of existing technologies are as follows: Morphologically similar defects are difficult to distinguish: Both hair and scratch defects appear as linear features in two-dimensional images. Traditional methods rely on single features such as aspect ratio and grayscale mean, and are unable to effectively distinguish differences such as curvature and uniformity. Insufficient feature representation: Existing algorithms do not fully integrate morphological energy and optical properties. Lack of consistency between human and machine inspection: Existing methods lack a calibration mechanism with manual inspection results, resulting in deviations between machine judgment and manual re-inspection, affecting production line quality control efficiency. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a screen defect detection method and system that integrates shape energy operators and multimodal features. By integrating the defect morphological energy operator with multimodal optical features, a quantitative classification model is constructed, and calibration mapping between machine vision results and human eye detection is achieved, thereby improving defect classification accuracy and production line detection consistency.

[0005] The present invention proposes a screen defect detection method that integrates energy operators and multimodal features, comprising the following steps:

[0006] Step S1: morphological energy calculation is performed on the extracted defect contour, and the curvature is characterized by the low-frequency component energy value E; that is, morphological energy calculation is performed on the extracted defect contour, and the curvature is accurately characterized by the energy value of the low-frequency component.

[0007] Step S2: Calculate the standard deviation of the defect width;

[0008] Step S3: The mean brightness of the defect area is extracted in HSV color space. The mean brightness of scratches is higher than that of hairs. The grayscale gradient variation along the long axis of the defect is calculated. The gradient fluctuation amplitude of hairs is smaller than that of scratches. The mean brightness of the defect area is accurately extracted in HSV color space. Generally, the mean brightness of scratches is higher than that of hairs. Furthermore, the grayscale gradient variation along the long axis of the defect is calculated. The gradient fluctuation amplitude of hairs is smaller than that of scratches.

[0009] Step S4: Construct a manually annotated dataset. For each defect, manually determine the defect type (hair or scratch). Take the feature vector (E, σ_w, V, G) as input and the manual score as output. Establish a mapping relationship between multiple features and manual scores, and determine the initial value of the joint classification threshold T_thr based on this.

[0010] Step S5: Based on the low-frequency component energy value E, width standard deviation σ_w, brightness mean V, and grayscale gradient fluctuation amplitude G extracted in steps S1 to S3, a joint discriminant function S = f(E, σ_w, V, G) is constructed; a joint classification threshold T_thr is set, and when S ≥ T_thr, the defect is judged to be a hair, otherwise it is judged to be a scratch; during the online detection process, the CoheNKappa consistency coefficient K between the machine judgment result and the manual re-inspection result is continuously counted; when K <0.85, T_thr is adaptively fine-tuned or the feature weights in the function f are adjusted according to the changing direction of the false negative rate and the false positive rate, so that K is always maintained at 0.85 or above.

[0011] As a further solution of the present invention, in step S1, the method for calculating the morphological energy of the defect contour includes Fourier descriptor calculation or Zernike moment calculation. The morphological energy calculation work is carried out on the extracted defect contour. The defect contour can be converted from the spatial domain to the frequency domain through Fourier descriptor calculation, and the energy distribution of each frequency component is analyzed to obtain the morphological energy. Alternatively, Zernike moment calculation can be used to accurately extract the geometric feature information of the defect contour by utilizing the orthogonality of Zernike polynomials, thereby completing the morphological energy calculation, and finally accurately characterizing the degree of curvature using the energy value of the low-frequency component.

[0012] As a further solution of the present invention, in step S1: the Fourier descriptor of the extracted defect contour is calculated, wherein the formula of the low-frequency component energy value is: E=(F1+F2) / (F1+F2+…+F N / 2 ), N is the Fourier descriptor, F N is the energy of the Nth Fourier descriptor. Hair is significantly lower than scratches due to its high curvature energy value, and E ≤ 0.98 is determined to be hair.

[0013] Specifically, the Fourier descriptor is calculated for the extracted defect contour. The spatial information of the defect contour is converted to the frequency domain through Fourier transform. The calculation of the Fourier descriptor is based on the contour point coordinate sequence (complex form): z(k)=x(k)+jy(k), where z(k) represents the complex representation of the kth contour point, x(k) and y(k) are its horizontal and vertical coordinates respectively, and j is the imaginary unit. Discrete Fourier transform (DFT) is performed on z(k) to obtain a series of complex coefficients C n : Where: N is the total number of contours, C n is the complex coefficient of the nth frequency component. The Fourier descriptor often uses the modulus of these complex coefficients to represent its "energy" or "intensity". Therefore, the energy of the Nth Fourier descriptor (denoted as F N ) is defined as: F N = The formula for calculating the energy value of the low-frequency component is: E=(F1+F2) / (F1+F2+…+F N / 2) Because hair has high curvature, its low-frequency component energy value is significantly lower than that of scratches. When E≤0.98, the defect is determined to be hair.

[0014] As a further solution of the present invention, in step S2: the hair σ_w is less than 0.2 mm, wherein σ_w is the standard deviation of the defect width.

[0015] As a further solution of the present invention, in step S3: convert to HSV space, extract the average brightness of the defective area, if V ≥ 200 and the grayscale gradient fluctuation ≥ 25%, it is determined to be a scratch, where V is the average brightness of the defective area.

[0016] As a further solution of the present invention, a defect extraction algorithm is first used to extract the target area, and then the defect results are refined, and the contour points are subjected to Fourier transform to obtain the Fourier energy ratio. If the energy ratio is less than 0.98, the width standard deviation is judged. If the standard deviation is less than 0.2, it is judged as hair. Otherwise, the brightness and grayscale gradient fluctuations are judged. When the brightness is greater than 200 and the grayscale gradient fluctuation is greater than 25%, it is a scratch, otherwise it is hair.

[0017] As a further solution of the present invention, the defect contour is extracted by first using Gaussian filtering to remove background noise, then using adaptive threshold segmentation to extract the defect area, and finally using a skeletonization algorithm to obtain the defect skeleton contour points.

[0018] As a further solution of the present invention, in step S4: the process of manually labeling the product image is as follows: based on the captured image, the degree of curvature of the defect is evaluated and sorted, and the defect type is manually observed and divided into two categories: "hair" and "scratches"; all labeling results are statistically analyzed, and the multimodal features (E, σ_w, V, G) output by the algorithm and the manual labeling results are input into the SVR model; after training, the model generates a mapping function: the mapping function S=f(E, σ_w, V, G) is obtained. To achieve dynamic correction of the classification threshold, it is first necessary to collect the classification results during the model operation, including true positive, false positive, true negative and false negative data, and adjust them according to the error changes fed back in real time: an increase in the false negative rate indicates an increase in missed detections, and the threshold should be appropriately lowered to classify more samples as positive; an increase in the false positive rate indicates an increase in false alarms, and the threshold should be increased to reduce false alarms; through this dynamic adjustment mechanism, that is, adjusting the weights of each feature in the function f, the system can adapt to changes in data distribution in real time, thereby maintaining high classification performance.

[0019] The present invention also provides a screen defect detection system that integrates an energy operator and multimodal features, including a morphological energy quantification module, an optical feature analysis module and a human-machine consistency calibration module; the morphological energy quantification module is used to calculate the Fourier descriptor of the extracted defect contour, characterize the curvature by the energy value of the low-frequency component, and calculate the standard deviation of the defect width; the optical feature analysis module is used to extract the mean brightness of the defect area in the HSV space, the mean brightness of the scratch is higher than that of the hair, and calculate the grayscale gradient change along the long axis of the defect, and the gradient fluctuation amplitude of the hair is smaller than that of the scratch; the human-machine consistency calibration module is used to establish a mapping relationship between the multimodal features (E, σ_w, V, G) and the manual score through a regression model, and dynamically adjust the classification threshold to ensure that the Kappa consistency coefficient of the machine judgment result and the manual re-inspection is ≥0.85.

[0020] As a further solution of the present invention, the regression model is a support vector regression SVR model.

[0021] In the present invention, the preliminary judgment result in step S1 can be further confirmed by the defect width standard deviation in step S2. If step S1 suspects a hair, the accuracy of the judgment is enhanced when the width standard deviation meets the hair standard (σ_w < 0.2mm); if not, a comprehensive analysis can be conducted in conjunction with subsequent steps. The HSV spatial feature extraction and determination complements the previous two steps. When steps S1 and S2 cannot clearly define the defect type, step S3 determines the defect based on the brightness mean and grayscale gradient fluctuation. If the scratch condition is met (V ≥ 200 and grayscale gradient fluctuation ≥ 25%), it is determined to be a scratch; otherwise, a comprehensive determination is made based on information from other steps to determine whether it is a hair. This step and the previous two steps analyze defect characteristics from different perspectives, jointly improving the defect type determination system.

[0022] The present invention integrates defect extraction and the step-by-step determination of defect types based on different features into an integrated whole. The initial defect extraction algorithm provides an accurate data foundation for subsequent feature calculation and determination. In subsequent steps, the determination results based on different features influence each other. For example, the Fourier energy ratio determines whether to proceed to the width standard deviation determination, while the width standard deviation and energy ratio jointly determine whether to proceed to the brightness and grayscale gradient fluctuation determination. Each step is progressive, continuously screening and confirming the defect type.

[0023] In this invention, the mapping function derived from manual labeling and model training provides a more scientific basis for defect type determination, mutually validating and complementing the judgment results of the previous steps. Dynamic adjustment of the classification threshold optimizes the judgment system established in all previous steps based on the true positive, false positive, true negative, and false negative data collected during model operation. When missed detections (increased false negative rate) or false alarms (increased false positive rate) occur, threshold adjustments are made to better adapt the entire defect detection process to different data distributions, improving detection accuracy and stability, and thus promoting the effectiveness of the previous steps.

[0024] The present invention has the following beneficial effects: the present invention removes background noise by using Gaussian filtering, extracts screen defects by using an adaptive threshold segmentation algorithm, and refines the defect image by using the Zhang-Suen algorithm to obtain a skeleton diagram of the defect; the contour points in the skeleton diagram are used for Fourier transform, and the first two coefficients are used to calculate the low-frequency energy to obtain the overall trend of the defect. The straighter the overall shape, the greater the low-frequency energy, and the greater the ratio of low-frequency energy to total energy; at the same time, the standard deviation of the width is calculated. First, the distance transformation is performed on the defect image to obtain the width of the defect. According to the distance from the starting point to the end of the defect, the width of the defect is obtained. The width value can be used to calculate the standard deviation of the width, which is equivalent to the uniformity of the defect width. The smaller the standard deviation, the more uniform the defect. Since scratches are long, thin lines, their width varies less than that of hair. Finally, the optical characteristics of scratches and hair are used for final judgment. Hair appears darker than scratches because scratches are on the screen surface, which has stronger reflectivity than hair. By converting the image to the HSV color space, the value of the V channel can be used to determine the brightness of the defect. Since scratches can have different depths, the intensity consistency of the reflected light is not as good as that of hair. Therefore, the fluctuation of the grayscale gradient can be used to determine the brightness variation of the defect. Based on the above method, the defect can be classified as hair or scratch.

[0025] In order to more clearly illustrate the structural features and effects of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is the overall logic flow chart of the present invention;

[0027] Figure 2 This is the defect extraction and skeletonization flowchart mentioned in the present invention;

[0028] Figure 3 This is a diagram of the defect extraction process mentioned in the present invention;

[0029] Figure 4 This is the Fourier descriptor result diagram mentioned in the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below with reference to the accompanying drawings and related knowledge, and described clearly and completely. Obviously, the described applications are only part of the embodiments of the present invention, rather than all of the embodiments.

[0031] Example 1, a screen defect detection method based on energy operator and multimodal feature fusion, includes the following steps:

[0032] Step S1: Calculate the morphological energy of the extracted defect contour and characterize the curvature by the low-frequency component energy value; specifically: Calculate the Fourier descriptor of the extracted defect contour and characterize the curvature by the low-frequency component energy value (formula: E=(F1+F2) / (F1+F2+…+F N / 2 ), where N is the Fourier descriptor, characterizes curvature. Hairs have a significantly lower energy value than scratches due to their high curvature (E≤0.98 is considered hair).

[0033] Step S2: Calculate the standard deviation of the defect width; specifically, introduce uniformity evaluation: calculate the standard deviation of the defect width (σ_w), and the hair σ_w is less than 0.2mm;

[0034] Step S3: Extract the average brightness of the defect area in the HSV space. The average brightness of the scratch is higher than that of the hair. Calculate the grayscale gradient change along the long axis of the defect. The gradient fluctuation amplitude of the hair is smaller than that of the scratch. Specifically: Brightness contrast: Extract the average brightness of the defect area in the HSV space (V channel). The average brightness of the scratch is higher than that of the hair (threshold is V ≥ 200). Grayscale gradient analysis: Calculate the grayscale gradient change along the long axis of the defect. The gradient fluctuation amplitude of the hair is smaller than that of the scratch.

[0035] Step S4: Construct a manually annotated dataset and establish a mapping relationship between the multimodal features (E, σ_w, V, G) output by the algorithm and the manual scores (0-1 confidence) through a regression model.

[0036] Step S5: Dynamically adjust the classification threshold to ensure that the Kappa consistency coefficient between the machine judgment result and the manual re-inspection is ≥0.85.

[0037] In this embodiment, the morphological energy calculation can use Zernike moments instead of Fourier descriptors. Specifically, Zernike moments are an image description method based on orthogonal Zernike polynomials, which can effectively capture the global and local shape features of the image area. Low-order moments mainly capture global shape features, such as overall contours, symmetry, and overall distribution information, while high-order moments focus more on local details, such as subtle changes in edges, local irregularities, and the degree of curvature. When detecting the degree of curvature, a feature vector can be constructed, which contains multiple Zernike moments of different orders. By comparing the amplitude distribution of these moments, the overall and local curvature information of the defect can be obtained. A higher high-order moment amplitude may indicate that the defect edge has a larger curvature or irregularity, while a lower high-order moment implies that the edge is smoother.

[0038] Furthermore, uniformity analysis can be replaced by the gray-level co-occurrence matrix (GLCM) contrast feature. Specifically, the contrast feature of the gray-level co-occurrence matrix (GLCM) mainly measures the degree of local variation between image gray levels, that is, the grayscale difference between pixels. The mathematical definition of GLCM contrast is as follows: ;in: is the joint probability distribution of pixel gray levels i and j in the GLCM matrix.

[0039] Represents the squared difference between gray levels.

[0040] A higher contrast value means a greater grayscale variation in the image and a higher degree of local unevenness; a lower contrast value means better grayscale uniformity.

[0041] Low contrast (close to 0): The grayscale value inside the defect area changes little, indicating that the defect area is relatively uniform.

[0042] High contrast (larger value): The grayscale value inside the defect area varies greatly, indicating that there is a large grayscale difference in the defect area.

[0043] This invention offers significant benefits in many areas. During defect extraction, Gaussian filtering effectively removes background noise, paving the way for subsequent accurate detection. Subsequently, an adaptive threshold segmentation algorithm is employed to precisely extract screen defects, ensuring that no critical information is missed. The Zhang-Suen algorithm then refines the defect image, successfully generating a skeleton image of the defect, providing a clear structural basis for in-depth analysis of defect characteristics.

[0044] During the defect feature analysis phase, a Fourier transform is performed on the contour points in the skeleton image, and the low-frequency energy is calculated using the first two coefficients to cleverly capture the overall trend of the defect. It is worth noting that the straighter the overall shape, the greater the low-frequency energy, and the greater the ratio of low-frequency energy to total energy. This provides a quantitative indicator for determining the approximate shape of the defect. At the same time, the defect width is obtained by performing a distance transform on the defect image, and the standard deviation of the width is calculated based on the width values ​​from the starting point to the end of the defect. This standard deviation reflects the uniformity of the defect width; the smaller the standard deviation, the higher the uniformity. Because scratches appear as long, thin lines, their width varies less than that of hair, and this characteristic is reflected in the calculation results of the width standard deviation.

[0045] In the final judgment stage, the optical characteristics of scratches and hair are fully utilized. Hair is dimmer than scratches because scratches are on the surface of the screen, and the screen's reflective ability is stronger than hair. By converting the image to the HSV color space, the value of the V channel can be used to effectively judge the brightness of the defect. In addition, due to the difference in depth of scratches, the intensity consistency of their reflected light is not as good as that of hair, so the fluctuation of the grayscale gradient can be used to judge the change in the brightness of the defect. Combining the above series of scientific and reasonable methods, the present invention can ultimately accurately determine whether the defect is hair or scratch, greatly improving the accuracy and reliability of defect detection.

[0046] Example 2. The present invention also provides a screen defect detection system that integrates an energy operator and multimodal features, including a morphological energy quantification module, an optical feature analysis module, and a human-machine consistency calibration module; the morphological energy quantification module is used to calculate the Fourier descriptor of the extracted defect contour, characterize the curvature by the energy value of the low-frequency component, and calculate the standard deviation of the defect width; the optical feature analysis module is used to extract the mean brightness of the defect area in the HSV space, the mean brightness of the scratch is higher than that of the hair, and calculate the grayscale gradient change along the long axis of the defect, and the gradient fluctuation amplitude of the hair is smaller than that of the scratch; the human-machine consistency calibration module is used to establish a mapping relationship between the multimodal features (E, σ_w, V, G) output by the algorithm and the manual score through a regression model, and dynamically adjust the classification threshold to ensure that the Kappa consistency coefficient of the machine judgment result and the manual re-inspection is ≥0.85; further preferably, the regression model is a support vector regression SVR model.

[0047] Example 3, reference Figures 1-4 The screen defect detection method that integrates the energy operator and multimodal features mainly includes: first using a defect extraction algorithm to extract the target area, then refining the defect results, performing Fourier transform on its contour points, and obtaining the Fourier energy ratio. If the energy ratio is less than 0.98, the width standard deviation is judged. If the standard deviation is less than 0.2, it is judged as hair. Otherwise, the brightness and grayscale gradient fluctuations are judged. When the brightness is greater than 200 and the grayscale gradient fluctuation is greater than 25%, it is a scratch, otherwise it is hair.

[0048] Reference 2- Figure 3 As shown in the figure, first use the Gaussian filter and other denoising filtering algorithms to remove background noise, use the adaptive threshold segmentation and other detection algorithms to extract the defect area, and finally use the skeletonization algorithm to obtain the defect skeleton contour points. Figure 3 From left to right in the figure are the original image, the background noise removal image, the defect area image extracted using detection algorithms such as adaptive threshold segmentation, and the defect skeleton image.

[0049] Referring to Figure 4, the extracted defect skeleton is subjected to Fourier transform to obtain the defect Fourier descriptor, as shown in Figure 4 (b) and Figure 4 (d), where the horizontal axis represents the Nth Fourier descriptor, the first 20 are intercepted for display, and the vertical axis represents the energy value of the Nth Fourier descriptor. It can be seen that the straighter the defect as a whole, the larger the value in the second Fourier descriptor, because the straighter the shape, the higher the low-frequency energy ratio, that is, E=(F1+F2) / (F1+F2+…+F N / 2 ) The larger the value is, the more likely it is that the defect is curved based on the ratio of its first two components to the total components.

[0050] The specific process is:

[0051] Image preprocessing:

[0052] The main purpose of using a Gaussian filter is to smooth the image and reduce the interference of background noise on subsequent processing. The Gaussian filter performs a convolution operation on the image, smoothing local pixels with a kernel with a normal distribution of weights (σ = 1.5). This not only removes noise but also effectively preserves edge information. Adaptive thresholding is used to extract defects. Unlike global thresholding, adaptive thresholding methods determine the threshold based on the grayscale statistics of each local area in the image, making it more responsive to uneven lighting or complex backgrounds. Common methods include calculating local thresholds based on the local mean or Gaussian weighted mean. Using common thinning methods such as the Zhang-Suen method, redundant pixels at the boundary are repeatedly removed through iterations, ultimately shrinking the binary defect area to a one-pixel-wide skeleton.

[0053] Morphological energy calculation:

[0054] Perform discrete Fourier transform (DFT) on the contour point sequence to obtain a set of Fourier coefficients {F N Among these coefficients, the first two have special meanings: F1 contains the overall position information of the contour (translation information), and F2 reflects the main extension direction of the curve in space (similar to the first main direction of principal component analysis). The straighter the contour point, i.e. the defect, the more certain the direction, and the larger the F2 component value. In order to calculate the curvature of the defect, we calculate F 1和 F2 energy ratio (E=(F 1+ F2) / (F1+F2+…+F N / 2 ), N is the Fourier descriptor).

[0055] If E≤0.98 and σ_w<0.2mm, it is determined to be a hair; otherwise, it enters the optical feature verification.

[0056] Optical feature verification:

[0057] Convert to HSV space and extract the mean V channel value of the defect area. If V ≥ 200 and the grayscale gradient fluctuation ≥ 25%, it is determined to be a scratch.

[0058] Human-machine calibration:

[0059] First, product images are manually annotated. Based on the captured images, the degree of curvature of defects is assessed and ranked. Defect types are also manually observed and classified into two categories: "hair" and "scratches." Finally, all annotation results are statistically analyzed. The multimodal features (E, σ_w, V, G) output by the algorithm and the manual annotations are fed into the SVR model. SVR (Support Vector Regression) is based on the principles of support vector machines (SVMs). It seeks an optimal hyperplane such that the prediction error for most data points falls within a preset ε-insensitive interval while maintaining model smoothness. After training, the model generates a mapping function: S = f(E, σ_w, V, G). To dynamically adjust the classification threshold, it is first necessary to collect classification results during model execution, including true positive (TP), false positive (FP), true negative (TN), and false negative (FN) data. Next, adjustments are made based on real-time feedback on error changes. An increase in the false negative rate indicates an increase in missed detections. In this case, the threshold should be appropriately lowered to classify more samples as positive.

[0060] An increase in the false positive rate indicates an increase in false positives. In this case, the threshold should be raised to reduce false alarms.

[0061] Through this dynamic adjustment mechanism, the system can adapt to changes in data distribution in real time, thereby maintaining high classification performance.

[0062] The technical principles of the present invention have been described above in conjunction with specific embodiments, which are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention fall within the scope of protection of the present invention. Those skilled in the art will be able to conceive of other specific embodiments of the present invention without inventive effort, and such methods will fall within the scope of protection of the present invention.

Claims

1. A screen defect detection method based on the fusion of energy operator and multimodal features, characterized in that: The following steps are involved: Step S1: Calculate the morphological energy of the extracted defect contour and characterize the curvature by the low-frequency component energy value E; Step S2: Calculate the defect width standard deviation σ_w; Step S3: Extract the mean brightness value V of the defect area in the HSV space. The mean brightness value of the scratch is higher than that of the hair. Also, calculate the grayscale gradient fluctuation amplitude G along the long axis of the defect. The gradient fluctuation amplitude of the hair is smaller than that of the scratch. Step S4: Construct a manually annotated dataset. For each manually determined defect type, use the feature vector as input, establish a mapping relationship between the feature vector and the manual score, and determine the initial value of the joint classification threshold T_thr. The feature vector includes the obtained low-frequency component energy value E, width standard deviation σ_w, brightness mean V, and grayscale gradient fluctuation amplitude G. Step S5: Based on the obtained low-frequency component energy value E, width standard deviation σ_w, brightness mean V, and grayscale gradient fluctuation amplitude G, obtain the mapping function S=f(E,σ_w,V,G); set the joint classification threshold T_thr, and when S ≥ T_thr, determine the defect as a hair, otherwise it is determined as a scratch; during the online inspection process, continuously calculate the consistency coefficient K between the machine judgment result and the manual re-inspection result. When K < 0.85, adaptively fine-tune T_thr or adjust the feature weights in the function f according to the change direction of the false negative rate and the false positive rate, so that K is always maintained at 0.85 or above.

2. The screen defect detection method using the energy operator and multimodal feature fusion method according to claim 1, characterized in that: In step S1, the defect contour is calculated using a morphological energy method including Fourier descriptor calculation or Zernike moment calculation.

3. The screen defect detection method using the energy operator and multimodal feature fusion method according to claim 2, characterized in that: In step S1: the Fourier descriptor is calculated for the extracted defect contour, wherein the formula for the energy value of the low-frequency component is: E=(F1+F2) / (F1+F2+…+F N / 2 ), N is the Fourier descriptor, F N is the energy of the Nth Fourier descriptor. Hair is significantly lower than scratches due to its high curvature and low-frequency energy value. When E≤0.98, it is determined to be hair.

4. The screen defect detection method using the energy operator and multimodal feature fusion method as claimed in claim 3, characterized in that: In the step S2: the hair σ_w is less than 0.2 mm.

5. The screen defect detection method using the energy operator and multimodal feature fusion method as claimed in claim 2, characterized in that: In step S3, the image is converted to the HSV space, and the average brightness of the defective area is extracted. If V ≥ 200 and the grayscale gradient fluctuation is ≥ 25%, it is determined to be a scratch.

6. The screen defect detection method using the energy operator and multimodal feature fusion method as claimed in claim 3, characterized in that: First, use the defect extraction algorithm to extract the target area. Then, refine the defect results and perform Fourier transform on the contour points to obtain the low-frequency component energy value. If the low-frequency component energy value is less than 0.98, the width standard deviation is determined. If the standard deviation is less than 0.2, it is judged as a hair. Otherwise, the brightness mean and grayscale gradient fluctuation are determined. If the brightness mean is greater than 200 and the grayscale gradient fluctuation is greater than 25%, it is a scratch, otherwise it is a hair.

7. The screen defect detection method of energy operator and multimodal feature fusion according to claim 6, characterized in that: The defect contour is extracted as follows: first, Gaussian filtering is used to remove background noise, adaptive threshold segmentation is used to extract the defect area, and finally, the skeletonization algorithm is used to obtain the defect skeleton contour points.

8. The screen defect detection method using the fusion of energy operator and multimodal features as claimed in claim 7, characterized in that: In step S4, the process of manually labeling the product image is as follows: based on the captured image, the degree of curvature of the defect is evaluated and sorted, and the defect type is manually observed and classified into two categories: "hair" and "scratches"; all labeling results are statistically analyzed, and the low-frequency component energy value E, width standard deviation σ_w, brightness mean V, and grayscale gradient fluctuation amplitude G are calculated. The feature vector and the manual score are input into the support vector regression model for training to obtain a mapping function S=f(E,σ_w,V,G); T_thr or the feature weights in the adjustment function f are adaptively fine-tuned according to the direction of change of the false negative rate and the false positive rate. First, it is necessary to collect classification results during the model operation, including true positive, false positive, true negative, and false negative data, and adjust T_thr or the feature weights in the adjustment function f according to the real-time feedback error change. An increase in the false negative rate indicates an increase in missed detections. In this case, the threshold should be appropriately lowered to classify more samples as positive. An increase in the false positive rate indicates an increase in false positives. In this case, the threshold should be increased to reduce false alarms.

9. A system for executing the screen defect detection method of energy operator and multimodal feature fusion as claimed in claim 1, characterized in that: It includes a morphological energy quantification module, an optical feature analysis module, and a human-machine consistency calibration module; the morphological energy quantification module is used to calculate the Fourier descriptor of the extracted defect contour, characterize the curvature by the low-frequency component energy value, and calculate the standard deviation of the defect width; The optical feature analysis module is used to extract the average brightness of the defect area in the HSV space. The average brightness of scratches is higher than that of hairs, and the grayscale gradient change along the long axis of the defect is calculated. The gradient fluctuation amplitude of hairs is smaller than that of scratches. The human-machine consistency calibration module is used to establish a mapping relationship between feature vectors and manual scores, and dynamically adjust the weights of each feature to ensure that the Kappa consistency coefficient between the machine judgment results and manual re-inspection is ≥0.85.

Images