Image recognition method and system applied to touch screen defect detection

Abstract

This invention relates to the field of touchscreen image recognition technology, specifically to an image recognition method and system for touchscreen defect detection. The invention first performs adaptive illumination equalization processing on the acquired touchscreen surface image to obtain an illumination-compensated image with target brightness distribution characteristics. Next, a multi-branch feature extraction network is used to extract point-like and line-like defect features from the illumination-compensated image, obtaining a set of point-like defect features containing contour curvature and grayscale distribution features, and a set of line-like defect features containing directional gradient and length continuity features. Then, feature decoupling processing is performed on these two feature sets to suppress cross-interference between different types of defect features, resulting in independent features. Finally, defect classification is performed based on the decoupled independent features, generating detection results containing defect type and defect region coordinates, thus improving the accuracy and reliability of the detection.

Description

Technical Field

[0001] This invention relates to the field of touch screen image recognition technology, and more specifically, to an image recognition method and system for detecting defects in touch screens. Background Technology

[0002] In the manufacturing process of touchscreens, the detection of surface defects is crucial, as the results directly impact the quality and performance of the touchscreen. Currently, existing technologies for touchscreen surface defect detection mainly employ traditional image processing and machine learning methods. Traditional image processing methods typically include steps such as image enhancement, edge detection, and thresholding. This involves preprocessing the acquired touchscreen surface image, extracting features such as grayscale values ​​and textures, and then using machine learning algorithms such as support vector machines and decision trees for defect classification and identification.

[0003] However, traditional image processing methods are highly sensitive to changes in lighting conditions. Uneven lighting can easily lead to inaccurate image feature extraction, affecting the accuracy of defect detection. Furthermore, traditional methods struggle to effectively distinguish between different types of defect features, and cross-interference between these types can result in inaccurate defect classification. In addition, existing machine learning algorithms exhibit poor generalization ability when processing complex defect features, failing to accurately identify and locate even small or complex defects, thus reducing the accuracy and reliability of touchscreen surface defect detection. Summary of the Invention

[0004] This invention provides an image recognition method for touchscreen defect detection. The method includes: performing adaptive illumination equalization processing on an acquired touchscreen surface image to obtain an illumination-compensated image with target brightness distribution characteristics; extracting point-like defect features and line-like defect features from the illumination-compensated image using a multi-branch feature extraction network to obtain a set of point-like defect features and a set of line-like defect features; wherein the set of point-like defect features includes the contour curvature features and gray-level distribution features of the defect region, and the set of line-like defect features includes the orientation gradient features and length continuity features of the defect region; performing feature decoupling processing on the set of point-like defect features and the set of line-like defect features to obtain decoupled independent point-like defect features and decoupled independent line-like defect features; wherein the feature decoupling processing is used to suppress cross-interference between different types of defect features; and performing defect classification judgment based on the decoupled independent point-like defect features and the decoupled independent line-like defect features to generate a defect detection result containing the defect type and defect region coordinates.

[0005] The present invention also provides an image recognition system for detecting defects in touch screens, comprising: a memory for storing program instructions and data; and a processor for coupling with the memory to execute the instructions in the memory to implement the method described above.

[0006] The present invention also provides a computer storage medium comprising instructions that, when executed on a processor, implement the above-described method.

[0007] This invention improves the lighting conditions of a touchscreen by performing adaptive illumination equalization on the acquired touchscreen surface image to obtain an illumination-compensated image with target brightness distribution characteristics. A multi-branch feature extraction network is used to extract point defect feature sets and line defect feature sets, where the point defect feature set includes contour curvature features and grayscale distribution features, and the line defect feature set includes directional gradient features and length continuity features, achieving comprehensive and targeted extraction of different types of defect features. Feature decoupling is performed on the above feature sets to suppress cross-interference between different types of defect features, resulting in decoupled independent point defect features and decoupled independent line defect features, improving feature independence and accuracy. Based on the decoupled independent features, defect classification is performed, generating defect detection results containing defect type and defect region coordinates, significantly improving the accuracy and reliability of touchscreen surface defect detection, and enabling more precise identification and location of different types of defects. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating the steps of an image recognition method for detecting defects in a touchscreen, provided by an embodiment of the present invention.

[0009] Figure 2 This is a structural block diagram of an image recognition system for touchscreen defect detection provided in an embodiment of the present invention. Detailed Implementation

[0010] Please see Figure 1 , Figure 1 This is a flowchart illustrating an image recognition method for touchscreen defect detection provided in an embodiment of the present invention. The method is executed by an image recognition system for touchscreen defect detection and may further include steps 110-140.

[0011] Step 110: Perform adaptive illumination equalization processing on the acquired touchscreen surface image to obtain an illumination-compensated image with target brightness distribution characteristics.

[0012] In touchscreen defect detection scenarios, the acquired touchscreen surface images may exhibit uneven illumination due to ambient lighting conditions, such as some areas being too bright while others are too dark. This uneven illumination can interfere with subsequent defect feature extraction and detection, making some defects difficult to identify accurately due to lighting issues. Therefore, adaptive illumination equalization processing is necessary for the acquired touchscreen surface images. Adaptive illumination equalization dynamically adjusts the brightness of different areas of the image based on its own brightness distribution. Specifically, it analyzes the brightness values ​​of different areas in the image, appropriately reducing the brightness of areas with higher brightness and increasing the brightness of areas with lower brightness. This process is achieved through a series of calculations and processing steps. First, the image is divided into multiple small regions, and the brightness of each small region is statistically analyzed to obtain information such as the average brightness of that region. Then, based on this statistical information, a suitable brightness adjustment coefficient is calculated for each small region. Next, these adjustment coefficients are used to adjust the pixel brightness within each small region. During the adjustment process, the brightness transition between adjacent small regions is considered to avoid obvious brightness abrupt changes. After the above adaptive adjustment, the overall brightness distribution of the image will be more uniform, achieving the target brightness distribution characteristics, thus obtaining the image after illumination compensation, which can more clearly display the details of the touch screen surface.

[0013] Step 120: Extract point defect features and line defect features from the illumination-compensated image using a multi-branch feature extraction network to obtain a set of point defect features and a set of line defect features; wherein, the set of point defect features includes the contour curvature features and gray-level distribution features of the defect region, and the set of line defect features includes the orientation gradient features and length continuity features of the defect region.

[0014] While illumination compensation makes the image more uniform in brightness, point and linear defects may still be hidden within the complex image information. A multi-branch feature extraction network is specifically designed to accurately extract these two different types of defect features. This network has two independent branches, one for point defects and one for linear defects. Point defects typically appear as relatively isolated small regions, and their contour curvature features reflect the degree of curvature of that region. Different shapes of point defects will have different degrees of contour curvature; by extracting contour curvature features, different types of point defects can be better distinguished. Gray-level distribution features describe the distribution of pixel gray-level values ​​within the defect region. For example, some point defects may have relatively uniform gray-level values, while others may have obvious gray-level variations. By analyzing gray-level distribution features, the characteristics of point defects can be further understood. For linear defects, directional gradient features indicate the direction of defect extension. Linear defects typically extend along a certain defined direction; by extracting directional gradient features, their extension direction can be accurately determined. The length continuity feature is used to describe the degree of continuity of linear defects, that is, whether the defect extends continuously without interruption.

[0015] As one embodiment, step 120 further includes:

[0016] Step 121: Input the illumination-compensated image into the input layer of the multi-branch feature extraction network. The feature channel separation module of the input layer divides the feature channel of the illumination-compensated image into a first feature branch and a second feature branch. The first feature branch is used to extract point defect features, and the second feature branch is used to extract line defect features.

[0017] The input layer of the multi-branch feature extraction network is the starting point of the entire feature extraction process, receiving the illumination-compensated image. The feature channel separation module in the input layer is a key component, responsible for rationally dividing the feature channels of the illumination-compensated image. Image feature channels contain various information about the image, such as color and texture. Based on the characteristics of point defects and linear defects, the feature channel separation module divides these feature channels into two independent branches. The first feature branch is specifically designed to process feature information related to point defects. It receives feature channel data that may be related to point defect features, such as channel information that reflects the shape and grayscale changes of small areas. The second feature branch focuses on processing feature information related to linear defects, receiving feature channel data related to line direction and continuity. This separation of feature channels allows the subsequent feature extraction process to be more targeted, avoiding mutual interference between different types of defect features and improving the accuracy and efficiency of feature extraction.

[0018] Step 122: In the first feature branch, perform multi-scale Gaussian filtering on the illumination-compensated image to generate a set of Gaussian filtered images with different degrees of blur. Perform difference operation on the Gaussian filtered image set to obtain a difference image containing edge enhancement information.

[0019] In the first feature branch, to more effectively extract point defect features, multi-scale Gaussian filtering is required on the illumination-compensated image. The purpose of multi-scale Gaussian filtering is to analyze and process the image at different scales to capture the feature information of point defects at different scales. First, a set of different Gaussian filtering parameters is initialized, which determines the scale and blur level of the Gaussian filter. Different filtering parameters will produce different degrees of blurring effect on the image. For smaller filtering parameters, the Gaussian filter has a smaller range of effect, resulting in less blurring and preserving more detail; while for larger filtering parameters, the Gaussian filter has a larger range of effect, resulting in more blurring and smoothing out some minor noise and interference. Based on these different filtering parameters, Gaussian convolution is performed on the illumination-compensated image. Gaussian convolution is a linear filtering operation that convolves each pixel in the image with a Gaussian kernel, thereby filtering the image. After multiple Gaussian convolution operations with different parameters, a set of Gaussian-filtered images with different degrees of blur is generated, where each image contains image information at different scales. Next, the Gaussian-filtered image set is processed using a difference operation. The difference operation involves subtracting adjacent Gaussian-filtered images pixel by pixel. Since adjacent Gaussian-filtered images are processed at different scales, the differences between them can highlight edge information in the image. For example, the edges of point defects will appear differently in filtered images at different scales. Through the difference operation, this edge information can be enhanced, resulting in a difference image containing edge enhancement information. This edge-enhanced difference image is more beneficial for subsequent contour detection and point defect feature extraction.

[0020] In an optional embodiment, step 122 further includes:

[0021] Step 1221: In the first feature branch, initialize the Gaussian filter parameter set, which contains filter parameters for generating different degrees of blur.

[0022] Before initiating multi-scale Gaussian filtering in the first feature branch, a set of Gaussian filter parameters needs to be initialized. These parameters are determined based on the characteristics of point defects and the properties of the image. Different filter parameters produce different Gaussian filtering effects, thus affecting the final extracted feature information. To comprehensively capture the features of point defects at different scales, multiple different filter parameters need to be set. These parameters can be a set of progressively increasing values ​​to cover a range from smaller to larger scales. Smaller filter parameters are suitable for capturing the detailed features of point defects, such as tiny edges and subtle internal variations. Larger filter parameters are used to smooth out minor noise and interference while highlighting the overall shape features of the point defects. During initialization, a suitable parameter range and interval are determined based on experience and preliminary image analysis. For example, a small initial value can be started, and the parameter values ​​can be gradually increased in steps until a larger upper limit is reached, thus forming a set of Gaussian filter parameters containing multiple different filter parameters.

[0023] Step 1222: Based on each filtering parameter in the Gaussian filtering parameter set, perform Gaussian convolution operation on the illumination-compensated image to generate a Gaussian filtered image that corresponds one-to-one with the filtering parameter; wherein, all Gaussian filtered images together constitute a Gaussian filtered image set.

[0024] After initializing the Gaussian filter parameter set, Gaussian convolution operations can be performed on the illumination-compensated image. For each filter parameter in the Gaussian filter parameter set, a Gaussian convolution operation is performed. Gaussian convolution is a linear filtering operation based on a Gaussian kernel. The Gaussian kernel is a two-dimensional matrix whose element values ​​are calculated according to the Gaussian distribution function. During the convolution operation, the center of the Gaussian kernel is aligned with a pixel in the image, and each element of the Gaussian kernel is multiplied by the corresponding image pixel value. All products are then summed to obtain the new value of the pixel. This process is repeated for each pixel in the image, thus filtering the entire image. Different filter parameters correspond to different Gaussian kernels. The larger the filter parameter, the larger the size of the Gaussian kernel, the wider its range of action, and the greater the degree of image blurring. By performing multiple Gaussian convolution operations with different parameters on the illumination-compensated image, a set of Gaussian-filtered images corresponding one-to-one with the filter parameters is generated. These images have varying degrees of blurring, reflecting information about the image at different scales.

[0025] Step 1223: Perform pixel-by-pixel difference operations on adjacent Gaussian filtered images in the Gaussian filtered image set according to the increasing order of the filtering parameters to generate a difference image set. Each difference image in the difference image set corresponds to the difference result of a set of adjacent Gaussian filtered images.

[0026] After obtaining the Gaussian-filtered image set, it needs to be processed by difference operations. To ensure the rationality and effectiveness of the difference operations, they are performed in ascending order of the filter parameters. This is because the Gaussian-filtered images corresponding to adjacent filter parameters gradually change in scale, and the differences between them can better reflect the edge information in the image. Pixel-by-pixel difference operation involves subtracting the pixel values ​​at corresponding positions in two adjacent Gaussian-filtered images. For example, for a pixel in the first Gaussian-filtered image, its pixel value is subtracted from the corresponding pixel value in the second Gaussian-filtered image, resulting in a difference value, which is the pixel value at the corresponding position in the difference image. Performing the above pixel-by-pixel difference operation on all adjacent images in the Gaussian-filtered image set generates a set of difference images, which together constitute the difference image set. Each difference image corresponds to a set of difference results from adjacent Gaussian-filtered images, highlighting the changes in the image at different scales, especially the edge information of point defects. Through difference operations, the edges of point defects can be separated from the complex image background.

[0027] Step 1224: Perform pixel value normalization processing on all differential images in the differential image set so that the pixel value of each differential image is within a preset grayscale value range.

[0028] After the difference operation, the pixel values ​​of the difference images in a set of difference images may be quite dispersed. Some pixel values ​​may be very large, while others may be very small. This dispersion of pixel values ​​is detrimental to subsequent processing and analysis. Therefore, pixel value normalization is required for all difference images in the set. The purpose of normalization is to unify the pixel values ​​of the difference images into a preset grayscale range. The preset grayscale range is usually a fixed interval, such as the common 0 to 255. During normalization, the maximum and minimum pixel values ​​in the difference images are first identified. Then, based on these two values ​​and the preset grayscale range, a normalization coefficient is calculated. Using this normalization coefficient, each pixel value in the difference image is linearly transformed, ensuring that the transformed pixel values ​​all fall within the preset grayscale range.

[0029] Step 1225: Input the normalized difference image set into the edge enhancement module, and perform edge enhancement processing on each difference image using the Laplacian operator to generate a target difference image containing edge enhancement information.

[0030] While the normalized difference images are more standardized in pixel values, their edge information may still be insufficiently prominent. To further enhance the edges of point defects, the normalized difference image set is input into the edge enhancement module. The edge enhancement module processes each difference image using the Laplacian operator. The Laplacian operator is a second-order differential operator, highly sensitive to grayscale changes in an image. When the Laplacian operator is applied to the difference images for convolution, edge regions in the image exhibit a larger response because the grayscale changes in edge regions are more drastic, and the Laplacian operator can detect the second derivative of these grayscale changes. During the convolution operation, the Laplacian operator calculates for each pixel and its neighboring pixels in the difference image. For pixels near edges, the calculation result is larger, while for pixels outside the edge region, the result is relatively smaller. In this way, the edge information in the difference images is highlighted, making the edges of point defects clearer. After processing with the Laplacian operator, each difference image receives edge enhancement, generating a target difference image containing edge enhancement information.

[0031] Step 1226: Perform connected component labeling on the target difference image, remove connected components with an area smaller than a preset threshold, and obtain a purified edge-enhanced difference image.

[0032] While edge information is enhanced in the target difference image, some noise and small interference regions may still exist, which could affect subsequent contour detection and feature extraction. Therefore, connected component labeling is necessary for the target difference image. Connected component labeling is an image analysis technique that labels interconnected pixel regions in the target difference image as connected regions. By traversing each pixel in the target difference image, they are divided into different connected regions based on the connectivity between pixels. Each connected region has a unique label. After obtaining all connected regions, the area of ​​each connected region is calculated. Some connected regions may be caused by noise or other interference, and their areas are usually relatively small. To remove these unnecessary interference regions, a preset threshold is set. Connected regions with areas smaller than the preset threshold are deleted from the target difference image, thus obtaining the cleaned edge-enhanced difference image. The cleaned edge-enhanced difference image retains only the larger connected regions, which are more likely to be real point-like defect areas.

[0033] Step 123: Perform contour detection processing on the difference image, extract the contour curvature features of the closed region in the difference image, calculate the gray value distribution curve of the pixels in the closed region as the gray value distribution feature, and combine the contour curvature feature and the gray value distribution feature to form a set of point defect features.

[0034] The cleaned edge-enhanced difference image contains edge information of point defects, and contour detection can further extract the contours of these defects. Contour detection searches for closed regions in the difference image, as point defects typically appear as relatively closed small areas. After finding a closed region, its contour is precisely located and tracked. For each closed region, its contour curvature features are extracted. Contour curvature features reflect the degree of curvature of the closed region's contour. To calculate the contour curvature features, each point on the contour line of the closed region is analyzed. By calculating the slope change between adjacent points on the contour line, the curvature value of each point can be obtained. These curvature values ​​are grouped into a sequence, and then statistical analysis is performed on this sequence. For example, statistics such as the maximum value, average value, and variance of the curvature are calculated; these statistics constitute the contour curvature features. Simultaneously, for each closed region, the grayscale value distribution curve of its internal pixels is also calculated. The grayscale value distribution curve describes the distribution of pixel grayscale values ​​within the closed region. By collecting the grayscale values ​​of all pixels within a closed region, these grayscale values ​​are statistically analyzed, such as calculating the frequency of each grayscale value, and then a grayscale value distribution curve is plotted. Finally, the contour curvature features and grayscale distribution features corresponding to each closed region are combined to form a feature vector for each potential point defect region. These feature vectors together constitute a set of point defect features.

[0035] Under one design approach, step 123 further includes:

[0036] Step 1231: Perform binarization processing on the difference image to generate a binarized image containing the target region and the background region.

[0037] Although the difference image has undergone edge enhancement and cleansing, it still contains complex grayscale information. To facilitate contour detection and feature extraction, the difference image needs to be binarized. Binarization transforms the pixel values ​​in the difference image into an image with only two possible values: the target region and the background region. A threshold is set during binarization. For each pixel in the difference image, if its pixel value is greater than the threshold, it is marked as a pixel in the target region, and the pixel value is set to a fixed value (e.g., 255); if its pixel value is less than or equal to the threshold, it is marked as a pixel in the background region, and the pixel value is set to another fixed value (e.g., 0). In this way, the difference image is converted into a binary image containing only the target and background regions. The target region typically corresponds to possible point-like defect areas, while the background region represents the rest of the image. Binarization simplifies the image information, making subsequent contour detection and analysis easier.

[0038] Step 1232: Perform contour extraction processing on the binarized image to identify all closed contour lines, with each closed contour line corresponding to a potential point-like defect region.

[0039] The target and background regions have been clearly distinguished in the binarized image. The next step is to perform contour extraction on the binarized image. The purpose of contour extraction is to find all closed contour lines in the binarized image. A closed contour line typically represents a relatively independent region; in this scenario, each closed contour line corresponds to a potential point defect region. A contour tracking algorithm is used for contour extraction. This algorithm starts from the edge of the binarized image and traverses along the boundary of the target region. When a pixel of a target region is encountered, it continues to track adjacent target region pixels according to certain rules (e.g., clockwise or counterclockwise) until it returns to the starting point, forming a closed contour line. By continuously traversing the binarized image, all closed contour lines can be identified. These closed contour lines provide the foundation for subsequent extraction of the contour curvature features and grayscale distribution features of point defects.

[0040] Step 1233: For each closed contour line, a curve fitting algorithm is used to perform polynomial fitting on the pixels on the contour line to generate a contour fitting curve.

[0041] After obtaining all closed contour lines, polynomial fitting is required for each pixel on the closed contour line to more accurately calculate the contour curvature features. The curve fitting algorithm finds a suitable polynomial function to approximate the distribution of these pixels based on their coordinates. The purpose of polynomial fitting is to connect discrete pixels into a continuous curve, i.e., the contour fitting curve. During polynomial fitting, an appropriate polynomial order is chosen. Too low an order may fail to accurately fit the shape of the contour line; too high an order may lead to overfitting, making the fitted curve overly complex. By continuously adjusting the polynomial coefficients, the polynomial function is made as close as possible to the pixels on the closed contour line. After polynomial fitting, the resulting contour fitting curve can more smoothly represent the shape of the closed contour line.

[0042] Step 1234: Based on the contour fitting curve, calculate the curvature value of each point on the contour fitting curve, generate a curvature value sequence, perform statistical analysis on the curvature value sequence, and extract the maximum curvature value, the average curvature value, and the curvature variance as contour curvature features.

[0043] Once the profile fitting curve is obtained, the curvature value at each point on the curve can be calculated. The curvature value reflects the degree of curvature of the curve at that point. Calculating the curvature value involves operations such as differentiation of the profile fitting curve based on the mathematical properties of the curve. For each point on the curve, the curvature value is obtained by calculating the rate of change of its tangent direction. The curvature values ​​of all points are then compiled into a curvature value sequence. To better describe the overall curvature characteristics of the closed profile, statistical analysis of the curvature value sequence is required. Statistical analysis calculates the maximum value, average value, and variance of the curvature value sequence. The maximum curvature value represents the curvature value at the point with the greatest curvature on the profile, reflecting the maximum degree of curvature. The average curvature value represents the average degree of curvature of the profile. The curvature variance reflects the dispersion of the curvature values ​​across the entire profile, i.e., whether the curvature of the profile is uniform. These three statistical quantities—maximum curvature, average curvature, and curvature variance—together constitute the profile curvature characteristics.

[0044] Step 1235: Within the area enclosed by each closed contour line, collect the gray values ​​of all pixels to generate a gray value sequence. Perform probability density estimation processing on the gray value sequence to generate a gray value distribution curve as a gray value distribution feature.

[0045] Besides contour curvature features, grayscale distribution is also a crucial characteristic of point defects. Within each closed contour line, the grayscale values ​​of all pixels are collected, forming a grayscale value sequence. This sequence reflects the specific grayscale values ​​of the pixels within the closed region. To further analyze the distribution patterns, probability density estimation is performed on the grayscale value sequence. This estimation calculates the probability of each grayscale value occurring based on the data in the sequence. Statistical analysis of the grayscale value sequence is then performed to calculate the frequency of each grayscale value, and a grayscale distribution curve is plotted based on these frequencies. This curve visually illustrates the distribution of pixel grayscale values ​​within the closed region. Different types of point defects may have different grayscale distribution curves; for example, some defects may have concentrated grayscale values, while others may have more dispersed distributions. Analyzing the grayscale distribution curve allows for better differentiation between different types of point defects, thus it is considered a grayscale distribution feature.

[0046] Step 1236: Align the contour curvature features and grayscale distribution features corresponding to each closed contour line according to the feature dimensions, and then concatenate the aligned contour curvature features and grayscale distribution features according to the channel dimension to form the feature vector of each potential point defect region; wherein, all feature vectors together constitute the point defect feature set.

[0047] After obtaining the contour curvature features and grayscale distribution features corresponding to each closed contour line, feature dimension alignment is required because these two features may have different dimensions. Feature dimension alignment ensures that the contour curvature features and grayscale distribution features can be concatenated along the same dimension. During alignment, features are adjusted as needed, such as through interpolation or dimensionality reduction, to ensure consistent dimensions. After feature dimension alignment, the aligned contour curvature features and grayscale distribution features are concatenated along the channel dimension. Channel dimension concatenation merges two different information channels into a richer information set. This concatenation method combines the contour curvature features and grayscale distribution features into a single feature vector. Each potential point defect region has a corresponding feature vector, and all these feature vectors together constitute the point defect feature set, which contains the shape and grayscale information of all potential point defect regions.

[0048] Step 124: In the second feature branch, the directional gradient histogram is calculated on the illumination-compensated image to generate gradient intensity distribution maps in different directions. Non-maximum suppression is performed on the gradient intensity distribution maps to obtain gradient feature maps with directional consistency.

[0049] The primary objective of the second feature branch is to extract the features of linear defects. Linear defects in the image after illumination compensation may be hidden within complex image information, requiring a series of processing steps to highlight their features. Histogram of Oriented Gradients (HOGs) calculation is a crucial step in extracting these features. This process calculates the gradient information for each pixel in the image, including gradient magnitude and gradient direction. The gradient magnitude represents the degree of grayscale change of a pixel in a certain direction, while the gradient direction indicates the direction of the greatest grayscale change. The gradient directions of the image are divided into multiple directional intervals, and the sum of the gradient magnitudes within each interval is calculated to generate an HOG histogram. Based on the HOG histogram, gradient intensity distribution maps for different directions can be constructed. Each gradient intensity distribution map corresponds to the gradient magnitude distribution of a directional interval, visually demonstrating the gradient intensity changes in different directions. However, the gradient intensity distribution map may contain gradient information that is not representative of true linear defects, such as gradients generated by noise. To remove this interference, non-maximum suppression processing is required on the gradient intensity distribution map. Non-maximum suppression (NMS) retains the pixel with the largest gradient magnitude in each gradient direction while suppressing the gradient magnitudes of other non-maximum points. This method highlights the true edges of linear defects, resulting in a gradient feature map with consistent orientation. This gradient feature map retains only the local maxima in the gradient direction, making the features of linear defects clearer and more prominent.

[0050] In an alternative embodiment, step 124 further includes:

[0051] Step 1241: In the second feature branch, the image after illumination compensation is converted to grayscale to generate a single-channel grayscale image.

[0052] The image after illumination compensation may be a multi-channel color image. While color images contain rich color information, this information is not essential for extracting the directional gradient features of linear defects and may even increase processing complexity. Therefore, in the second feature branch, the image after illumination compensation is first converted to grayscale. Grayscale conversion transforms the multi-channel color image into a single-channel grayscale image. A grayscale image contains only the grayscale information of the image, i.e., the brightness value of each pixel. During grayscale conversion, the corresponding grayscale value is calculated based on the pixel values ​​of each channel (e.g., red, green, and blue channels) in the color image, according to certain weights. Different weighting methods may yield different grayscale effects, but factors such as the human eye's sensitivity to different colors are usually considered. After grayscale conversion, the generated single-channel grayscale image retains the main brightness information of the image while simplifying the image's data structure.

[0053] Step 1242: Perform Sobel convolution on the grayscale image in the horizontal and vertical directions to generate a horizontal gradient image and a vertical gradient image.

[0054] After obtaining a single-channel grayscale image, to calculate the gradient information of each pixel, the grayscale image needs to be convolved with the Sobel operator in both the horizontal and vertical directions. The Sobel operator is a commonly used edge detection operator that can detect grayscale changes in an image. In the horizontal direction, the Sobel operator performs a convolution operation on the grayscale image, calculating the rate of grayscale change of each pixel in the horizontal direction. By convolving the Sobel operator with each pixel and its neighboring pixels in the grayscale image, the gradient values ​​in the horizontal direction are obtained. These horizontal gradient values ​​are then combined into a matrix to generate the horizontal gradient image. The horizontal gradient image reflects the grayscale changes in the grayscale image in the horizontal direction. Similarly, in the vertical direction, the Sobel operator is used to convolve the grayscale image, calculating the rate of grayscale change of each pixel in the vertical direction. The resulting vertical gradient values ​​are then combined into a matrix to generate the vertical gradient image. The vertical gradient image reflects the grayscale changes in the grayscale image in the vertical direction.

[0055] Step 1243: Based on the horizontal gradient image and the vertical gradient image, calculate the gradient magnitude and gradient direction angle of each pixel, and generate the gradient magnitude matrix and gradient direction angle matrix.

[0056] With the horizontal and vertical gradient images, we can calculate the gradient magnitude and gradient direction angle for each pixel. The gradient magnitude represents the degree of grayscale change of a pixel in a certain direction, reflecting the drastic degree of grayscale change around that point. The gradient direction angle indicates the direction of the greatest grayscale change. For each corresponding pixel in the horizontal and vertical gradient images, the gradient magnitude and gradient direction angle of that pixel are obtained through certain mathematical calculations based on its horizontal and vertical gradient values. The gradient magnitude matrices of all pixels are then combined into a matrix to obtain the gradient magnitude matrix. The gradient magnitude matrix visually displays the gradient magnitude distribution of each pixel in the image. Similarly, the gradient direction angle matrices of all pixels are then combined into a matrix to obtain the gradient direction angle matrix.

[0057] Step 1244: Divide the gradient direction angle matrix into a preset number of direction intervals, and accumulate the gradient magnitude in each direction interval based on the gradient magnitude matrix to generate a directional gradient histogram.

[0058] To better analyze the directional distribution of gradients in an image, the gradient orientation matrix is ​​divided into a preset number of directional intervals. This preset number of intervals is set based on actual needs and experience; for example, the gradient orientation matrix can be divided into 8 or 16 intervals. Each interval corresponds to a defined angle range. After dividing the intervals, the gradient magnitudes within each interval are accumulated based on the gradient magnitude matrix. For each pixel in the gradient orientation matrix, its corresponding gradient magnitude is accumulated into the statistical value of that interval, according to the interval to which its gradient orientation belongs. By traversing all pixels in both the gradient magnitude matrix and the gradient orientation matrix, the accumulation of gradient magnitudes within each interval is completed. The accumulated results for each interval are then combined into a sequence, which forms the directional gradient histogram.

[0059] Step 1245: Based on the directional gradient histogram, construct gradient intensity distribution maps for different directions, with each gradient intensity distribution map corresponding to the gradient magnitude distribution in a directional interval.

[0060] The Histogram of Oriented Gradients (HGP) shows the distribution of gradients in an image across different directions, but it is only a statistical result. To more intuitively display the gradient magnitude distribution within each directional interval, gradient intensity distribution maps are constructed based on the HGP. For each directional interval, a corresponding gradient intensity distribution map is created. When constructing the gradient intensity distribution map, the value of each pixel in the map is determined based on the statistical values ​​of that directional interval in the HGP and the gradient magnitude values ​​of the pixels within that interval in the gradient magnitude matrix. For example, for a given directional interval, the gradient magnitude values ​​of the pixels within that interval in the gradient magnitude matrix are mapped to the gradient intensity distribution map according to certain rules. In this way, each gradient intensity distribution map corresponds to the gradient magnitude distribution within a specific directional interval, intuitively showing the changes in gradient intensity in different directions of the image.

[0061] Step 1246: Perform non-maximum suppression processing on each gradient intensity distribution map, retaining local maximum points in the gradient direction and suppressing the gradient magnitude of non-maximum points to generate a preliminary gradient feature map.

[0062] While the gradient intensity distribution map shows the distribution of gradient magnitudes in different directions, it may contain gradient information that is not representative of true linear defects, such as gradients generated by noise. To remove this interference, non-maximum suppression (NMS) is applied to each gradient intensity distribution map. NMS aims to preserve local maxima along the gradient direction while suppressing gradient magnitudes at other non-maximum points. During processing, for each pixel, the gradient magnitudes of its neighboring pixels along the gradient direction are checked. If the gradient magnitude of a pixel is not the maximum among its neighboring pixels along the gradient direction, its gradient magnitude is suppressed to zero. Only when the gradient magnitude of a pixel is a local maximum along its gradient direction is its gradient magnitude preserved. By performing the above processing on each gradient intensity distribution map, a preliminary gradient feature map is obtained. The preliminary gradient feature map only preserves local maxima along the gradient direction, making the edges of true linear defects more prominent and reducing the influence of noise and interference.

[0063] Step 1247: Threshold the preliminary gradient feature map to remove pixels with gradient magnitudes lower than a preset threshold, and obtain a gradient feature map with consistent orientation.

[0064] Although some non-maximum gradient magnitudes have been removed from the initial gradient feature map, some pixels with small gradient magnitudes may still remain. These pixels may be caused by noise or other weak interference and are not true linear defect edges. To further refine the initial gradient feature map, thresholding is performed. Thresholding sets a preset threshold. For each pixel in the initial gradient feature map, if its gradient magnitude is lower than the preset threshold, its gradient magnitude is set to zero, i.e., the pixel is removed. Only pixels with gradient magnitudes higher than the preset threshold are retained. In this way, interference points with small gradient magnitudes can be removed, resulting in a gradient feature map with consistent orientation. A gradient feature map with consistent orientation retains only pixels with large gradient magnitudes. These pixels are more likely to be true linear defect edges, making the features of linear defects clearer and more prominent.

[0065] Step 1248: Merge the gradient feature maps of different directions to generate multi-channel gradient feature maps, with each channel corresponding to the gradient features of a directional interval.

[0066] After thresholding, gradient feature maps with directional consistency are obtained in different directions. Each gradient feature map corresponds to the gradient features of a specific directional interval. To integrate these gradient features from different directions for easier subsequent processing and analysis, channel merging is performed on the gradient feature maps from different directions. Channel merging treats each gradient feature map as a channel and combines them into a multi-channel gradient feature map. During the merging process, the gradient feature maps from different directions are arranged in a specific order to form a new image structure. Each channel corresponds to the gradient features of a specific directional interval, thus the multi-channel gradient feature map contains gradient information of the image in multiple directions. The multi-channel gradient feature map provides comprehensive gradient data for the subsequent extraction of directional gradient features and length continuity features of linear defects, enabling more accurate analysis and identification of linear defects.

[0067] Step 125: Perform connected component analysis on the gradient feature map, extract the directional gradient features and length continuity features of continuous gradient regions, and combine the directional gradient features and length continuity features to form a linear defect feature set.

[0068] Multi-channel gradient feature maps contain gradient information of linear defects, but further extraction of directional gradient features and length continuity features is needed. This is achieved through connected component analysis. Connected component analysis identifies interconnected pixel regions in the gradient feature map and marks these regions as connected regions. Each connected region may correspond to a potential linear defect. For each connected region, its directional gradient features are extracted. Directional gradient features reflect the extension direction of the linear defect. Within a connected region, the average gradient direction of the region is calculated based on the gradient direction information of the pixels, serving as the directional gradient feature. Simultaneously, length continuity features are extracted. Length continuity features describe the continuity of the linear defect. By analyzing the shape and pixel distribution of the connected regions, the length and continuity index of the connected regions are calculated. For example, determining whether the connected region extends continuously and uninterruptedly, and whether the extension length reaches a certain threshold. The directional gradient features and length continuity features of each connected region are combined to form the feature vector of each potential linear defect region. All these feature vectors together constitute the linear defect feature set.

[0069] Step 126: Input the point defect feature set output by the first feature branch and the line defect feature set output by the second feature branch into the feature alignment module of the multi-branch feature extraction network for spatial position calibration processing to generate a point defect feature set and a line defect feature set with a unified coordinate reference system.

[0070] The point defect feature set output by the first feature branch and the line defect feature set output by the second feature branch contain feature information of point and line defects, respectively. However, they may be inconsistent in spatial location because the two branches may have used different coordinate systems or processing methods during processing. To ensure accurate subsequent defect classification and detection, these two feature sets need to undergo spatial location calibration. The feature alignment module of the multi-branch feature extraction network is responsible for this task. The feature alignment module adjusts the features in the point defect feature set and the line defect feature set according to certain calibration rules, so that they have a unified coordinate reference system in spatial location. For example, by finding some feature points or reference regions in the image, the coordinates of the two feature sets are mapped and transformed so that they are represented in the same coordinate system. After spatial location calibration, point defect feature sets and line defect feature sets with a unified coordinate reference system are generated. In subsequent processing, the features of point defects and line defects can be compared and analyzed more accurately, improving the accuracy of defect detection.

[0071] Step 130: Perform feature decoupling processing on the set of point defects and the set of line defects to obtain decoupled independent point defects and decoupled independent line defects; wherein, the feature decoupling processing is used to suppress cross-interference between different types of defect features.

[0072] The feature sets for point defects and linear defects may contain cross-interferences between different types of defect features. For example, in some cases, the features of point defects may affect the feature extraction of linear defects, and vice versa. Such cross-interferences can affect the accuracy of subsequent defect classification and detection, thus requiring feature decoupling. The purpose of feature decoupling is to separate point defect features from linear defect features, remove their mutual interference, and obtain decoupled independent point defect features and decoupled independent linear defect features. During the decoupling process, the correlation between the various feature components in the feature set is analyzed, identifying those feature components that cause cross-interference, and removing or adjusting them. Through feature decoupling, point defect features and linear defect features can be made more independent and pure.

[0073] As a design approach, step 130 includes:

[0074] Step 131: Construct a feature correlation matrix, which is used to represent the correlation coefficient between each feature component in the point defect feature set and the line defect feature set.

[0075] To perform feature decoupling, a feature correlation matrix must first be constructed. This matrix reflects the correlation between the various feature components in both the point defect feature set and the line defect feature set. During construction, each feature component in both sets is compared pairwise. For each pair, a correlation coefficient is calculated. The correlation coefficient indicates the degree of correlation between the two feature components; a higher value indicates a stronger correlation, and a lower value indicates a weaker correlation. By calculating the correlation coefficients between all feature component pairs, these coefficients are combined into a matrix—the feature correlation matrix. The rows and columns of the feature correlation matrix correspond to all feature components in the point defect feature set and the line defect feature set, respectively. Analyzing the feature correlation matrix provides a clear understanding of the correlation between each feature component.

[0076] In one embodiment, step 131 further includes:

[0077] Step 1310: Convert the point defect feature set and the linear defect feature set into feature matrix form; calculate the Pearson correlation coefficient between every two feature components in the feature matrix to generate a correlation coefficient matrix, wherein the rows and columns of the correlation coefficient matrix correspond to all feature components in the point defect feature set and the linear defect feature set, respectively; adjust the correlation coefficient matrix so that the matrix element values ​​are within a preset range to obtain an adjusted correlation coefficient matrix; construct a feature association matrix based on the adjusted correlation coefficient matrix, wherein each element of the feature association matrix represents the correlation strength between two corresponding feature components.

[0078] To construct the feature correlation matrix, the point defect feature sets and linear defect feature sets are first converted into feature matrix form. A feature matrix is ​​a matrix structure that arranges the feature components in the feature sets into rows and columns according to certain rules, facilitating subsequent calculations and processing. After obtaining the feature matrix, the Pearson correlation coefficient between every two feature components is calculated. The Pearson correlation coefficient is a commonly used indicator to measure the linear correlation between two variables. By calculating the Pearson correlation coefficient for each pair of feature components in the feature matrix, these correlation coefficients are combined into a matrix, namely the correlation coefficient matrix. The rows and columns of the correlation coefficient matrix correspond to all feature components in the point defect feature set and the linear defect feature set, respectively. The element values ​​in the correlation coefficient matrix may be distributed over a wide range. To facilitate subsequent processing and analysis, the correlation coefficient matrix needs to be adjusted. The adjustment will bring the matrix element values ​​within a preset range. The preset range is set according to actual needs and experience; for example, the element values ​​can be adjusted to be between 0 and 1. After adjustment, the adjusted correlation coefficient matrix is ​​obtained. Finally, the feature correlation matrix is ​​constructed based on the adjusted correlation coefficient matrix. Each element of the feature correlation matrix represents the correlation strength between two corresponding feature components.

[0079] Step 132: Perform eigenvalue decomposition on the feature correlation matrix to obtain eigenvalues ​​and corresponding eigenvectors, and determine the main correlation feature directions based on the magnitude of the eigenvalues.

[0080] After obtaining the feature correlation matrix, eigenvalue decomposition is performed to further analyze the relationships between features. Eigenvalue decomposition is a matrix analysis method that decomposes the feature correlation matrix into eigenvalues ​​and corresponding eigenvectors. Eigenvalues ​​represent the scaling factor of the matrix along a specific feature direction, while eigenvectors represent that feature direction. Eigenvalue decomposition yields a set of eigenvalues ​​and corresponding eigenvectors. The magnitude of these eigenvalues ​​reflects the importance of different feature directions. Larger eigenvalues ​​correspond to features with stronger correlations, while smaller eigenvalues ​​correspond to features with weaker correlations. Based on the magnitude of the eigenvalues, the primary correlation feature directions are determined. The primary correlation feature direction is the direction corresponding to the eigenvector with the largest eigenvalue, representing the most significant correlation between features. By determining the primary correlation feature directions, we can better understand the main correlation patterns between the various feature components in the feature set.

[0081] Step 133: Construct a feature projection matrix based on the main associated feature directions, and project the point defect feature set and the line defect feature set onto mutually orthogonal feature subspaces based on the feature projection matrix.

[0082] After determining the main associated feature directions, a feature projection matrix needs to be constructed to achieve feature decoupling. The function of the feature projection matrix is ​​to project the set of point defects and the set of line defects into mutually orthogonal feature subspaces. Mutually orthogonal feature subspaces mean that the features in different subspaces are independent of each other, thus effectively separating point defects from line defects. Based on the main associated feature directions, two projection matrices are constructed, one parallel and one perpendicular to the main associated feature directions, respectively. When constructing the projection matrices, the element values ​​are determined based on the eigenvectors of the main associated feature directions. Through the action of the projection matrices, the set of point defects and the set of line defects are projected separately. The set of point defects is projected into two mutually orthogonal subspaces: one subspace contains feature components related to the main associated feature directions, and the other contains feature components perpendicular to the main associated feature directions. Similarly, the set of line defects is also projected into these two mutually orthogonal subspaces. Thus, point defects and line defects can be separated into different subspaces.

[0083] In the following steps, step 133 includes:

[0084] Step 1331: Based on the feature vectors obtained from the eigenvalue decomposition process, select the feature vector corresponding to the largest eigenvalue as the main associated feature direction.

[0085] After performing eigenvalue decomposition on the feature correlation matrix, a set of eigenvalues ​​and corresponding eigenvectors are obtained. To determine the primary correlation feature directions, it is necessary to select from these eigenvectors. Since the feature direction corresponding to the largest eigenvalue indicates the strongest correlation between features, the eigenvector corresponding to the largest eigenvalue is selected as the primary correlation feature direction. This eigenvector represents the most significant correlation pattern among the feature components in the feature set. By determining the primary correlation feature direction, the main correlation direction between features can be clearly identified.

[0086] Step 1332: Based on the main associated feature direction, construct a first projection matrix and a second projection matrix. The first projection matrix is ​​used to project the feature to a subspace parallel to the main associated feature direction, and the second projection matrix is ​​used to project the feature to a subspace perpendicular to the main associated feature direction.

[0087] After determining the main associated feature directions, two projection matrices need to be constructed: a first projection matrix and a second projection matrix. The first projection matrix projects the features onto a subspace parallel to the main associated feature directions. When constructing the first projection matrix, the element values ​​are determined based on the eigenvectors of the main associated feature directions. These element values ​​ensure that the projection operation accurately projects the features onto the subspace parallel to the main associated feature directions. The second projection matrix projects the features onto a subspace perpendicular to the main associated feature directions. Similarly, when constructing the second projection matrix, the element values ​​are determined based on the eigenvectors of the main associated feature directions and the geometric relationship of the perpendicular direction. Using these two projection matrices, the set of point-like defect features and the set of line-like defect features can be projected into two mutually orthogonal subspaces, achieving feature separation and initial decoupling.

[0088] Step 1333: Input the point defect feature set into the first projection matrix to obtain the projection components of the point defect feature set in the main associated direction, and input the point defect feature set into the second projection matrix to obtain the projection components of the point defect feature set in the vertical direction.

[0089] After constructing the first and second projection matrices, the point defect feature set is input into these two matrices respectively. When the point defect feature set is input into the first projection matrix, it projects the feature set into a subspace parallel to the main associated feature direction, obtaining the projection component of the feature set in the main associated direction. This projection component contains feature information related to the main associated feature direction within the point defect feature set. Similarly, when the point defect feature set is input into the second projection matrix, it projects the feature set into a subspace perpendicular to the main associated feature direction, obtaining the projection component of the feature set in the vertical direction. This vertical projection component contains feature information unrelated to the main associated feature direction within the point defect feature set.

[0090] Step 1334: Input the linear defect feature set into the first projection matrix to obtain the projection components of the linear defect feature set in the main associated direction, and input the linear defect feature set into the second projection matrix to obtain the projection components of the linear defect feature set in the vertical direction.

[0091] Similar to the point-like defect feature set, the linear defect feature set is also input into the first projection matrix and the second projection matrix, respectively. When the linear defect feature set is input into the first projection matrix, the first projection matrix projects the linear defect feature set into a subspace parallel to the main associated feature direction, obtaining the projection component of the linear defect feature set in the main associated direction. This projection component contains feature information in the linear defect feature set related to the main associated feature direction. When the linear defect feature set is input into the second projection matrix, the second projection matrix projects the linear defect feature set into a subspace perpendicular to the main associated feature direction, obtaining the projection component of the linear defect feature set in the vertical direction. The projection component in the vertical direction contains feature information in the linear defect feature set unrelated to the main associated feature direction.

[0092] Step 1335: Compare and analyze the projection components of the point defect feature set in the main associated direction with the projection components of the line defect feature set in the main associated direction to determine the cross-interference feature components.

[0093] After obtaining the projection components of the point defect feature set and the linear defect feature set along the main associated directions, these two projection components need to be compared and analyzed. The purpose of the comparative analysis is to identify the cross-interference feature components, that is, those feature components that simultaneously affect both point defect features and linear defect features. During the comparison, feature elements at the same position in the two projection components are examined. If some feature elements have large values ​​in both projection components, it indicates that these feature elements may be cross-interference feature components. For example, some features may be related to a feature of both point defects and a feature of both linear defects, thus causing cross-interference between features. Through comparative analysis, these cross-interference feature components can be identified.

[0094] Step 1336: Adjust the projection directions of the first projection matrix and the second projection matrix according to the cross-interference feature components. The adjusted projection matrix is ​​used to enhance the separation of cross-interference feature components.

[0095] After determining the cross-interference feature components, to more effectively separate point-like and line-like defect features, the projection directions of the first and second projection matrices need to be adjusted according to these components. This adjustment makes the projection matrices more focused on separating the cross-interference feature components. During the adjustment process, the element values ​​of the projection matrices are modified based on the characteristics and distribution of the cross-interference feature components. For example, if a cross-interference feature component has a large projection component in the main associated direction, adjusting the projection matrix will cause this feature component to be allocated more to the vertical subspace during projection, thereby reducing its interference with other features. By adjusting the projection direction of the projection matrices, the ability to separate the cross-interference feature components is strengthened, allowing point-like and line-like defect features to be extracted more independently.

[0096] Step 1337: Using the adjusted projection matrix, project the point defect feature set and the line defect feature set onto the new orthogonal feature subspace again.

[0097] After adjusting the projection matrix, the point defect feature sets and line defect feature sets are projected again into a new orthogonal feature subspace using the adjusted projection matrix. This new orthogonal feature subspace is generated based on the adjusted projection matrix and can better separate point and line defect features. During the reprojection process, the point and line defect feature sets are redistributed to different subspaces. Because the projection matrix has been adjusted, the cross-interference feature components are more effectively separated into different subspaces, making the point and line defect features more independent and pure in the new subspace. This reprojection further improves the feature decoupling effect.

[0098] Step 134: In the point defect feature subspace, remove the feature components containing linear defect features and retain the components containing point defect features to form decoupled independent point defect features.

[0099] After reprojection, some feature components containing linear defect features may still remain in the point defect feature subspace. To obtain decoupled independent point defect features, these feature components containing linear defect features need to be removed. In the point defect feature subspace, each feature component is analyzed and evaluated. Based on the previously determined correlation between interfering feature components and features, those feature components containing linear defect features are identified. These feature components are then deleted from the point defect feature subspace. Only those components that truly belong to point defect features are retained. In this way, decoupled independent point defect features are obtained. The decoupled independent point defect features contain only the feature information of point defects and are unaffected by linear defect features.

[0100] Step 135: In the linear defect feature subspace, remove the feature components containing point defect features and retain the components containing linear defect features to form decoupled independent linear defect features.

[0101] Similarly, within the linear defect feature subspace, there may be feature components that contain elements of point-like defects. To obtain the decoupled independent linear defect features, these feature components containing point-like defect elements need to be removed. In the linear defect feature subspace, each feature component undergoes detailed analysis and evaluation. Based on the previously determined correlations between interfering feature components and features, those feature components containing point-like defect elements are identified. These feature components are then deleted from the linear defect feature subspace, retaining only those components that truly belong to the linear defect features. In this way, the decoupled independent linear defect features are obtained. The decoupled independent linear defect features contain only the feature information of linear defects and are unaffected by point-like defect features.

[0102] Step 136: Using principal component analysis algorithm, perform feature dimension compression processing on the decoupled independent point defect features and the decoupled independent line defect features, retaining the main feature components and removing redundant feature components.

[0103] Decoupled independent point-like defect features and decoupled independent linear defect features may contain numerous feature components, some of which may be redundant, meaning they contribute little to defect classification and identification. To reduce data dimensionality, improve processing efficiency, and retain important feature information, principal component analysis (PCA) is used to compress the feature dimensions of the decoupled independent point-like and independent linear defect features. PCA analyzes the variance distribution of the feature data, identifying feature directions with large variances, which represent the most important information in the data. Feature dimension compression is achieved by projecting the feature data onto these principal feature directions. During compression, the main feature components, i.e., those that contribute significantly to defect classification and identification, are retained. Simultaneously, redundant feature components, i.e., those with small variances and little impact on classification and identification, are removed. Through PCA processing, dimensionality-compressed independent point-like and independent linear defect features are obtained, making the feature data more concise and effective.

[0104] Step 137: Calculate the cross-correlation coefficient between the decoupled independent point defect features and independent line defect features. If the cross-correlation coefficient is higher than the preset correlation coefficient, readjust the feature projection matrix and perform projection and decoupling processing again until the cross-correlation coefficient is lower than the preset correlation coefficient.

[0105] To verify the effectiveness of feature decoupling, the cross-correlation coefficient between the decoupled independent point defect features and independent linear defect features is calculated. The cross-correlation coefficient represents the degree of association between these two types of features. If the cross-correlation coefficient is higher than a preset correlation coefficient, it indicates that the feature decoupling is not thorough enough, and there is still a certain degree of cross-interference. The preset correlation coefficient is a threshold set based on actual needs and experience, representing the maximum acceptable degree of association between features. When the cross-correlation coefficient is higher than the preset correlation coefficient, the feature projection matrix needs to be readjusted. This adjustment further optimizes the projection matrix to better separate point defect features and linear defect features. After reading the projection matrix, the projection and decoupling process is performed again. This process is repeated until the cross-correlation coefficient is lower than the preset correlation coefficient. When the cross-correlation coefficient is lower than the preset correlation coefficient, it indicates that the feature decoupling has achieved a good effect, and the cross-interference between point defect features and linear defect features has been effectively suppressed.

[0106] Step 140: Based on the decoupled independent point defect features and the decoupled independent line defect features, perform defect classification and judgment, and generate defect detection results containing defect type and defect area coordinates.

[0107] After feature decoupling and dimensionality compression, decoupled independent point defect features and independent linear defect features are obtained. Defect classification is performed based on these features to determine the type and location of the defects. For decoupled independent point defect features, a targeted point defect classifier is used. The point defect classifier classifies point defects into different types based on their feature patterns, such as scratches and bubbles. Simultaneously, the coordinate range of the point defect region is determined based on the spatial location information in the features. For decoupled independent linear defect features, a linear defect classifier is used. The linear defect classifier classifies linear defects into different types based on their features, such as long linear scratches and short linear blemishes, and determines their region coordinates. After obtaining the classification results and coordinate ranges of point and linear defects, this information is integrated to generate a defect detection result containing the defect type and defect region coordinates.

[0108] In an optional embodiment, step 140 further includes:

[0109] Step 141: Input the decoupled independent point defect features into the point defect classifier for defect type identification processing, output the point defect type and corresponding confidence level, and determine the coordinate range of the point defect region based on the spatial location information in the decoupled independent point defect features.

[0110] The decoupled independent point defect features contain key feature information about point defects and are input into a point defect classifier for defect type identification. The point defect classifier is a trained model that can determine the type of point defect based on the input feature information. During classification, the classifier analyzes and compares the input features, matching them with pre-learned feature patterns of different types of point defects. Based on the matching results, it outputs the most probable point defect type. Simultaneously, to evaluate the reliability of the classification results, the classifier outputs a corresponding confidence score. The confidence score represents the degree of credibility of the classification result, and can be represented by a probability value, for example. In addition to defect type and confidence score, the decoupled independent point defect features also contain spatial location information. By extracting this spatial location information, the coordinate range of the point defect region can be determined. This spatial location information may be stored in a predetermined format, such as the minimum bounding rectangle parameter. Based on these parameters, the coordinates of the upper left and lower right corners of the point defect region can be calculated, thus determining its specific coordinate range.

[0111] Preferably, step 141 includes:

[0112] Step 1411: Perform feature optimization processing on the decoupled independent point defect features, input the optimized independent point defect features into the input layer of the point defect classifier, and perform feature mapping processing through the multilayer perceptron network in the input layer of the point defect classifier to generate the target feature vector.

[0113] Although the decoupled, independent point defect features have undergone the previous processing, they may still contain some factors that are detrimental to classification, such as unreasonable feature distribution or unclear representation of certain features. Therefore, feature optimization is performed first. Feature optimization adjusts and transforms the features according to the characteristics of point defects and the requirements of the classifier. For example, feature normalization may be performed to ensure that all features are within the same scale range, avoiding the impact of excessive differences in feature scale on classification performance. Features may also be screened, removing features that contribute little to classification and retaining the most representative features.

[0114] After feature optimization, the optimized independent point defect features are input into the input layer of the point defect classifier. The input layer of the point defect classifier contains a multilayer perceptron network (MPB), a feedforward neural network consisting of an input layer, hidden layers, and an output layer. In the input layer, the MPB performs feature mapping on the input features. Feature mapping refers to mapping the input features from the original feature space to a new feature space, making the features more suitable for classification in the new space. The MPB implements feature mapping through a series of neurons and connection weights. Each neuron receives a weighted sum of the input features, which is then nonlinearly transformed through an activation function to obtain the neuron's output. Multiple neurons in the input layer process the input features simultaneously, passing the processed results to the hidden layer. Through the feature mapping processing of the MPB, the target feature vector is finally generated.

[0115] Step 1412: Input the target feature vector into the hidden layer of the classifier for nonlinear transformation processing to extract discriminative features for classification.

[0116] After obtaining the target feature vector, it is input into the hidden layer of the classifier. The hidden layer is the core part of the multilayer perceptron network, enabling deeper processing and analysis of the input features. In the hidden layer, a non-linear transformation is performed on the target feature vector. This non-linear transformation is achieved through activation functions, such as ReLU and Sigmoid. Activation functions introduce non-linearity, allowing the multilayer perceptron network to learn more complex feature patterns.

[0117] The hidden layer contains multiple neurons, each receiving a weighted sum of the target feature vector. This sum is then transformed non-linearly through an activation function to obtain the neuron's output. Different neurons process different parts of the target feature vector, extracting different feature information. Through the processing of the hidden layer, latent feature information within the target feature vector can be mined, forming discriminative features for classification. Discriminative features are those that can effectively distinguish different types of point defects. For example, for point defects of different shapes and grayscale distributions, the hidden layer can extract corresponding features, enabling the classifier to classify them more accurately.

[0118] Step 1413: Input the discriminative features into the output layer of the classifier, calculate the probability value of each point defect type using the softmax function, take the defect type with the highest probability value as the predicted point defect type, and use the corresponding probability value as the confidence level.

[0119] After processing by the hidden layer, discriminative features for classification are obtained. These discriminative features are then input into the output layer of the classifier. The main function of the output layer is to calculate the probability value of each point defect type based on the discriminative features. This calculation is performed using the softmax function in the output layer. The softmax function is a commonly used classification function that maps the input features to a probability distribution such that the sum of all probability values ​​is 1. For each point defect type, the softmax function calculates its corresponding probability value based on the discriminative features. The probability value represents the likelihood that the point defect belongs to that type. The probability values ​​of all point defect types are compared, and the defect type with the highest probability value is selected as the predicted point defect type. This probability value is then used as the confidence score. The confidence score reflects the reliability of the classification result; the higher the probability value, the more reliable the classification result. In this way, the type of point defect can be accurately predicted, and the corresponding confidence score can be provided.

[0120] Step 1414: Extract spatial location features from the decoupled independent point-like defect features, wherein the spatial location features include the minimum bounding rectangle parameter of the defect region.

[0121] The decoupled independent point defect features include not only the defect's shape and grayscale, but also spatial location features. These spatial location features are crucial for determining the precise location of point defects. Extracting these features from the decoupled independent point defect features includes the minimum bounding rectangle parameter of the defect region. The minimum bounding rectangle is the smallest rectangle that completely encloses the point defect region. By analyzing the boundary pixels of the point defect region, the coordinates of the top-left and bottom-right corners, as well as the width and height of the minimum bounding rectangle, can be determined. These parameters accurately describe the spatial location of the point defect region.

[0122] Step 1415: Based on the minimum bounding rectangle parameters, calculate the coordinates of the upper left corner and the lower right corner of the defect area to determine the coordinate range of the point defect area.

[0123] After obtaining the parameters of the minimum bounding rectangle, the coordinates of the top-left and bottom-right corners of the defect region are calculated based on these parameters. The coordinates of the top-left and bottom-right corners of the minimum bounding rectangle directly determine the location range of the defect region. Based on the definition and parameter relationships of the minimum bounding rectangle, the coordinates of the top-left and bottom-right corners can be obtained through simple calculation. If the relative position of the top-left vertex of the minimum bounding rectangle and the width and height of the rectangle are known, the specific coordinate values ​​can be calculated. By calculating the top-left and bottom-right corner coordinates, the coordinate range of the point defect region can be determined, and this coordinate range clearly defines the specific location of the point defect in the image.

[0124] Step 1416: Associate and store the predicted point defect type, the corresponding confidence level, and the coordinate range of the point defect region to generate a point defect classification result record.

[0125] After obtaining the predicted point defect type, corresponding confidence score, and coordinate range of the point defect region, this information needs to be stored in a linked manner. This linked storage facilitates subsequent querying, statistics, and analysis. The predicted point defect type, confidence score, and coordinate range are combined to form a complete record. This record contains the classification and location information of the point defect, comprehensively describing its characteristics. Through linked storage, a point defect classification result record is generated.

[0126] Step 142: Input the decoupled independent linear defect features into the linear defect classifier for defect type identification processing, output the linear defect type and corresponding confidence level, and determine the coordinate range of the linear defect region based on the spatial location information in the decoupled independent linear defect features.

[0127] The decoupled independent linear defect features contain key feature information about linear defects and are input into a linear defect classifier for defect type identification. The linear defect classifier is a model specifically designed for identifying linear defect types. Similar to the point defect classifier, the linear defect classifier also performs a series of processing and analysis on the input features. In the linear defect classifier, the input independent linear defect features are first preprocessed, possibly including feature normalization and filtering to improve classification accuracy. Then, through the internal neural network structure of the classifier, feature extraction and classification are performed. The classifier matches the input features with pre-learned feature patterns of different types of linear defects, calculating the probability value for each type. Using classification functions such as the softmax function, the type with the highest probability value is determined as the predicted linear defect type, and the corresponding probability value is output as the confidence score. Simultaneously, the decoupled independent linear defect features contain spatial location information. By extracting this spatial location information, the coordinate range of the linear defect region can also be determined. Depending on the characteristics of the linear defect, different methods may be used to determine the coordinate range. For example, for continuous linear defects, the coordinate range of the entire defect area can be determined by identifying the coordinates of its starting and ending points, as well as the intermediate extension path. Ultimately, the linear defect type, its corresponding confidence level, and the coordinate range of the linear defect area are obtained.

[0128] Step 143: Perform threshold judgment processing on the confidence scores output by the point defect classifier and the linear defect classifier, and retain the defect types and corresponding coordinate ranges whose confidence scores are higher than the preset confidence scores.

[0129] The confidence scores output by the point defect classifier and the line defect classifier reflect the reliability of the classification results. To ensure the accuracy of defect detection results, a threshold judgment process is needed for the confidence scores. The preset confidence score is a threshold set based on actual needs and experience, representing the minimum acceptable confidence level. The confidence scores output by the point defect classifier and the line defect classifier are compared with the preset confidence score. If the confidence score for a certain defect type is higher than the preset confidence score, it indicates that the classification result is relatively reliable, and the defect type and its corresponding coordinate range are retained. If the confidence score is lower than the preset confidence score, it indicates that the classification result may have a large error, and the defect type and its corresponding coordinate range are discarded. Through threshold judgment, reliable defect detection results can be filtered out, improving the accuracy of defect detection.

[0130] Step 144: Input the coordinate ranges of the retained point defect area and the coordinate range of the line defect area into the area conflict detection module to determine whether there is spatial overlap between different types of defect areas.

[0131] After thresholding, defect types and their corresponding coordinate ranges with confidence levels higher than a preset confidence level are retained. These coordinate ranges for point-like and line-like defect regions are then input into the region conflict detection module. The module's function is to determine whether spatial overlap exists between different types of defect regions. Within this module, the coordinate ranges of each point-like and line-like defect region are compared. By checking for overlap in the coordinate ranges, spatial overlap is determined. For example, it checks whether the coordinate ranges of a point-like defect region intersect with those of a line-like defect region. If an intersection exists, it indicates spatial overlap between the two defect regions. The region conflict detection module performs this check on all retained defect regions to determine which defect regions exhibit spatial overlap.

[0132] Step 145: If there is spatial overlap, then according to the priority rules of defect types, retain the defect types and coordinate ranges of the defect types that have reached the set priority level, and discard the defect types and coordinate ranges of the defect types that have not reached the set priority level.

[0133] When the area conflict detection module determines that spatial overlap exists, it needs to process the defect type according to priority rules. These priority rules are set based on actual conditions and requirements; different types of defects may have different priorities. For example, some defect types that severely impact touchscreen performance may have higher priorities, while some minor defect types may have lower priorities. For spatially overlapping defect areas, the priorities of different defect types are compared. If the priority of a defect type reaches the set level, that defect type and its corresponding coordinate range are retained. If the priority does not reach the set level, that defect type and its corresponding coordinate range are discarded. In this way, in the case of spatial overlap, more important defect information can be prioritized, avoiding the impact of improper handling of overlapping areas on the accuracy and effectiveness of defect detection.

[0134] Step 146: Combine the processed defect type and the corresponding coordinate range to generate a defect detection result containing the defect type name, the coordinates of the upper left corner and the lower right corner of the defect area.

[0135] After the preceding processing, the processed defect types and their corresponding coordinate ranges are obtained. This information is then integrated to generate a defect detection result containing the defect type name, the coordinates of the top-left corner of the defect area, and the coordinates of the bottom-right corner. The defect detection result is the final output of the entire defect detection process, containing detailed information on point and line defects on the touchscreen surface.

[0136] For each defect type, its name is associated with a corresponding coordinate range. The coordinate range is represented by the coordinates of the top-left corner and the bottom-right corner, which accurately locates the defect area. All defect type information is organized according to a specific format to form a complete defect detection result.

[0137] As a non-limiting embodiment, it further includes: statistically analyzing the falsely detected defect regions and missed defect regions in the defect detection results to determine the weak region information of feature extraction in the multi-branch feature extraction network; the weak region information includes the defect type, the degree of feature loss, and the corresponding image region coordinates; based on the weak region information, identifying the feature extraction parameter defects of the first feature branch and the second feature branch in the multi-branch feature extraction network, the feature extraction parameter defects including the insufficient scale coverage defect of the Gaussian filter parameter set of the first feature branch and the unreasonable division of the directional gradient interval of the second feature branch; for the insufficient scale coverage defect of the Gaussian filter parameter set, in the original Gaussian filter... New filter parameters are inserted into the parameter set to ensure that the scale interval of the filter parameters meets the feature scale requirements of the corresponding defect type in the weak region information, generating an adjusted first branch parameter set. For defects with unreasonable directional gradient interval division, the number and angle range of directional gradient intervals in the second feature branch are re-divided to match the angle coverage of the directional gradient intervals with the directional feature distribution of the corresponding defect type in the weak region information, generating an adjusted second branch parameter set. The original feature extraction parameters of the multi-branch feature extraction network are replaced with the adjusted first branch parameter set and the adjusted second branch parameter set to obtain the optimized multi-branch feature extraction network.

[0138] In the above embodiments, the original feature extraction parameters of the multi-branch feature extraction network are replaced with the adjusted first branch parameter set and the adjusted second branch parameter set. After the replacement, the multi-branch feature extraction network can better extract various types of defect features, resulting in an optimized multi-branch feature extraction network. The optimized multi-branch feature extraction network can improve the accuracy and reliability of defect detection.

[0139] As a non-limiting embodiment, the method further includes: obtaining a preset set of real defect labels, which contains the actual defect type and corresponding region coordinates of each defect region; performing spatial matching and type comparison between the predicted defect type, confidence level, and region coordinates in the defect detection results and the actual defect type and region coordinates in the set of real defect labels to generate a classification error matrix, which contains the type deviation value and confidence level deviation value of each matched defect region; extracting a set of deviated defect samples based on the classification error matrix, which contains defect samples whose type deviation value is greater than a preset deviation threshold or whose confidence level deviation value is greater than a preset confidence level threshold; and processing the set of deviated defect samples... The importance of decoupled independent point-like and independent linear defect features is evaluated to identify key feature components that affect the classification results. Based on the key feature components, the hidden layer weight parameters of the point-like and linear defect classifiers are adjusted to strengthen the weight of key feature components in the classification process and suppress the interference of non-key feature components. Based on the adjusted hidden layer weight parameters, optimized point-like and linear defect classifiers are obtained. Using the optimized point-like and linear defect classifiers, the decoupled independent point-like and independent linear defect features are reclassified to generate optimized defect detection results.

[0140] This design allows the optimized defect detection results to more accurately reflect the defects on the touchscreen surface, thus improving the performance of defect detection.

[0141] This invention improves the lighting conditions of a touchscreen by performing adaptive illumination equalization on the acquired touchscreen surface image to obtain an illumination-compensated image with target brightness distribution characteristics. A multi-branch feature extraction network is used to extract point defect feature sets and line defect feature sets, where the point defect feature set includes contour curvature features and grayscale distribution features, and the line defect feature set includes directional gradient features and length continuity features, achieving comprehensive and targeted extraction of different types of defect features. Feature decoupling is performed on the above feature sets to suppress cross-interference between different types of defect features, resulting in decoupled independent point defect features and decoupled independent line defect features, improving feature independence and accuracy. Based on the decoupled independent features, defect classification is performed, generating defect detection results containing defect type and defect region coordinates, significantly improving the accuracy and reliability of touchscreen surface defect detection, and enabling more precise identification and location of different types of defects.

[0142] Furthermore, Figure 2A structural block diagram of an image recognition system 300 for touchscreen defect detection is shown, including: a memory 310 for storing program instructions and data; and a processor 320 for coupling with the memory 310 to execute the instructions in the memory 310 to implement the above-described method.

[0143] Furthermore, a computer storage medium is also provided, comprising instructions that, when executed on a processor, implement the above-described method.

[0144] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image recognition method for detecting defects in touchscreens, characterized in that, The method includes: Adaptive illumination equalization processing is performed on the acquired touchscreen surface image to obtain an illumination-compensated image with target brightness distribution characteristics; The illumination-compensated image is processed by a multi-branch feature extraction network to extract point defects and line defects, respectively, resulting in a set of point defects and a set of line defects. The set of point defects includes the contour curvature features and gray-level distribution features of the defect region, while the set of line defects includes the orientation gradient features and length continuity features of the defect region. The point defect feature set and the line defect feature set are subjected to feature decoupling processing to obtain decoupled independent point defect features and decoupled independent line defect features; wherein, the feature decoupling processing is used to suppress cross-interference between different types of defect features; Based on the decoupled independent point defect features and the decoupled independent line defect features, defect classification and judgment are performed to generate defect detection results that include defect type and defect region coordinates.

2. The method according to claim 1, characterized in that, The step involves extracting point-like defect features and line-like defect features from the illumination-compensated image using a multi-branch feature extraction network, resulting in a set of point-like defect features and a set of line-like defect features, including: The illumination-compensated image is input into the input layer of the multi-branch feature extraction network. The feature channel separation module of the input layer divides the feature channel of the illumination-compensated image into a first feature branch and a second feature branch. The first feature branch is used to extract point defect features, and the second feature branch is used to extract line defect features. In the first feature branch, the illumination-compensated image is subjected to multi-scale Gaussian filtering to generate a set of Gaussian filtered images with different degrees of blur. The Gaussian filtered image set is then subjected to difference operation to obtain a difference image containing edge enhancement information. Contour detection processing is performed on the difference image to extract the contour curvature features of the closed region in the difference image, and the gray value distribution curve of the pixels in the closed region is calculated as the gray value distribution feature. The contour curvature feature and the gray value distribution feature are combined to form a set of point defect features. In the second feature branch, the image after illumination compensation is processed by calculating the histogram of directional gradients to generate gradient intensity distribution maps in different directions. The gradient intensity distribution maps are then processed by non-maximum suppression to obtain gradient feature maps with directional consistency. The gradient feature map is subjected to connected component analysis to extract the directional gradient features and length continuity features of continuous gradient regions. The directional gradient features and length continuity features are then combined to form a linear defect feature set. The point defect feature set output by the first feature branch and the line defect feature set output by the second feature branch are input into the feature alignment module of the multi-branch feature extraction network for spatial position calibration processing, generating a point defect feature set and a line defect feature set with a unified coordinate reference system.

3. The method according to claim 2, characterized in that, In the first feature branch, the illumination-compensated image undergoes multi-scale Gaussian filtering to generate a set of Gaussian-filtered images with different degrees of blur. The Gaussian-filtered image set is then subjected to difference operations to obtain a difference image containing edge enhancement information, including: In the first feature branch, a Gaussian filter parameter set is initialized, the Gaussian filter parameter set containing filter parameters for generating different degrees of blur; Based on each filtering parameter in the Gaussian filtering parameter set, the illumination-compensated image is processed by Gaussian convolution to generate a Gaussian filtered image that corresponds one-to-one with the filtering parameter; wherein, all Gaussian filtered images together constitute a Gaussian filtered image set. According to the increasing order of the filtering parameters, the adjacent Gaussian filtered images in the Gaussian filtered image set are processed by pixel-by-pixel difference operation to generate a difference image set. Each difference image in the difference image set corresponds to the difference result of a set of adjacent Gaussian filtered images. Pixel values ​​of all differential images in the differential image set are normalized so that the pixel values ​​of each differential image are within a preset grayscale range. The normalized set of difference images is input into the edge enhancement module, and the Laplacian operator is used to perform edge enhancement processing on each difference image to generate a target difference image containing edge enhancement information. The target difference image is subjected to connected component labeling processing, and connected components with an area smaller than a preset threshold are removed to obtain a purified edge-enhanced difference image.

4. The method according to claim 3, characterized in that, The process involves performing contour detection processing on the differential image, extracting the contour curvature features of closed regions in the differential image, calculating the grayscale value distribution curve of pixels within the closed regions as grayscale distribution features, and combining the contour curvature features and the grayscale distribution features to form a set of point-like defect features, including: The difference image is binarized to generate a binarized image containing the target region and the background region; The binarized image is subjected to contour extraction processing to identify all closed contour lines, and each closed contour line corresponds to a potential point-like defect region; For each closed contour line, a curve fitting algorithm is used to perform polynomial fitting on the pixels on the contour line to generate a contour fitting curve. Based on the contour fitting curve, the curvature value of each point on the contour fitting curve is calculated, a curvature value sequence is generated, and the curvature value sequence is statistically analyzed to extract the maximum curvature value, the average curvature value, and the curvature variance as contour curvature features. Within the area enclosed by each closed contour line, the gray values ​​of all pixels are collected to generate a gray value sequence. The probability density estimation process is performed on the gray value sequence to generate a gray value distribution curve as a gray value distribution feature. The contour curvature features and grayscale distribution features corresponding to each closed contour line are aligned according to the feature dimensions. The aligned contour curvature features and grayscale distribution features are then concatenated according to the channel dimension to form the feature vector of each potential point defect region. All feature vectors together constitute the point defect feature set.

5. The method according to claim 4, characterized in that, In the second feature branch, the image after illumination compensation undergoes orientation gradient histogram calculation to generate gradient intensity distribution maps in different directions. Non-maximum suppression is then applied to these gradient intensity distribution maps to obtain gradient feature maps with consistent orientations, including: In the second feature branch, the illumination-compensated image is converted to grayscale to generate a single-channel grayscale image; The grayscale image is convolved with Sobel operators in the horizontal and vertical directions to generate horizontal gradient images and vertical gradient images. Based on the horizontal gradient image and the vertical gradient image, calculate the gradient magnitude and gradient direction angle of each pixel, and generate a gradient magnitude matrix and a gradient direction angle matrix. The gradient direction angle matrix is ​​divided into a preset number of direction intervals, and the gradient magnitude in each direction interval is accumulated based on the gradient magnitude matrix to generate a direction gradient histogram. Based on the directional gradient histogram, gradient intensity distribution maps for different directions are constructed, with each gradient intensity distribution map corresponding to the gradient magnitude distribution in a directional interval. Non-maximum suppression is performed on each gradient intensity distribution map to retain local maxima points along the gradient direction and suppress the gradient magnitudes of non-maxima points, thereby generating a preliminary gradient feature map. The preliminary gradient feature map is thresholded to remove pixels with gradient magnitudes lower than a preset threshold, resulting in a gradient feature map with consistent orientation. Gradient feature maps from different directions are merged to generate multi-channel gradient feature maps, with each channel corresponding to the gradient features of a directional interval.

6. The method according to claim 1, characterized in that, The step of performing feature decoupling processing on the set of point defects and the set of line defects to obtain decoupled independent point defect features and decoupled independent line defect features includes: Construct a feature correlation matrix, which is used to represent the correlation coefficient between each feature component in the point defect feature set and the line defect feature set; The feature correlation matrix is ​​subjected to eigenvalue decomposition to obtain eigenvalues ​​and corresponding eigenvectors. The main correlation feature directions are determined based on the magnitude of the eigenvalues. Based on the main associated feature directions, a feature projection matrix is ​​constructed, and the point defect feature set and the line defect feature set are projected onto mutually orthogonal feature subspaces based on the feature projection matrix. In the point defect feature subspace, remove the feature components containing linear defect features and retain the components containing point defect features to form decoupled independent point defect features. In the linear defect feature subspace, feature components containing point defect features are removed, while components containing linear defect features are retained to form decoupled independent linear defect features. Principal component analysis algorithm is used to compress the feature dimension of the decoupled independent point defect features and the decoupled independent line defect features, retaining the main feature components and removing redundant feature components. Calculate the cross-correlation coefficient between the decoupled independent point defect features and independent line defect features. If the cross-correlation coefficient is higher than the preset correlation coefficient, readjust the feature projection matrix and perform projection and decoupling processing again until the cross-correlation coefficient is lower than the preset correlation coefficient.

7. The method according to claim 6, characterized in that, The construction of the feature correlation matrix, which represents the correlation coefficient between each feature component in the point defect feature set and the line defect feature set, includes: The point defect feature set and the line defect feature set are converted into feature matrix form; Calculate the Pearson correlation coefficient between every two feature components in the feature matrix to generate a correlation coefficient matrix. The rows and columns of the correlation coefficient matrix correspond to all feature components in the point defect feature set and the line defect feature set, respectively. The correlation coefficient matrix is ​​adjusted so that the matrix element values ​​are within a preset range, resulting in an adjusted correlation coefficient matrix. Based on the adjusted correlation coefficient matrix, a feature correlation matrix is ​​constructed, where each element of the feature correlation matrix represents the correlation strength between two corresponding feature components.

8. The method according to claim 7, characterized in that, The step of constructing a feature projection matrix based on the main associated feature directions, and projecting the point defect feature set and the line defect feature set onto mutually orthogonal feature subspaces based on the feature projection matrix, includes: Based on the feature vectors obtained from the eigenvalue decomposition process, the feature vector corresponding to the largest eigenvalue is selected as the main associated feature direction; Based on the main associated feature direction, a first projection matrix and a second projection matrix are constructed. The first projection matrix is ​​used to project the feature into a subspace parallel to the main associated feature direction, and the second projection matrix is ​​used to project the feature into a subspace perpendicular to the main associated feature direction. The point defect feature set is input into the first projection matrix to obtain the projection components of the point defect feature set in the main associated direction. The point defect feature set is input into the second projection matrix to obtain the projection components of the point defect feature set in the vertical direction. The linear defect feature set is input into the first projection matrix to obtain the projection components of the linear defect feature set in the main associated direction. The linear defect feature set is input into the second projection matrix to obtain the projection components of the linear defect feature set in the vertical direction. The projection components of the point defect feature set in the main associated direction are compared and analyzed with the projection components of the line defect feature set in the main associated direction to determine the cross-interference feature components. Based on the cross-interference feature components, the projection directions of the first projection matrix and the second projection matrix are adjusted, and the adjusted projection matrix is ​​used to enhance the separation of cross-interference feature components. Using the adjusted projection matrix, the point defect feature set and the line defect feature set are projected again into the new orthogonal feature subspace.

9. The method according to claim 1, characterized in that, The defect classification and judgment based on the decoupled independent point defect features and the decoupled independent line defect features, generating a defect detection result including the defect type and defect region coordinates, includes: The decoupled independent point defect features are input into a point defect classifier for defect type identification, and the point defect type and corresponding confidence level are output. Based on the spatial location information in the decoupled independent point defect features, the coordinate range of the point defect region is determined. The decoupled independent linear defect features are input into a linear defect classifier for defect type identification, and the linear defect type and corresponding confidence level are output. Based on the spatial location information in the decoupled independent linear defect features, the coordinate range of the linear defect region is determined. Threshold judgment processing is performed on the confidence scores output by the point defect classifier and the linear defect classifier, and defect types and their corresponding coordinate ranges with confidence scores higher than the preset confidence scores are retained. Input the coordinate ranges of the retained point-like defect area and the coordinate range of the linear defect area into the area conflict detection module to determine whether there is spatial overlap between different types of defect areas; If there is spatial overlap, then according to the priority rules of defect types, retain the defect types and coordinate ranges that have reached the set priority level, and discard the defect types and coordinate ranges that have not reached the set priority level. By combining the processed defect type and its corresponding coordinate range, a defect detection result is generated, which includes the defect type name, the coordinates of the upper left corner of the defect area, and the coordinates of the lower right corner. The process of inputting the decoupled independent point defect features into a point defect classifier for defect type identification, outputting the point defect type and corresponding confidence level, and determining the coordinate range of the point defect region based on the spatial location information in the decoupled independent point defect features includes: The decoupled independent point defect features are subjected to feature optimization processing. The optimized independent point defect features are input into the input layer of the point defect classifier. The feature mapping is performed through the multilayer perceptron network in the input layer of the point defect classifier to generate the target feature vector. The target feature vector is input into the hidden layer of the classifier and subjected to nonlinear transformation to extract discriminative features for classification. The discriminative features are input into the output layer of the classifier, and the probability value of each point defect type is calculated by the softmax function. The defect type with the highest probability value is used as the predicted point defect type, and the corresponding probability value is used as the confidence level. Spatial location features are extracted from the decoupled independent point-like defect features, wherein the spatial location features include the minimum bounding rectangle parameter of the defect region; Based on the minimum bounding rectangle parameters, calculate the coordinates of the upper left and lower right corners of the defect area to determine the coordinate range of the point defect area. The predicted point defect type, the corresponding confidence level, and the coordinate range of the point defect region are associated and stored to generate a point defect classification result record.

10. An image recognition system for detecting defects in touchscreens, characterized in that, include: Memory is used to store program instructions and data; A processor, coupled to a memory, for executing instructions in the memory to implement the method as described in any one of claims 1-9.

Images