A method and system for identifying and processing image data of minor defects on sealing surfaces of ferrule connectors

Abstract

The application provides a kind of data processing method and system for identifying image of small defect of ferrule joint sealing surface, and relates to the technical field of machine vision, the method comprises: the original image of the surface of the ferrule joint obtained is preprocessed to obtain an enhanced image;A feature extraction and detection model based on attention-enhanced YOLOv8 network is constructed, the defect feature of the enhanced image is extracted and preliminarily positioned and identified, and a preliminary detection prediction containing a candidate box and a category is obtained, the feature extraction and detection model includes a backbone network, an optimized neck network and a detection head;The preliminary detection prediction is post-processed and optimized to obtain a refined detection box set.The application strengthens the defect feature through targeted image preprocessing, realizes accurate identification through attention-enhanced network, improves the reliability of the results through post-processing optimization, and preserves the measurement accuracy through local geometric correction, and finally outputs high-precision quantitative defect data that can be directly used for quality control.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method and system for image recognition and data processing of minute defects on the sealing surface of a ferrule connector. Background Technology

[0002] In semiconductor manufacturing, the integrity of the sealing surface of ferrule connectors (such as VCR connectors) directly affects the system's airtightness. Micrometer-level surface defects (such as scratches ≥5μm, pinholes ≥3μm, and microcracks) can easily cause leaks; therefore, 100% online inspection of the sealing surface is essential on the production line. Currently, automated optical inspection based on machine vision is the mainstream method, its key being the data processing and defect analysis of surface images acquired at 150x magnification. However, existing technologies still have many shortcomings when processing such high-magnification, micrometer-level defect images, mainly the following:

[0003] First, in the preprocessing stage, traditional contrast enhancement algorithms amplify surface texture noise while enhancing defects, and conventional filtering methods struggle to balance smoothing noise with preserving sub-pixel-level defect edges. Second, in the intelligent recognition stage, feature extraction based on standard YOLO and other networks easily loses shallow details of small targets and lacks a targeted attention mechanism to focus on key regions and feature channels in complex texture backgrounds; multi-scale feature fusion is also insufficiently adaptable to small defects. Third, in the post-processing optimization stage, non-maximum suppression strategies using a fixed cross-union ratio threshold may not be able to accurately remove duplicates of heterogeneous defects such as densely distributed pinholes and partially overlapping linear scratches, and generally lack logical verification based on prior knowledge of defect physical size and distribution. Finally, in the geometric calibration stage, a single global transformation model cannot correct the nonlinear and non-uniform spatial distortion introduced by 150x magnification imaging, which may lead to significant errors in defect localization and size measurement in image edge regions. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for image recognition and data processing of minute defects on the sealing surface of a ferrule connector. The method enhances defect features through targeted image preprocessing, achieves accurate recognition through attention enhancement network, improves the reliability of results through post-processing optimization, and ensures measurement accuracy through local geometric correction. The final output is high-precision quantitative defect data that can be directly used for quality control.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A first aspect is a method for image recognition and data processing of minute defects on the sealing surface of a ferrule connector, the method comprising:

[0007] The original image of the ferrule connector surface is preprocessed to obtain an enhanced image;

[0008] A feature extraction and detection model based on attention-enhanced YOLOv8 network is constructed to extract defect features and perform preliminary localization and recognition on enhanced images, and to obtain preliminary detection predictions including candidate boxes and categories. The feature extraction and detection model includes a backbone network, an optimized neck network, and a detection head.

[0009] The preliminary detection prediction is post-processed and optimized to obtain a refined set of detection boxes;

[0010] The imaging process images known spatial coordinate reference points calibrated on the physical plane of the sealing surface, obtains a calibration image, and extracts the pixel coordinates of each reference point. Based on the mapping relationship between the known spatial coordinates and pixel coordinates, a local distortion correction parameter lookup table is established for the imaging area of ​​the sealing surface. According to the center pixel coordinates of each detection frame in the refined detection frame set, the local distortion correction parameter lookup table is queried to obtain the corresponding local correction parameters. Using the local correction parameters, coordinate mapping transformation is performed on the vertex pixel coordinates and dimensions of each detection frame to obtain the precise position and size of each detection frame on the physical plane. Based on the precise position and size of each detection frame on the physical plane, the final defect detection data is obtained.

[0011] Secondly, a data processing system for image recognition of minute defects on the sealing surface of a ferrule connector includes:

[0012] The preprocessing module is used to preprocess the acquired raw image of the ferrule connector surface to obtain an enhanced image;

[0013] The intelligent detection module is used to construct a feature extraction and detection model based on the attention-enhanced YOLOv8 network, to extract defect features and perform preliminary localization and identification on the enhanced image, and to obtain preliminary detection predictions including candidate boxes and categories. The feature extraction and detection model includes a backbone network, an optimized neck network, and a detection head.

[0014] The optimization module is used to perform post-processing optimization on the preliminary detection prediction to obtain a refined detection box set;

[0015] The geometric calibration module is used to image known spatial coordinate reference points calibrated on the physical plane of the sealing surface through the imaging process, acquire calibration images, and extract the pixel coordinates of each reference point; based on the mapping relationship between the known spatial coordinates and pixel coordinates, a local distortion correction parameter lookup table is established for the imaging area of ​​the sealing surface; according to the center pixel coordinates of each detection box in the refined detection box set, the local distortion correction parameter lookup table is queried to obtain the corresponding local correction parameters; using the local correction parameters, coordinate mapping transformation is performed on the vertex pixel coordinates and dimensions of each detection box to obtain the precise position and size of each detection box on the physical plane; based on the precise position and size of each detection box on the physical plane, the final defect detection data is obtained by summarizing.

[0016] Thirdly, a computing device includes:

[0017] One or more processors;

[0018] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0019] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0020] The above-described solution of the present invention has at least the following beneficial effects:

[0021] The system optimizes the quality of the original image, reduces noise interference, and enhances the distinction between defects and background; it provides a clearer and more easily identifiable data foundation for subsequent feature extraction, improving the effectiveness and stability of feature extraction; an attention enhancement mechanism can focus on key feature channels and regions of defects, optimizing the targeting of feature extraction; the collaborative architecture of the backbone network, neck network, and detection head enables the full mining and fusion of multi-scale defect features; it performs feature extraction and preliminary localization and recognition in parallel, balancing the depth and efficiency of data processing, and generating structured detection results containing candidate boxes and categories; it filters low-reliability and redundant candidate boxes, reducing the amount of data to reduce the subsequent processing load; through targeted post-processing logic, it enhances the effectiveness and regularity of the detection results, forming a high-quality refined set of detection boxes, providing accurate data input for geometric correction; it establishes a local distortion correction parameter lookup table based on reference point calibration, adapting to the nonlinear and non-uniform distortion of high-magnification imaging, and achieving regionalized accurate correction; it matches exclusive correction parameters according to the detection box position, ensuring accurate mapping from pixel coordinates to physical coordinates; and it summarizes and generates physical detection data, transforming image-level results into quantitative data that can be directly used for quality control, improving the practicality and traceability of detection data. Attached Figure Description

[0022] Figure 1 This is a schematic flowchart of an image recognition data processing method for minor defects on the sealing surface of a ferrule connector, provided by an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of an image recognition and data processing system for minute defects on the sealing surface of a ferrule connector, provided by an embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0025] like Figure 1 As shown, an embodiment of the present invention proposes a method for image recognition and data processing of minute defects on the sealing surface of a ferrule connector, the method comprising the following steps:

[0026] Step 100: Preprocess the original image of the obtained ferrule connector surface to obtain an enhanced image;

[0027] Step 200: Construct a feature extraction and detection model based on attention-enhanced YOLOv8 network to extract defect features and perform preliminary localization and recognition on the enhanced image, and obtain preliminary detection predictions including candidate boxes and categories. The feature extraction and detection model includes a backbone network, an optimized neck network, and a detection head.

[0028] Step 300: Post-processing optimization is performed on the preliminary detection prediction to obtain a refined detection box set;

[0029] Step 400: Imaging is performed on known spatial coordinate reference points calibrated on the physical plane of the sealing surface to obtain a calibration image and extract the pixel coordinates of each reference point; based on the mapping relationship between the known spatial coordinates and pixel coordinates, a local distortion correction parameter lookup table is established for the imaging area of ​​the sealing surface; according to the center pixel coordinates of each detection box in the refined detection box set, the local distortion correction parameter lookup table is queried to obtain the corresponding local correction parameters; using the local correction parameters, coordinate mapping transformation is performed on the vertex pixel coordinates and dimensions of each detection box to obtain the precise position and size of each detection box on the physical plane; based on the precise position and size of each detection box on the physical plane, the final defect detection data is obtained.

[0030] In this embodiment of the invention, the grayscale contrast between micron-level defects and the substrate is enhanced, surface texture noise is suppressed, and defect details are fully preserved. Focusing on micro-defect regions, the weights of key defect feature channels are strengthened, improving the ability to express micro-defect features and achieving efficient defect feature extraction and preliminary localization. The overlapping recognition problem caused by densely distributed defects is specifically addressed, selecting accurate detection boxes and improving the regularity of detection results. A precise mapping between the physical plane and pixel coordinates is established, correcting imaging distortion and ensuring accurate measurement of the position and size of defects on the physical plane, resulting in reliable final defect detection data.

[0031] In a preferred embodiment of the present invention, step 100 above, which preprocesses the acquired original image of the ferrule connector surface to obtain an enhanced image, includes:

[0032] Step 101: Convert the acquired original image into a grayscale image matrix, and define a local neighborhood window centered on each pixel in the grayscale image matrix. Calculate the mean grayscale value and standard deviation of each pixel within the local neighborhood window. Specifically, this includes: First, perform a grayscale conversion operation on the original image of the card sleeve connector surface acquired at 150x magnification. Calculate the grayscale value of each pixel in the original image according to the conventional weighting ratio of the RGB three-color channels, that is, multiply the grayscale values ​​of the red, green, and blue channels by their corresponding weighting coefficients and then sum them to obtain the grayscale value of each pixel, thereby constructing a complete grayscale image matrix to simplify the dimensionality and computational complexity of subsequent data processing. Next, using each pixel in the grayscale image matrix as the center, pre-calibrate the size of the local neighborhood window based on the correspondence between the minimum size of the micron-level defect after 150x magnification and the pixel. For example, select a 3×3, 5×5, or 7×7 rectangular window to ensure that the local neighborhood window of each pixel can effectively cover the surrounding grayscale variation area without missing minute defect details.

[0033] Based on this, all pixels within the local neighborhood window corresponding to each pixel are traversed, and the grayscale value of each pixel within the window is counted one by one. First, the grayscale values ​​of all pixels are summed and then divided by the total number of pixels in the window to obtain the grayscale mean of the local neighborhood window. Then, the deviation of the grayscale value of each pixel in the window from the grayscale mean is calculated. The squares of all deviations are summed and then divided by the total number of pixels. The square root of the result is then performed to obtain the grayscale standard deviation of the local neighborhood window, providing accurate basic data support for the subsequent implementation of adaptive local contrast enhancement.

[0034] Step 102: Based on the ratio of the grayscale standard deviation to the preset global image grayscale standard deviation, dynamically calculate the local contrast enhancement coefficient of the pixel. Specifically, this includes: based on the grayscale standard deviation of the local neighborhood window of each pixel, first determine the preset method of the global image grayscale standard deviation. This preset method is not a fixed value, but is obtained by dynamic statistical calculation of the grayscale image matrix corresponding to the original image of the 150x magnified card sleeve connector surface to be processed, so as to adapt to the actual grayscale distribution characteristics of the current image. Specifically, traverse all pixels of the entire grayscale image matrix, count the grayscale values ​​of all pixels and calculate the global grayscale mean, then calculate the deviation between the grayscale value of each pixel and the global grayscale mean, sum the square values ​​of all deviations and divide them by the total number of pixels in the entire image, and perform a square root operation on the result to obtain the preset global image grayscale standard deviation. This global grayscale standard deviation can objectively reflect the overall grayscale dispersion of the entire image.

[0035] Subsequently, for each pixel, the standard deviation of its corresponding local neighborhood window grayscale is divided by the standard deviation of the global image grayscale to obtain the ratio. This ratio quantifies the degree of difference in grayscale distribution between the local area of ​​the pixel and the entire image. Finally, based on this ratio and a preset coefficient adjustment rule, the local contrast enhancement coefficient for each pixel is determined. For example, when the ratio is greater than 1.2, it indicates that the local grayscale change is more drastic than the overall change, and an enhancement coefficient between 1.5 and 2.0 is assigned to strengthen the difference between defects and the background. When the ratio is between 0.8 and 1.2, it indicates that the local and overall grayscale changes tend to be consistent, and an enhancement coefficient between 1.0 and 1.5 is assigned. When the ratio is less than 0.8, it indicates that the local grayscale change is gradual, and an enhancement coefficient between 0.8 and 1.0 is assigned, so that the enhancement coefficient can accurately adapt to the local grayscale characteristics of different pixels.

[0036] Step 103: Using the enhancement coefficient, local gray-level mean, and preset global contrast stretching parameter, perform a linear transformation calculation on the original gray-level value of the pixel to enhance the gray-level difference between the micron-level defect area and the background substrate, obtaining a contrast-enhanced intermediate image. Specifically, this includes: based on the local contrast enhancement coefficient of each pixel, combining the local gray-level mean calculated in step 101, and simultaneously introducing a preset global contrast stretching parameter. The preset global contrast stretching parameter needs to be based on the image features of the semiconductor ferrule connector under a 150x magnification scenario. Specifically, this involves collecting a large number of surface gray-level image samples of similar semiconductor ferrule connectors after 150x magnification, covering various working conditions such as qualified parts, parts with micron-level scratches, parts with pinholes, and parts with microcracks. A comprehensive statistical analysis is performed on the gray-level values ​​of all samples to determine the gray-level difference between the defect area and the background substrate in the samples. The grayscale distribution range is determined to maximize the highlighting of minute defects and background differences without causing grayscale information overflow or loss. Based on this, parameter calibration is performed in conjunction with conventional technical standards for image grayscale representation. Typically, the upper limit of the effective grayscale value range is set to 255 and the lower limit is set to 0. If sample statistics show that there is a grayscale distribution bias under specific working conditions, the upper and lower limits are slightly adjusted to adapt to the actual detection requirements. This preset method ensures that the parameters conform to both general technical specifications and specific detection scenarios, thereby ensuring the overall contrast balance of the entire image. Then, based on the original grayscale value of each pixel, the difference between the original grayscale value and the local grayscale mean is calculated. This difference is multiplied by the corresponding local contrast enhancement coefficient to obtain the enhanced grayscale deviation value. Finally, the enhanced grayscale deviation value is added to the local grayscale mean to obtain the preliminary transformed grayscale value.

[0037] Finally, the grayscale values ​​after the initial transformation are cropped based on the global contrast stretching parameters. If the grayscale values ​​after the initial transformation are higher than the stretching upper limit, the stretching upper limit value is used; if they are lower than the stretching lower limit value, the stretching lower limit value is used. Through this series of linear transformation operations, the grayscale difference between the micron-level defect area and the background substrate, which is magnified by 150 times, is specifically enhanced, avoiding the problem of simultaneous noise amplification in traditional enhancement methods, and finally obtaining the contrast-enhanced intermediate image.

[0038] Step 104: For each pixel to be processed in the contrast-enhanced intermediate image, define a neighborhood range centered on it with a preset filter kernel size; calculate the spatial distance weight and grayscale similarity weight between each pixel within the neighborhood range and the pixel to be processed. Specifically, this includes: for the contrast-enhanced intermediate image, selecting each pixel to be processed one by one, and determining the preset method for the preset filter kernel size centered on each pixel. This preset method needs to be closely combined with the minute defect features of the semiconductor ferrule connector under 150x magnification and the accuracy parameters of the imaging system. Specifically, this is achieved through the following process: First, collect a large number of similar semiconductor ferrule connector surface image samples after 150x magnification. The sample includes typical defects such as pinholes of 3μm or larger, scratches of 5μm or larger, and microcracks. Image calibration technology is used to establish the correspondence between physical size and pixel size, and to determine the actual pixel occupancy range of various micro defects at 150x magnification. For example, actual measurements show that pinholes of 3μm or larger correspond to 3 to 4 pixels in the image, and scratches of 5μm or larger correspond to 5 to 6 pixels. Based on this, and considering the need to cover the surrounding pixels of the defect to preserve edge details during the filtering process, while avoiding the introduction of irrelevant noise by the filter kernel being too large, the filter kernel size is set to be slightly larger than the maximum pixel size of various defects, and an odd-dimensional rectangular kernel is used to ensure that the pixel to be processed is located at the center of the kernel.

[0039] Subsequently, through testing and verification with multiple sets of filter kernels of different sizes, the noise suppression effect and the degree of defect detail preservation under different sizes were compared to calibrate and obtain an appropriate filter kernel size. For example, when a pinhole corresponds to a 3-pixel size, a 5×5 filter kernel is selected; when a scratch corresponds to a 5-pixel size, a 7×7 filter kernel is selected to ensure that the filter kernel can completely cover the pixels surrounding the defect without exceeding the effective feature range. Based on this, the spatial distance weight and grayscale similarity weight between each other pixel and the pixel to be processed within the neighborhood are calculated. The spatial distance weight is calculated according to the Euclidean distance between pixels. The closer the pixels are, the higher the weight value; the farther away the pixels are, the lower the weight value gradually becomes according to a preset ratio. For example, the weight of adjacent pixels is 0.8, the weight of pixels separated by one pixel is 0.4, and the weight of pixels outside the radius of the filter kernel is 0.1. The grayscale similarity weight is calculated based on the absolute difference between the grayscale values ​​of two pixels. The smaller the difference, the higher the similarity and the larger the weight value. The weight value gradually decreases according to a preset ratio. For example, the weight is 1.0 when the grayscale difference is 0, and the weight decreases by 0.2 for every 10 increase in the difference. The minimum weight is no less than 0.1. The calculation of dual weights provides comprehensive parameter support for subsequent accurate filtering.

[0040] Step 105: Multiply the grayscale similarity weight by a local variance modulation coefficient based on dynamic adjustment of local grayscale variance within the neighborhood to obtain a comprehensive weight. Specifically, this includes: based on the obtained grayscale similarity weight, firstly, statistically analyze the grayscale values ​​of all pixels within the neighborhood corresponding to each pixel to be processed. Combined with the neighborhood grayscale mean calculated in step 101, calculate the squared deviation of each pixel's grayscale value from this mean. Sum all squared deviations and divide by the total number of pixels in the neighborhood to obtain the local grayscale variance of that neighborhood. This local grayscale variance directly reflects the degree of grayscale change within the neighborhood. Then, based on this local grayscale variance, dynamically adjust the local variance modulation coefficient according to a preset modulation rule. For example, when the local grayscale variance is greater than 50, it indicates... Dramatic grayscale changes within the neighborhood may indicate defective edges. Therefore, the modulation coefficient is set to 1.2 to 1.5 to enhance detail preservation. When the local grayscale variance is between 20 and 50, it indicates smooth grayscale changes within the neighborhood with no significant noise. The modulation coefficient is set to 1.0 to maintain the original characteristics. When the local grayscale variance is less than 20, it indicates weak grayscale changes within the neighborhood, possibly indicating texture noise or grain noise. The modulation coefficient is set to 0.7 to 0.9 to enhance noise suppression. Finally, the grayscale similarity weight obtained in step 104 is multiplied by the local variance modulation coefficient to obtain the comprehensive weight corresponding to each neighboring pixel. This comprehensive weight simultaneously considers the grayscale similarity between pixels and the local grayscale change characteristics of the neighborhood, improving the targeting and adaptability of the filtering operation.

[0041] Step 106: Calculate a weighted average of the grayscale values ​​of all pixels in the neighborhood using the comprehensive weights. This preserves the micron-level defect edge details in the contrast-enhanced intermediate image while suppressing surface texture noise and particle noise introduced by magnified imaging, resulting in an enhanced image. Specifically, this includes: after obtaining the comprehensive weights of each pixel within its neighborhood, performing a weighted average calculation based on these comprehensive weights. First, multiply the grayscale value of each pixel in the neighborhood by its corresponding comprehensive weight to obtain the weighted grayscale value of each pixel. Then, sum the weighted grayscale values ​​of all pixels, simultaneously summing the comprehensive weights of all pixels in the neighborhood. Finally, use the weighted grayscale values... The final grayscale value of the pixel to be processed is obtained by dividing the sum of the total weights by the sum of the total weights. This weighted average calculation process can make full use of the differentiated allocation characteristics of the total weights. For areas with drastic grayscale changes, such as defect edges, the corresponding pixels have higher total weights, and their grayscale information dominates the calculation, thus fully preserving the edge details of micron-level defects in the contrast-enhanced intermediate image. For noisy areas such as processing textures and material particles, the corresponding pixels have lower total weights, and their grayscale information is effectively weakened, thereby suppressing the surface texture noise and particle noise introduced during 150x magnification imaging. Finally, the synergistic optimization of noise suppression and detail preservation is achieved, outputting an enhanced image that meets the accuracy requirements of subsequent feature extraction and defect recognition.

[0042] In this embodiment of the invention, the image data format is converted, the local neighborhood range of the pixel is determined, and the local gray-level mean and standard deviation are obtained to provide basic data support for subsequent dynamic contrast enhancement. An enhancement coefficient is dynamically generated based on the gray-level standard deviation ratio, allowing the enhancement intensity to adapt to the local gray-level distribution characteristics of the pixel, achieving differentiated contrast adjustment. Linear transformation is used to enhance the gray-level distinction between micron-level defects and the background matrix, making the features of minute defects more clearly presented in the image. The neighborhood range of the filter kernel is defined, simultaneously considering spatial distance and gray-level similarity, providing dual targeted weighting criteria for filtering. The local gray-level variance modulation coefficient is fused to optimize the comprehensive weight, making the weight allocation conform to the local gray-level change pattern of the neighborhood, improving the filter adaptability. Through weighted average calculation, while fully preserving the edge details of micron-level defects, surface texture noise and particle noise caused by magnified imaging are effectively suppressed, outputting a high-quality enhanced image.

[0043] In a preferred embodiment of the present invention, step 200 above involves constructing a feature extraction and detection model based on an attention-enhanced YOLOv8 network to extract defect features and perform preliminary localization and identification on the enhanced image, obtaining preliminary detection predictions including candidate boxes and categories. The feature extraction and detection model includes a backbone network, an optimized neck network, and a detection head, comprising:

[0044] Step 201: Perform multi-level convolutional downsampling on the enhanced image to extract and output an initial feature map set containing semantic information at different scales. Specifically, this includes: First, for the enhanced image, sequentially perform convolutional downsampling operations using 4 to 6 pre-set convolutional layers in the backbone network. The pre-set number of 4 to 6 convolutional layers needs to be closely combined with the pixel-scale features of micron-level defects under 150x magnification and the feature extraction requirements. The specific pre-set process is as follows: First, collect a large number of image samples of the sealing surface of the ferrule connector containing pinholes greater than or equal to 3μm and scratches greater than or equal to 5μm, and test the impact of different convolutional layer levels on the feature extraction effect. With fewer than 4 layers, it is impossible to fully extract the deep semantic features of defects, making it difficult to distinguish defects from background textures. With more than 6 layers, multiple downsampling can easily lead to the loss of details such as shallow edges and grayscale abrupt changes in micron-level defects. Therefore, considering both extraction effect and computational efficiency, the final preset number of convolutional layers is 4 to 6. The kernel size of each convolutional layer is pre-calibrated based on the pixel scale of the aforementioned defects after 150x magnification, and a 3×3 kernel is uniformly selected. This size has been verified through multiple tests and can balance feature extraction accuracy and computational efficiency while covering the smallest defect pixel scale, avoiding the loss of details due to excessively large kernels and the surge in computational load due to excessively small kernels.

[0045] The preset number of convolutional kernels follows the principle of progressively increasing feature dimensions. The lower convolutional layers focus on extracting shallow details such as defect edges and textures, so the number is relatively small. The higher convolutional layers focus on extracting deep semantic features such as the overall shape of defects and category associations, so the number gradually increases. For example, the first layer is set to 64 kernels, the second layer to 128, the third layer to 256, and the fourth layer to 512. If extended to the fifth and sixth layers, the number is set to 512 and 1024 respectively, ensuring that the feature dimensions of each layer are adapted to the feature extraction target of that layer. The stride of the convolutional operation is dynamically adjusted according to the target downsampling factor. The stride of the first two convolutional layers is set to 1. This parameter has been tested and can completely preserve shallow details such as edges and textures of micron-level defects, avoiding the fragmentation of detail information caused by excessive stride. The stride of the third to sixth convolutional layers is set to 2, so as to achieve progressive downsampling from 2x to 32x, balancing downsampling efficiency and feature integrity.

[0046] After each convolution operation, the convolution result is nonlinearly transformed using a pre-defined SiLU activation function. SiLU, also known as SigmoidLinearUnits, is a nonlinear activation function that combines the smooth nonlinearity of the Sigmoid function with the non-negative range characteristics of the ReLU function. Its core feature is its self-gating mechanism, which adaptively adjusts the output weights according to the input value, thus enhancing the nonlinear expression of effective features while precisely suppressing ineffective features. The pre-defined process for SiLU activation is as follows: First, the performance of mainstream activation functions such as ReLU, LeakyReLU, and Swish in micrometer-level defect feature extraction is compared. It is found that ReLU is prone to gradient vanishing and cannot effectively enhance the nonlinear features of small defects. LeakyReLU... The suppression effect of background texture noise is insufficient. However, the SiLU activation function, with its self-gating mechanism, can accurately suppress invalid feature information such as background texture while enhancing the nonlinear expression of defect features. Therefore, this function was selected and preset in the backbone network. It has no additional adjustable parameters, only the core computation logic needs to be integrated, and it is fully compatible with the computation process of convolutional layers. Through the synergistic effect of the above multi-level convolution and downsampling, feature information of different scales from shallow details to deep semantics in the enhanced image is gradually extracted. Finally, the initial feature map set containing five downsampling factors of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 is output. This set covers shallow detail features such as the edge of small defects and gray-level abrupt changes, as well as deep semantic features such as the overall shape of defects and category associations. It provides comprehensive and multi-level basic feature support for subsequent attention enhancement and multi-scale fusion.

[0047] Step 202: Based on the initial feature map set, select at least one feature map at the target scale; based on the feature map at the target scale, generate a channel description vector through global pooling operation in the spatial dimension, and obtain a channel attention weight vector after processing and fusion by a multilayer perceptron. Specifically, this includes: based on the initial feature map set, firstly, combining the scale distribution characteristics of small defects in a 150x magnification scene, select feature maps with downsampling factors of 1 / 2, 1 / 4, and 1 / 8 as target scale feature maps, among which the 1 / 2 and 1 / 4 downsampling feature maps can clearly retain needles larger than or equal to 3μm. For detailed textures such as holes and scratches of 5μm or larger, the 1 / 8x downsampled feature map has a certain semantic expressive ability, ensuring that both detailed and global information are taken into account. For each selected target scale feature map, a global pooling operation is performed along the spatial dimension. Specifically, a combination of global average pooling and global max pooling is used. Global average pooling calculates the average gray value of all pixels in each channel, and global max pooling extracts the maximum gray value of all pixels in each channel. The average value and the maximum value of each channel are concatenated to form a one-dimensional description vector of length 2. The description vectors of all channels together constitute the channel description vector set.

[0048] The channel description vector set is then input into a pre-defined multilayer perceptron. This pre-defined multilayer perceptron must closely consider the channel feature dimension, computational efficiency, and model overfitting risk of micro-defects magnified 150x. The specific pre-defined process is as follows: First, analyze the number of channels in the target-scale feature map and the dimensionality of the channel description vector set. Combined with feature extraction test results from numerous samples of ferrule sealing surfaces containing micron-level defects, determine the core hierarchical structure of the multilayer perceptron as an input layer, a single hidden layer, and an output layer. This structure, after multiple rounds of verification, can ensure the cross-correlation effect of channel information while avoiding the surge in computational load and gradient vanishing problems caused by multilayer structures. The pre-defined dimension of the input layer is consistent with the dimension of the channel description vector set, ensuring that the input channel feature information can be completely transmitted to the perceptron without dimensional truncation or redundant padding. The pre-defined number of neurons in the hidden layer follows the dual objectives of dimensionality compression and feature fusion. After multiple tests comparing the feature extraction effects under different ratios, it is finally set to 1 / 4 of the number of channels in the target-scale feature map. This ratio can effectively compress features... While defining the feature dimension, the model fully explores the correlation features between channels to avoid computational redundancy caused by too many neurons and loss of key features due to too few neurons. The preset number of channels in the output layer dimension is the same as the number of channels in the target scale feature map, ensuring that the output attention weight vector can correspond one-to-one with the channels in the original feature map, achieving accurate channel weight allocation. Fully connected layers are used to realize cross-correlation and nonlinear mapping of information between channels. To avoid overfitting during training with limited defect samples, a dropout mechanism is introduced with a preset probability of 0.2. This probability value was determined through multiple sets of comparative experiments and can suppress overfitting while preserving effective channel feature information to the greatest extent. Finally, the output of the multilayer perceptron is mapped to the interval between 0 and 1 through a sigmoid normalization operation to obtain a channel attention weight vector that can quantify the importance of each channel. This vector can accurately capture the difference in contribution of different channels to the recognition of minor defects. For example, the channel weight corresponding to defect edge features is close to 1, and the channel weight corresponding to background texture is close to 0.

[0049] Step 203: The channel attention weight vector is used to perform channel weighting on the feature map of the target scale to obtain a channel-enhanced feature map. Based on the channel-enhanced feature map, a spatial feature map is generated through channel-dimensional pooling operations. After convolution and activation processing, a spatial attention weight matrix is ​​obtained. Specifically, the channel attention weight vector is first weighted with the corresponding target scale feature map through a channel-by-channel operation. That is, the feature values ​​of all pixels in each channel are multiplied one by one with the attention weight value corresponding to that channel. This enhances key feature channels such as defect edges and gray-scale abrupt changes, while weakening the interference of irrelevant background channels, thereby obtaining a channel-enhanced feature map.

[0050] Based on this, pooling operations are performed along the channel dimension for the channel-enhanced feature map. The channel average pooling and channel max pooling are processed in parallel. Channel average pooling calculates the sum of gray values ​​of each spatial location in all channels and divides it by the number of channels to obtain the average response value of that location. Channel max pooling extracts the maximum gray value of each spatial location in all channels as the maximum response value. The two response values ​​are concatenated along the channel dimension to generate a two-dimensional spatial feature map with two channels. This spatial feature map can centrally reflect the feature response intensity of each spatial location.

[0051] Subsequently, a 1×1 convolution operation is performed on the spatial feature map. By using a single 1×1 convolution kernel, channel dimension compression and spatial correlation feature integration are performed, reducing computational complexity while strengthening the feature correlation in the spatial dimension. After convolution, a nonlinear transformation is performed through the SiLU activation function to highlight the feature response of potential defect regions and suppress the response of background regions. Finally, a spatial attention weight matrix is ​​generated that is completely consistent with the spatial dimension of the channel enhancement feature map. In this matrix, the weight values ​​of the defect region are close to 1, and the weight values ​​of the background region are close to 0, which can accurately locate the defect candidate region in the spatial dimension.

[0052] Step 204: Spatial modulation of the channel enhancement feature map using the spatial attention weight matrix to obtain an enhanced feature map fused with dual attention. Specifically, this includes: performing element-wise multiplication of the spatial attention weight matrix with the channel enhancement feature map using spatial modulation. That is, the weight value at each spatial position in the spatial attention weight matrix is ​​multiplied one by one with the feature elements of all channels at the corresponding spatial position in the channel enhancement feature map. This further enhances the features of the high-weight spatial region where the defect is located, while effectively suppressing the features of the low-weight spatial region where background noise is located. Through the coordinated operation of the above channel weighting and spatial modulation, the enhancement feature map is further enhanced. The current approach combines channel attention and spatial attention. For example, if a pixel belongs to the defect edge channel (high channel weight) and is located in the defect region (high spatial weight), its feature value will be doubly amplified. If it belongs to the background channel (low channel weight) and is located in the background region (low spatial weight), its feature value will be doubly weakened. The result is an enhanced feature map that combines dual attention. This feature map highlights the feature channels that are key to defect identification and accurately focuses on the defect region in the spatial dimension. It significantly improves the distinguishability between small defect features and background noise, effectively adapting to the needs of small defect detection in 150x magnification scenarios.

[0053] Step 205: The enhanced feature map fused with dual attention is input together with other scale feature maps in the initial feature map set and fed into the optimized neck network for multi-scale feature fusion. The optimized neck network uses a feature pyramid architecture that includes upsampling, downsampling, and cross-layer connection operations to adaptively weight and aggregate information from the input features at multiple scales to obtain a multi-scale fused feature map for detection. Specifically, this includes: first, integrating the enhanced feature map fused with dual attention with other scale feature maps downsampled by 1 / 16 and 1 / 32 times in the initial feature map set and inputting them together into the optimized neck network. The neck network, based on the original YOLOv8 PANet structure, is improved by adding a dedicated feature fusion branch for minute defects. It is specifically designed for shallow detail features downsampled by 1 / 2 and 1 / 4 times. A 1×1 convolution adjusts the number of shallow feature channels to match that of mid-layer features, and then fuses them element-wise with the upsampled high-layer feature maps. During feature fusion, bilinear interpolation is used to upsample the high-layer feature maps downsampled by 1 / 16 and 1 / 32 times, enlarging the 1 / 32x feature map by 2x to 1 / 16x and the 1 / 16x feature map by 2x to 1 / 8x, respectively, so that their sizes are comparable to those of the mid-layer and shallow feature maps. Figure 1 Furthermore, a 3×3 convolution downsampling operation with a stride of 2 is performed on the low-level feature maps that are downsampled by 1 / 2 and 1 / 4 times to improve their semantic expressive power.

[0054] Subsequently, through cross-layer connection operations, the original low-level feature map is directly associated with the downsampled feature map, and the original high-level feature map is associated with the upsampled feature map, preserving the original information of each level of features and avoiding the loss of details during feature transfer. On this basis, an adaptive weighting mechanism is introduced to calculate the global average response value of the feature map at each scale, and normalize it as the weight coefficient for that scale. For example, the weight of the 1 / 2 downsampled feature map is set to 0.6, 1 / 4 downsampled to 0.3, and 1 / 8 downsampled and above to 0.1, so that the shallow detail features corresponding to small defects receive higher weights. Finally, the information of all input features is aggregated through the feature pyramid architecture, organically fusing shallow details, mid-level structure and deep semantic features, and finally obtaining a multi-scale fused feature map that can fully adapt to the needs of detecting small defects at different scales.

[0055] Step 206: Input the multi-scale fused feature map into the detection head and perform bounding box regression calculation and category probability distribution calculation respectively. The bounding box regression calculation generates the bounding box coordinates and size offset corresponding to each preset anchor point, and the category probability distribution calculation generates the defect category probability vector corresponding to each preset anchor point. Specifically, this includes: inputting the multi-scale fused feature map into the detection head, which includes a bounding box regression branch and a category probability distribution branch. The two branches are calculated in parallel to ensure detection efficiency. For the bounding box regression branch, regression prediction is performed on each preset anchor point in the multi-scale fused feature map through the collaborative operation of 3 layers of 3×3 convolutional layers and 1 layer of fully connected layers. The size and proportion of the preset anchor points need to be preset based on the actual pixel size characteristics of the three types of defects after 150x magnification, the detection accuracy requirements, and the anchor point adaptability test results. The specific preset process is as follows: First, a large number of 150x magnified image samples of the sealing surface of semiconductor ferrule connectors containing linear micro-scratches, pinholes, and irregular micro-cracks are collected. The samples need to cover the common sizes, shapes, and distribution scenarios of various defects to ensure the comprehensiveness of the statistical results.

[0056] Subsequently, precise pixel size annotations were performed on all real defect areas in the sample. The pixel diameter distribution range of pinholes, the pixel length and width range and aspect ratio distribution of linear micro-scratches, and the pixel length and width range and aspect ratio distribution of irregular micro-cracks were statistically analyzed. The pixel size and aspect ratio with the highest frequency of each type of defect were extracted as the initial basis for anchor point design. Based on the above statistical results, for pinholes, the high-frequency pixel diameter is concentrated between 6 and 8 pixels (corresponding to a physical size of 3 to 4 μm), therefore, 3×3 and 4×4 squares were initially designed. Anchor points: For linear micro-scratches, the length of high-frequency pixels is concentrated in 10 to 12 pixels and the width is concentrated in 2 to 4 pixels (corresponding to a physical length of 5 to 6 μm and a width of 1 to 2 μm), with an aspect ratio of about 5:2 to 6:2. Therefore, 5×2 and 6×2 rectangular anchor points are initially set. For irregular micro-cracks, the length of high-frequency pixels is concentrated in 8 to 10 pixels and the width is concentrated in 3 to 6 pixels (corresponding to a physical length of 4 to 5 μm and a width of 1.5 to 3 μm), with an aspect ratio of about 4:3 to 5:3. Therefore, 4×3 and 5×3 rectangular anchor points are initially set.

[0057] Finally, the initial anchor points were substituted into the detection head for multiple rounds of field testing and verification. The intersection-union ratio (IU) of each anchor point with the real defect bounding box was calculated. Anchor points with IU below a preset threshold were removed, and the optimal combination of anchor points was retained. The size and proportion of the anchor points were finally determined to ensure the fit between the anchor points and the defect size. During the regression calculation, the horizontal and vertical coordinate offsets Δx and Δy and the width and height offsets Δw and Δh of the anchor points relative to the real bounding box were predicted to generate the bounding box coordinates and size offsets corresponding to each anchor point. For the category probability distribution branch, the feature information of the multi-scale fused feature map was mapped to a category space containing three types of defects: linear micro-scratches, pinholes, and irregular micro-cracks through two 3×3 convolutional layers and one fully connected layer. The output results were converted into probability values ​​for each category through the Softmax activation function to generate a defect category probability vector corresponding to each preset anchor point. The sum of the probabilities of the three types of defects in this vector is 1, which can quantify the possibility that the anchor point region belongs to each type of defect.

[0058] Step 207: Decode the bounding box coordinates and size offsets, and defect category probability vectors corresponding to all anchor points to generate and output a preliminary detection prediction. This preliminary prediction includes the bounding box coordinates, confidence level, and the highest probability of its category for each candidate box. Specifically, this includes: First, decoding the bounding box coordinates and size offsets corresponding to all anchor points. The original coordinates of the anchor points are preset to be the center coordinates of the feature map grid. These original horizontal and vertical coordinates are added to the predicted Δx and Δy, respectively, to obtain the actual horizontal and vertical coordinates of each candidate bounding box in the image coordinate system. Simultaneously, the original width and height of the anchor points are multiplied by (1+Δw) and (1+Δh), respectively, to obtain the actual width and height of the candidate bounding box. Second, for each candidate bounding box, the defect category probability vector is... The system extracts the category with the highest probability value from the candidate boxes and uses it as the predicted category. This highest probability value is then weighted and fused with the localization accuracy score of the bounding box at a ratio of 0.6:0.4 to obtain the confidence score for each candidate box, used to measure the reliability of the detection results. Finally, all decoded candidate boxes are normalized. A pre-set confidence threshold of 0.5 is used to filter out candidate boxes with confidence scores higher than this threshold. These are then integrated according to a fixed format: the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, the y-coordinate of the bottom-right corner, the confidence score, and the highest probability of the category. This generates and outputs the preliminary detection prediction results. These results fully contain the core information required for defect detection, providing comprehensive and reliable basic data for accurate deduplication and logical verification in subsequent post-processing optimization stages.

[0059] In this embodiment of the invention, an initial feature set containing semantic information at different scales is obtained, providing comprehensive basic feature support for subsequent attention enhancement and multi-scale fusion; channel dimension correlation information of the target scale feature map is captured, and targeted channel attention weights are generated to enhance the expression priority of key feature channels of defects; the information distribution of feature channels is optimized, and important regional features of the spatial dimension are mined to lay the spatial feature foundation for dual attention fusion; key feature channels and core spatial regions are collaboratively focused to enhance the feature recognition of micron-level defects and improve the targeting and distinguishability of features; adaptive aggregation of features at different scales is achieved through multi-dimensional feature processing operations, integrating shallow details and deep semantic information to adapt to the multi-scale feature expression needs of micro-defects; accurate bounding box parameters and category probability data are generated to provide reliable quantitative basis for candidate box localization and category determination; detection information of all anchor points is integrated to output regular preliminary detection results, including core parameters of candidate boxes and category determination basis, ensuring the efficient implementation of subsequent post-processing optimization.

[0060] In a preferred embodiment of the present invention, step 300 above involves post-processing optimization of the preliminary detection prediction to obtain a refined detection box set, including:

[0061] Step 301: Based on preset confidence thresholds and category probability thresholds, filter the preliminary detection predictions to obtain an initial candidate box set. Specifically, this includes: First, determining the specific setting logic for the preset confidence thresholds and category probability thresholds. The preset thresholds need to be combined with large-scale measured sample statistics and online detection quality control indicators. The specific process is as follows: First, collect more than 10,000 sets of semiconductor ferrule connector defect detection samples magnified 150 times. The samples need to comprehensively cover connector images under different lighting conditions (such as strong light, weak light, side light), different defect degrees (such as critically qualified, minor defects, severe defects), and different processing batches to ensure the diversity and representativeness of the samples and avoid threshold errors due to a single sample. The set deviation was determined; then, the true defect candidate boxes in all samples were labeled, and the confidence and category probability distribution ranges of the true defect candidate boxes in various samples were statistically analyzed to determine the correlation between confidence, category probability and false negative rate and false positive rate; combined with the hard requirements of online inspection on the production line for a false negative rate of less than or equal to 0.3% and a false positive rate controlled within 1%, through multiple rounds of threshold debugging and verification, the false negative and false positive data under different combinations of confidence and category probability thresholds were tested, and finally it was determined that the confidence threshold was set at 0.5 and the category probability threshold was set at 0.6. This threshold combination was verified to be able to filter low reliability candidate boxes while controlling the false negative and false positive rates within the preset range, ensuring that no true defects are missed.

[0062] Based on this, all candidate boxes in the preliminary detection prediction are traversed from left to right and from top to bottom. The confidence value and the highest probability value of the category to which each candidate box belongs are extracted one by one. During the extraction process, a unique identifier for each candidate box is used to avoid parameter confusion. The confidence value of each candidate box is compared with a confidence threshold of 0.5, and the highest probability value of its category is compared with a category probability threshold of 0.6. Only candidate boxes with a confidence value greater than 0.5 and a highest probability value of the category greater than 0.6 are retained. Candidate boxes with a confidence value less than or equal to 0.5, a category probability less than or equal to 0.6, or any value that does not meet the threshold requirements are directly removed. Finally, all retained candidate boxes are initially classified according to defect categories to form an initial candidate box set. This set effectively removes more than 80% of low-reliability candidate boxes, reducing the amount of computational data in subsequent post-processing and laying a solid foundation for accurate optimization.

[0063] Step 302: Based on the defect category of each candidate box in the initial candidate box set, obtain a preset basic cross-union ratio (CUP) threshold; based on the confidence level of the candidate boxes in the initial candidate box set and the spatial distribution density calculated from the bounding box coordinates of the candidate boxes in the initial candidate box set, dynamically adjust the basic CUP threshold to generate an adaptive CUP threshold. Specifically, this includes: First, for the morphological differences and detection characteristics of the three types of defects—linear micro-scratches, pinholes, and irregular micro-cracks—preset corresponding basic CUP thresholds. The preset basic CUP threshold needs to be determined after multiple rounds of verification by actual test samples, taking into account the inherent morphological characteristics, actual distribution patterns, and the precise deduplication requirements of non-maximum suppression for the three types of defects. The specific process is as follows: First, analyze the core detection characteristics of the three types of defects. Pinholes are characterized by small size, low pixel ratio, and easy dense distribution. If the basic CUP threshold is set too high, it is easy to falsely suppress adjacent dense pinholes, resulting in the loss of real defects. Linear micro-scratches extend in a long strip shape and are prone to partial overlap during processing. A slightly higher threshold needs to be set to match their linear shape to ensure accurate identification of overlapping areas. Irregular microcracks have no fixed shape and often extend with branches, requiring a balance between overlap detection and preservation of crack branch details. Based on the above feature analysis, a large number of 150x magnified detection samples containing different distribution scenarios of the three types of defects were collected. Multiple rounds of cross-union ratio (CUP) threshold debugging and verification were conducted, testing the false suppression rate and missed suppression rate under different threshold values. Finally, the basic CUP thresholds suitable for various defects were determined: 0.3 for pinholes, 0.4 for linear micro-scratches, and 0.35 for irregular microcracks. Subsequently, each candidate box in the initial candidate box set was traversed according to the defect category. The defect category of the candidate box was extracted through its category identifier field. Based on the preset mapping relationship between the category and the basic CUP threshold, the basic CUP threshold corresponding to each candidate box was obtained.

[0064] Based on this, a threshold adjustment is performed on the confidence score of each candidate box. If the confidence score of a candidate box is higher than 0.8, it indicates that its localization accuracy and class determination reliability are extremely high. The corresponding basic intersection-union ratio (IU / R) threshold is increased by 0.08, ensuring that the adjusted threshold does not exceed 0.5, thus reducing the risk of falsely suppressing high-reliability candidate boxes. If the confidence score is between 0.5 and 0.8, the basic IU / R threshold remains unchanged, maintaining the conventional screening criteria. If the confidence score is lower than 0.5, the basic IU / R threshold is decreased by 0.04 to enhance the suppression effect on low-confidence redundant boxes. Simultaneously, the spatial distribution density of the initial candidate box set is calculated. The 150x magnified image is first divided into several non-overlapping grids of 10×10 pixels, and all candidate boxes are traversed. The bounding box coordinates of the selection box are used to determine the grid to which the center coordinates of each candidate box belong. The number of candidate boxes contained in each grid is counted. If the number of candidate boxes in a grid exceeds 5, the area is determined to be a high-density defect distribution area, and the basic cross-union ratio (CUP) threshold for all candidate boxes in that area is uniformly lowered by 0.05. If the number of candidate boxes in a grid is less than 2, the area is determined to be a low-density defect distribution area, and the corresponding basic CUP threshold is increased by 0.03. During the adjustment process, it is ensured that the lower limit of the threshold is not lower than 0.2 and the upper limit is not higher than 0.5. Through the dual dynamic adjustment of confidence and spatial distribution density, an adaptive CUP threshold is generated for each candidate box, achieving accurate adaptation of the CUP threshold with defect category, positioning reliability, and spatial distribution characteristics.

[0065] Step 303: Based on the adaptive intersection-union ratio (IU / R) threshold, perform iterative non-maximum suppression processing on the initial candidate box set to obtain a suppressed candidate box set. Specifically, this includes: First, grouping the initial candidate box set according to the category labels of linear micro-scratches, pinholes, and irregular micro-cracks to ensure that the non-maximum suppression processing of the same type of defect is more targeted and avoids interference between defects of different morphologies; For each group of defects, the candidate boxes are sorted from high to low according to their confidence scores. If there are candidate boxes with the same confidence score, they are sorted from high to low according to the highest probability score of their respective categories, with the candidate box with the highest confidence score serving as the reference box for the first round of suppression processing; Calculating the IU / R of the reference box with all other candidate boxes in the same group. Specifically, first extracting the bounding box coordinates of the reference box and the target candidate box, including the x-coordinate of the upper left corner, the y-coordinate of the upper left corner, the x-coordinate of the lower right corner, and the y-coordinate of the lower right corner, and calculating the x-coordinate of the upper left corner of the intersection region. The intersection area is calculated as the maximum x-coordinate of the top-left corner of each of the two bounding boxes, the maximum y-coordinate of the top-left corner of each bounding box, the minimum x-coordinate of the bottom-right corner of each bounding box, and the minimum y-coordinate of the bottom-right corner of each bounding box. If the width or height of the intersection area is negative, the intersection area is 0; otherwise, the intersection area is the product of the intersection width and height. The area of ​​the reference bounding box is calculated as the product of its width and height, and the area of ​​the target candidate bounding box is calculated as the product of its width and height. The area of ​​their union is the sum of the areas of the reference bounding box and the target candidate bounding box minus the intersection area. The intersection-union ratio (IUR) is the ratio of the intersection area to the union area. The calculated IUR is compared with the adaptive IUR threshold corresponding to the target candidate bounding box. If the IUR is higher than the adaptive IUR threshold, the candidate bounding box is considered redundant and is removed. If the IUR is lower than or equal to the adaptive IUR threshold, the candidate bounding box is retained.

[0066] After removing redundant boxes, the candidate box with the highest confidence is selected from the remaining candidate boxes as the new baseline box. The above operations of cross-union ratio calculation, threshold comparison and redundant box removal are repeated. This iterative logic is continued until all candidate boxes in the same group have participated in the suppression process as baseline boxes, and there are no redundant boxes to be removed in two consecutive iterations. After completing the iterative suppression process of candidate boxes for all categories of defects, the candidate boxes retained in each group are summarized to obtain the suppressed candidate box set. This set effectively solves the problem of repeated recognition caused by the dense distribution of dot-like pinholes and the partial overlap of linear micro-scratches.

[0067] Step 304: Based on the preset physical size range and distribution pattern parameters, the size and positional relationship of the suppressed candidate box set is verified. The candidate boxes that pass the verification are output as a refined detection box set. Specifically, this includes: First, determining the specific setting basis for the preset physical size range and distribution pattern parameters. The preset basis must be closely combined with the high airtightness requirements of the semiconductor ferrule connector, the actual influence law of defects, and the 150x magnification imaging characteristics. The specific process is as follows: First, a special calibration experiment is carried out. A standard calibration plate with known precise physical size markings is selected. The calibration plate is placed in the same 150x magnification imaging environment as the ferrule connector sealing surface detection. After acquiring the image of the calibration plate, the number of pixels corresponding to the marked physical size is measured. By repeatedly measuring and taking the average value, the fixed mapping relationship of 1μm corresponding to 2 pixels under 150x magnification is finally determined by actual measurement. For the physical size range, the critical value of the influence of micron-level defects on airtightness is combined with the... Based on statistical analysis of numerous actual defect samples, it was determined that pinholes must be at least 3μm in diameter to potentially cause leakage, and the maximum common failure pinhole is no more than 10μm. Therefore, the physical diameter range was set to 3μm to 10μm, which is equivalent to 6 to 20 pixels in diameter according to the mapping relationship. Linear micro-scratches of 5μm or larger will affect the sealing effect, and their common length does not exceed 20μm and width does not exceed 5μm. Therefore, the physical length range was set to 5μm to 20μm and the physical width range was set to 1μm to 5μm, which corresponds to 10 to 40 pixels in length and 2 to 10 pixels in width. The critical failure length of irregular microcracks is 8μm, and the common maximum length does not exceed 30μm. Moreover, their morphology often exhibits a distinct main branch. Therefore, the physical length range was set to 8μm to 30μm and the aspect ratio range was set to 5:1 to 20:1, which corresponds to 16 to 60 pixels in length and 5:1 to 20:1 pixel aspect ratio.

[0068] For the distribution pattern parameters, the parameters are determined based on the actual formation mechanism and physical distribution law of the three types of defects, combined with the statistical analysis of a large number of qualified and unqualified compression fittings. A sector area calculation algorithm is also incorporated. This algorithm determines the sector boundary based on the major axis direction of the candidate frame, the preset central angle of the sector, and the defect extension length. By calculating the area of ​​the sector and combining it with the distribution of candidate frames of the same category within the region, the algorithm verifies whether the defect distribution conforms to the actual formation mechanism. The core is to use the directionality and regional constraints of the sector to screen candidate frames that conform to the defect growth law. If the density of pinholes exceeds 10 per square millimeter, it will significantly increase the risk of airtightness failure. Based on the mapping relationship between pixels magnified 150 times and physical area, this translates to no more than 3 pinholes per 100×100 pixels in a 150x magnified image. Linear micro-scratches are mostly formed by continuous friction or scratches during processing, and their extension direction is continuous. Therefore, linear continuity requires adjacent candidate frames to maintain continuity. The included angle of the extension direction of the selection box does not exceed 30°. This angle is calculated by the vector angle of the candidate box along its major axis. At the same time, a sector area calculation algorithm is introduced for auxiliary verification. For example, the physical length of a linear micro-scratch candidate box is 10μm, and the preset central angle of the sector is 30°. Taking the end of the candidate box as the vertex and the extension direction of the major axis as the angle bisector, the sector radius is set at 1.5 times the physical length, i.e., 15μm. The area of ​​the sector region is obtained through the sector area calculation logic. The sector area calculation logic takes the ratio of the square of the radius to the central angle to the entire circumference as the core calculation basis. Then, it is judged whether the adjacent candidate boxes of the same category fall completely within the sector region. If they do not fall within it, the continuity is judged to be inconsistent. The growth of irregular microcracks is centered on the main crack. The length of the branch cracks is usually much smaller than that of the main crack. Otherwise, it will aggravate the damage to the sealing surface. Therefore, the branching characteristic requires that the length ratio of the main crack to the branch crack is not less than 3:1, and the longest microcrack candidate box in the same region is defined as the main crack.

[0069] Based on this, each candidate box in the suppressed candidate box set is traversed. According to the pixel-to-physical size mapping relationship determined above, the pixel width and pixel length of the candidate box are converted into the corresponding physical width and physical length. Among them, the pinhole candidate box is converted according to the diameter, and the scratch and crack candidate boxes are converted according to the major axis length. The converted physical size is checked one by one to see if it is completely within the preset physical size range of the corresponding defect category. At the same time, the distribution pattern is checked by combining the bounding box coordinates of other candidate boxes of the same category: for point pinhole candidate boxes, the 100×100 pixel area where its center coordinates are located is determined, and the number of candidate boxes of the same category in this area is counted. If it exceeds 3, the density is judged to be excessive. For linear micro-scratch candidate boxes, the major axis direction is first determined by the bounding box coordinates and the direction vector is calculated. The adjacent candidate boxes of the same category within the preset distance are found and their major axis direction vectors are calculated. The included angle is obtained by vector dot product operation. If it exceeds 30°, the continuity is judged to be unsatisfactory. At the same time, the sector area is calculated. If adjacent candidate boxes do not fall within the preset sector area, the continuity is also deemed inconsistent. For irregular microcrack candidate boxes, the area where the current candidate box is located is used as a reference to define the range. All candidate boxes of the same category within this range are identified, and the longest one is selected as the main crack. The length of the surrounding candidate boxes is measured as the length of the branch crack. The length ratio of the main crack to the branch crack is calculated. If it is less than 3:1, the branch characteristics are deemed inconsistent. Alternatively, the sector area calculation algorithm can be used to assist in the verification. With the midpoint of the main crack as the vertex and the major axis of the main crack as the angle bisector, the preset central angle of the sector is 60° and the radius is 0.8 times the length of the main crack. After calculating the sector area, it is determined whether the branch crack is within the sector area. If not, the branch distribution is deemed inconsistent with the growth law. Only candidate boxes whose physical dimensions are completely within the preset range and whose distribution mode parameters are all satisfied are retained. Invalid candidate boxes whose dimensions exceed the range or whose distribution mode is inconsistent are removed. Finally, the candidate boxes that have passed the double verification are sorted by defect category and physical coordinates and integrated and output to obtain a refined detection box set.

[0070] In this embodiment of the invention, low-reliability candidate boxes are quickly eliminated, reducing the amount of data for subsequent processing and improving post-processing efficiency; the intersection-union ratio (IUU) threshold is adapted to different defect categories, confidence levels, and spatial distribution characteristics, taking into account the identification needs of various defects; the problem of densely distributed or partially overlapping defects is efficiently solved, retaining candidate boxes corresponding to real defects; and the physical characteristics of the defects are combined for verification, eliminating invalid candidate boxes that do not conform to actual characteristics, ensuring the rationality of the detection boxes.

[0071] In a preferred embodiment of the present invention, step 400 involves imaging a known spatial coordinate reference point calibrated on the physical plane of the sealing surface through an imaging process, acquiring a calibration image, and extracting the pixel coordinates of each reference point; based on the mapping relationship between the known spatial coordinates and pixel coordinates, establishing a local distortion correction parameter lookup table for the imaging area of ​​the sealing surface; querying the local distortion correction parameter lookup table according to the center pixel coordinates of each detection frame in the refined detection frame set to obtain the corresponding local correction parameters; using the local correction parameters, performing coordinate mapping transformation on the vertex pixel coordinates and dimensions of each detection frame to obtain the precise position and size of each detection frame on the physical plane; and summarizing the final defect detection data based on the precise position and size of each detection frame on the physical plane, including:

[0072] Step 401 involves acquiring a calibration image containing known spatial coordinate reference points through an imaging process, and identifying and extracting the pixel coordinates of each reference point within the calibration image. Specifically, this includes: First, preparing a dedicated calibration plate with a material identical to the sealing surface of the semiconductor ferrule connector. Multiple reference points are pre-set on the calibration plate surface with a uniform grid density of 50μm × 50μm. These reference points are marked with black circular markers, and their physical diameter is set to 10μm to accommodate the recognition requirements of 150x magnification imaging. Using a laser interferometer with an accuracy of 0.1μm, a physical coordinate system is established with the geometric center of the calibration plate as the origin. The pixel coordinates of each reference point within this system are measured. In the physical coordinate system, a plane completely parallel to the sealing surface of the ferrule is selected as the coordinate reference plane for the three-dimensional spatial coordinates. After discarding the z-axis height data, the two-dimensional known spatial coordinates of each reference point are determined. Each coordinate value is retained to three decimal places to ensure accuracy. Then, the calibration plate is fixed in the same position of the ferrule testing fixture to ensure that its imaging distance and angle are completely consistent with the actual testing of the sealing surface of the ferrule. The imaging equipment is started to acquire calibration images containing all reference points at a magnification of 150x. During the acquisition process, the illumination intensity, light source angle, focal length, and object distance are kept synchronized with the actual testing scene to avoid coordinate deviations caused by environmental factors.

[0073] Based on this, preprocessing is performed on the acquired calibration images. The Otsu automatic thresholding algorithm is used to determine the grayscale segmentation threshold, separating the black markers of the reference points from the white calibration board background. The Otsu automatic thresholding algorithm, also known as the maximum inter-class variance method, is an adaptive image grayscale thresholding algorithm. Its core is to calculate the optimal grayscale threshold to divide image pixels into foreground and background classes, maximizing the inter-class variance between the two classes, thus achieving accurate separation of foreground and background without requiring manual threshold setting, adapting to image segmentation needs under different lighting and contrast conditions. Next, a circular contour detection algorithm is used to traverse the connected regions in the image. This algorithm is based on the geometric features of image regions to select circular targets; its core is to traverse the connected regions in the image. The circularity of a region is calculated, and the closer the circularity is to 1, the closer the region is to a perfect circle. This method is used to accurately identify target contours that conform to circular features. It is often used to extract circular markers, holes, and other circular targets in images. Regions with a circularity greater than 0.9 are selected as candidate contours for reference points. The geometric center coordinates of each candidate contour are calculated using the least circumscribed circle method. These center coordinates are the pixel coordinates of each reference point in the calibration image, and the pixel coordinates are kept to the integer part. Finally, a unique identifier is assigned to each reference point, and the identifier, the corresponding pixel coordinates (x pixel, y pixel), and the known spatial coordinates (x physical, y physical) are associated one by one and stored as a structured data table to form a complete set of coordinate correspondences. This provides high-precision basic data support for subsequent calculation of geometric transformation parameters.

[0074] Step 402: Based on the pixel coordinates and the corresponding known spatial coordinates, the imaging area of ​​the sealing surface is divided into multiple continuous imaging sub-regions; for each imaging sub-region, using the corresponding point pairs of the pixel coordinates and known spatial coordinates on its boundary, the geometric transformation parameters of the sub-region from the image pixel coordinate system to the physical coordinate system of the sealing surface are calculated; all imaging sub-regions and their corresponding geometric transformation parameters are collected to construct a local distortion correction parameter lookup table, specifically including: based on the correspondence between pixel coordinates and known spatial coordinates, firstly, according to the overall pixel size of the imaging area of ​​the sealing surface and the distortion distribution law obtained from previous tests, where the overall pixel size is, for example, 1024×1024 pixels, the imaging area is divided into multiple square imaging sub-regions according to the principle of equal size and continuity. Each sub-region has a side length of 40 pixels. During the division, it is ensured that adjacent sub-regions are seamlessly connected along the x-axis and y-axis, without gaps or overlaps. At the same time, through the pre-deployment design, it is ensured that each imaging sub-region contains at least 4 reference points, and the reference points are evenly distributed inside the sub-region and at the boundary to meet the accuracy requirements of geometric transformation parameter calculation. For each imaging sub-region, based on its position coordinates in the overall imaging area, such as the x-pixel range of the m-th row and n-th column sub-region being n×40 to (n+1)×40-1 and the y-pixel range being m×40 to (m+1)×40-1, the pixel coordinates corresponding to all reference points within this range and the known spatial coordinates are extracted to form a set of coordinate point pairs specific to this sub-region. Each point pair contains four parameters: pixel x, pixel y, physical x, and physical y.

[0075] Subsequently, a bivariate quadratic polynomial fitting algorithm was used to perform calculations on the set of coordinate points. This algorithm is a numerical fitting method based on a multivariate polynomial model to construct the mapping relationship between pixel coordinates and physical coordinates. Its core is to introduce a set of coordinates including x, y, xy, and x... 2 y 2 A five-term bivariate quadratic polynomial is used as the fitting model. The error equation is constructed using the least squares method, and the coefficients of each term in the polynomial are solved. This allows for accurate fitting of the nonlinear mapping relationship between pixel coordinates and physical coordinates, adapting to various geometric deformations such as translation, scaling, rotation, and nonlinear distortion that occur during the imaging process. The polynomial includes x, y, xy, x... 2 y 2Five terms are used to construct an error equation using the least squares method. Solving for the coefficients of each term in the polynomial yields the geometric transformation parameters of the sub-region from the image pixel coordinate system to the physical coordinate system of the sealing surface. These parameters comprehensively cover translation, scaling, rotation, and nonlinear distortion correction coefficients, accurately describing the mapping relationship between pixel coordinates and physical coordinates within the sub-region. After calculating the geometric transformation parameters for all imaging sub-regions, each sub-region is uniquely numbered according to its row and column order within the imaging region. The sub-region number, x-pixel start value, x-pixel end value, y-pixel start value, and y-pixel end value are associated with the corresponding polynomial coefficients, constructing a structured local distortion correction parameter lookup table. This lookup table is indexed and stored, supporting rapid matching of the corresponding sub-region and correction parameters using pixel coordinates, laying a solid foundation for efficient subsequent queries.

[0076] Step 403: Based on the center pixel coordinates of each detection box in the refined detection box set, query the local distortion correction parameter lookup table to obtain its corresponding local geometric transformation parameters as local correction parameters. Specifically, this includes: First, traversing each detection box in the refined detection box set, calculating its center pixel coordinates based on the bounding box coordinates of the detection box, specifically extracting the top left corner vertex pixel coordinates (x1, y1) and bottom right corner vertex pixel coordinates (x3, y3) of the detection box, and calculating the center x coordinate as (x1+x3) / 2 and the center y coordinate as (y1+y3) / 2 using the arithmetic mean method, keeping two decimal places to improve matching accuracy; then reading the local distortion correction parameter lookup table constructed in step 402, and comparing the center pixel coordinates of each detection box with the x-pixel interval and y-pixel interval of each imaging sub-region in the lookup table according to the sub-region number from left to right and from top to bottom; the comparison judgment rule is: if the center x coordinate is greater than or equal to the x-pixel starting value of a certain sub-region, and less than 1, the center pixel coordinates of each detection box are compared with the x-pixel interval and y-pixel interval of each imaging sub-region in the lookup table; the comparison judgment rule is: if the center x coordinate is greater than or equal to the x-pixel starting value of a certain sub-region, and less than 1, the center pixel coordinates of each detection box are compared with the x-pixel interval and y-pixel interval of each imaging sub-region in the lookup table. If the center y-coordinate is equal to the x-pixel termination value of the sub-region, and the center y-coordinate is greater than or equal to the y-pixel start value of the sub-region, and less than or equal to the y-pixel termination value of the sub-region, then the detection box is determined to belong to the imaging sub-region. After determining the imaging sub-region to which the detection box belongs, the coefficients of each term of the corresponding binary quadratic polynomial of the sub-region are directly extracted and used as the local correction parameters specific to the detection box. If the center pixel coordinate of the detection box falls exactly on the boundary of two adjacent sub-regions, such as the center x-coordinate being equal to the x-pixel termination value of sub-region A and equal to the x-pixel start value of sub-region B, or the center y-coordinate being equal to the y-pixel termination value of sub-region C and equal to the y-pixel start value of sub-region D, then the polynomial coefficients of the two adjacent sub-regions are extracted respectively, and the arithmetic mean is calculated for each set of corresponding coefficients. The average coefficient obtained is used as the local correction parameter of the detection box, ensuring the continuity and accuracy of boundary region correction. Through this targeted query method, the correction parameters are accurately matched with the distortion features of the region where the detection box is located.

[0077] Step 404: Based on the local correction parameters, for each detection box in the refined detection box set, read its vertex pixel coordinates and dimensions, and call the corresponding local correction parameters to construct a geometric transformation relationship; use the geometric transformation relationship to perform coordinate mapping transformation on the vertex pixel coordinates and dimensions to obtain the precise position and dimensions of the detection box in the physical coordinate system of the sealing surface. Specifically, this includes: assuming the local correction parameters for each detection box are obtained, firstly, reading the four vertex pixel coordinates of each detection box in a fixed order, namely the upper left corner (x1, y1) and the upper right corner (x2, y2). The bottom right corner (x3, y3) and bottom left corner (x4, y4) are used to ensure that the vertex coordinates correspond perfectly with the geometry of the detection box. Simultaneously, the pixel length (x2-x1) and pixel width (y3-y2) of the detection box are recorded. Then, the local correction parameters corresponding to the detection box are called, i.e., the coefficients of each term in the bivariate quadratic polynomial. These are substituted into a preset quadratic polynomial transformation model to construct the geometric transformation relationship specific to this sub-region. The physical x-coordinate is calculated by combining the first, second, and cross terms of pixel x and pixel y. The physical y-coordinate is calculated similarly. This relationship can comprehensively cover the nonlinearity introduced by 150x magnification imaging. Based on the geometric transformation relationship, the pixel coordinates of the four vertices are mapped one by one. First, the x and y pixel values ​​of each vertex are substituted into the transformation model to calculate the corresponding physical x and y coordinates. Each physical coordinate value is rounded to three decimal places. Based on the transformed physical coordinates of the four vertices, the actual size of the detection box in the physical coordinate system is calculated: for linear micro-scratches and irregular micro-cracks, the physical length is the difference between the physical x coordinates of the upper right corner and the upper left corner, and the physical width is the difference between the physical y coordinates of the lower right corner and the upper right corner; for pinhole defects, the physical diameter is... The straight-line distance between the physical coordinates of the top left and bottom right vertices is obtained by taking the square root of the sum of the squares of the horizontal and vertical distances between the two points. During this process, it is simultaneously verified whether the transformed physical coordinates are within the effective range of the physical plane of the sealing surface. The effective range is set with the geometric center of the sealing surface as the origin, and the x-axis and y-axis directions are both -5mm to 5mm. If the physical coordinates of a vertex exceed this range, it is marked as abnormal and the transformation parameters and calculation process are rechecked to ensure the rationality of the transformation results. Finally, the precise position (represented by the physical coordinates of the center of the detection box) and size of each detection box in the physical coordinate system are obtained.

[0078] Step 405: Based on the precise position and size of each detection frame on the physical plane, the final defect detection data, characterized by physical position and size values, is generated. Specifically, this includes: First, based on the precise physical position and size of each detection frame, all detection frames are classified according to defect type. The classification criteria are the shape and size ratio of the detection frames: those with an aspect ratio greater than 5:1 and an elongated shape are classified as linear micro-scratches; those with an aspect ratio between 0.8:1 and 1.2:1 and approximately circular are classified as pinholes; those with irregular shapes, no fixed aspect ratio, and branching extension characteristics are classified as irregular micro-cracks. For each type of defect, they are sorted along the positive x-axis of the physical coordinate system. If the difference in x-axis coordinates among multiple detection frames is within 0.1 μm, they are further sorted along the positive y-axis. The sorting results facilitate location lookup in subsequent quality control. The process involves defect tracing; subsequently, the core physical parameters of each inspection frame are extracted, including: defect ID (numbered in sorted order by category), category name, physical center x-coordinate, physical center y-coordinate, physical length, physical width, physical diameter, and pass / fail indicator. The physical position coordinates are characterized with the geometric center of the sealing surface as the origin, the physical dimensions are quantified in micrometers, and the pass / fail indicator is determined based on whether the physical dimensions are within a preset pass range (e.g., a pinhole diameter less than 3μm is considered pass). Finally, all the sorted physical parameters of the inspection frames are integrated in a unified structured format to form the final defect detection data containing all the above parameters. The data format is compatible with the import requirements of the production line quality control system and can be directly used for defect judgment, statistical analysis, and product quality traceability, providing accurate decision-making basis for ensuring the airtightness of semiconductor ferrule connectors.

[0079] In this embodiment of the invention, the correspondence between the pixel coordinates of the reference point and the known spatial coordinates is obtained, providing accurate raw data support for subsequent geometric transformation parameter calculations; adapting to the local nonlinear distortion characteristics of the sealing surface imaging, a targeted correction parameter system is established through regional division and parameter calculation, providing structured support for rapid and accurate queries; the correction parameters of the imaging area where the detection frame is located are accurately matched to ensure the targeting of the correction process and the efficiency of the query; coordinate transformation is performed based on the region-specific geometric transformation relationship to achieve accurate mapping of pixel information to physical quantities, ensuring the accuracy of the physical location and size of defects; and the detection data representing physical quantities are summarized to provide an intuitive and reliable quantitative basis for the quality control of high-precision parts.

[0080] like Figure 2 As shown, embodiments of the present invention also provide an image recognition data processing system for minute defects on the sealing surface of a ferrule connector, comprising:

[0081] The preprocessing module is used to preprocess the acquired original image of the ferrule connector surface to obtain an enhanced image;

[0082] The intelligent detection module is used to construct a feature extraction and detection model based on the attention-enhanced YOLOv8 network, to extract defect features and perform preliminary localization and identification on the enhanced image, and to obtain preliminary detection predictions including candidate boxes and categories. The feature extraction and detection model includes a backbone network, an optimized neck network, and a detection head.

[0083] The optimization module is used to perform post-processing optimization on the preliminary detection prediction to obtain a refined detection box set;

[0084] The geometric calibration module is used to image known spatial coordinate reference points calibrated on the physical plane of the sealing surface through the imaging process, acquire calibration images, and extract the pixel coordinates of each reference point; based on the mapping relationship between the known spatial coordinates and pixel coordinates, a local distortion correction parameter lookup table is established for the imaging area of ​​the sealing surface; according to the center pixel coordinates of each detection box in the refined detection box set, the local distortion correction parameter lookup table is queried to obtain the corresponding local correction parameters; using the local correction parameters, coordinate mapping transformation is performed on the vertex pixel coordinates and dimensions of each detection box to obtain the precise position and size of each detection box on the physical plane; based on the precise position and size of each detection box on the physical plane, the final defect detection data is obtained by summarizing.

[0085] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0086] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0087] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0088] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0089] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0090] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0091] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0093] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0094] Furthermore, it should be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Moreover, the steps performing the above series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order; some steps can be executed in parallel or independently of each other. Those skilled in the art will understand that all or any step or component of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or a combination thereof. This can be achieved by those skilled in the art using basic programming skills after reading the description of the present invention.

[0095] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a known general-purpose device. Therefore, the object of the present invention can also be achieved simply by providing a program product containing program code implementing the method or apparatus. That is, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should also be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent to the present invention. Furthermore, the steps performing the above series of processes can naturally be performed in the order described, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0096] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for image recognition and data processing of minute defects on the sealing surface of a compression fitting, characterized in that, The method includes: Step 100: Preprocess the original image of the ferrule connector surface obtained at 150x magnification to obtain an enhanced image, including: The original image obtained by magnification 150 times is converted into a grayscale image matrix, and a local neighborhood window is defined with each pixel in the grayscale image matrix as the center; the grayscale mean and grayscale standard deviation of each pixel in the local neighborhood window are calculated respectively. The local contrast enhancement coefficient of the pixel is dynamically calculated based on the ratio of the grayscale standard deviation to the preset global image grayscale standard deviation. Using the enhancement coefficient, local gray-scale mean, and preset global contrast stretching parameters, a linear transformation is performed on the original gray-scale value of the pixel to enhance the gray-scale difference between the micron-scale defect area and the background substrate that appears due to 150x magnification, thus obtaining a contrast-enhanced intermediate image. For each pixel to be processed in the contrast-enhanced intermediate image, a neighborhood range of a preset filter kernel size centered on it is defined; the spatial distance weight and gray value similarity weight between each pixel in the neighborhood range and the pixel to be processed are calculated. The grayscale similarity weight is multiplied by a local variance modulation coefficient that is dynamically adjusted based on the local grayscale variance within the neighborhood to obtain the comprehensive weight. The gray values ​​of all pixels in the neighborhood are weighted and averaged using the comprehensive weights to retain the micron-level defect edge details in the contrast-enhanced intermediate image while suppressing surface texture noise and particle noise introduced by magnified imaging, thus obtaining an enhanced image. Step 200: Construct a feature extraction and detection model based on an attention-enhanced YOLOv8 network to extract defect features and perform preliminary localization and identification on the enhanced image, obtaining preliminary detection predictions including candidate boxes and categories. The feature extraction and detection model includes a backbone network, an optimized neck network, and a detection head, comprising: Perform multi-level convolutional downsampling operations on the enhanced image to extract and output an initial set of feature maps containing semantic information at different scales; Based on the initial feature map set, at least one feature map of the target scale is selected; based on the feature map of the target scale, a channel description vector is generated through global pooling operation in the spatial dimension, and the channel attention weight vector is obtained after processing and fusion by a multilayer perceptron. The channel attention weight vector is used to perform channel weighting on the feature map at the target scale to obtain a channel-enhanced feature map; based on the channel-enhanced feature map, a spatial feature map is generated through channel-dimensional pooling operations, and then a spatial attention weight matrix is ​​obtained through convolution and activation processing; The channel enhancement feature map is spatially modulated using the spatial attention weight matrix to obtain an enhancement feature map that incorporates dual attention. Step 300: Post-processing optimization is performed on the preliminary detection prediction to obtain a refined detection box set; Step 400: Image the known spatial coordinate reference points calibrated on the physical plane of the sealing surface using a 150x imaging process to obtain the calibration image and extract the pixel coordinates of each reference point; based on the mapping relationship between the known spatial coordinates and pixel coordinates, establish a local distortion correction parameter lookup table for the imaging area of ​​the sealing surface; according to the center pixel coordinates of each detection frame in the refined detection frame set, query the local distortion correction parameter lookup table to obtain the corresponding local correction parameters; using the local correction parameters, perform coordinate mapping transformation on the vertex pixel coordinates and dimensions of each detection frame to obtain the precise position and size of each detection frame on the physical plane; based on the precise position and size of each detection frame on the physical plane, summarize to obtain the final defect detection data.

2. The method for image recognition and data processing of minute defects on the sealing surface of a ferrule connector according to claim 1, characterized in that, Step 200 further includes: The enhanced feature map fused with dual attention, along with other scale feature maps from the initial feature map set, are input into the optimized neck network for multi-scale feature fusion. The optimized neck network uses a feature pyramid architecture that includes upsampling, downsampling, and cross-layer connection operations to adaptively weight and aggregate information from the input features at multiple scales to obtain a multi-scale fused feature map for detection. The multi-scale fused feature map is input into the detection head, and bounding box regression calculation and category probability distribution calculation are performed respectively. The bounding box regression calculation generates the bounding box coordinates and size offset corresponding to each preset anchor point, and the category probability distribution calculation generates the defect category probability vector corresponding to each preset anchor point. By comprehensively decoding the bounding box coordinates and size offsets corresponding to all anchor points, as well as the defect category probability vector, a preliminary detection prediction is generated and output. The preliminary detection prediction includes the bounding box coordinates, confidence level, and the highest probability of the category to which each candidate box belongs.

3. The method for image recognition and data processing of minute defects on the sealing surface of a ferrule connector according to claim 2, characterized in that, Step 300 includes: Based on preset confidence thresholds and category probability thresholds, the preliminary detection predictions are filtered to obtain an initial candidate box set; Based on the defect category of each candidate box in the initial candidate box set, a preset basic cross-union ratio (CUI) threshold is obtained; based on the confidence level of the candidate boxes in the initial candidate box set and the spatial distribution density calculated according to the bounding box coordinates of the candidate boxes in the initial candidate box set, the basic CUI threshold is dynamically adjusted to generate an adaptive CUI threshold. Based on the adaptive intersection-union ratio threshold, iterative non-maximum suppression processing is performed on the initial candidate box set to obtain the suppressed candidate box set; Based on preset physical size range and distribution pattern parameters, the size and positional relationship of the suppressed candidate box set is verified, and the candidate boxes that pass the verification are output as a refined detection box set.

4. The method for image recognition and data processing of minute defects on the sealing surface of a ferrule connector according to claim 3, characterized in that, Step 400 includes: The 150x imaging process is used to obtain a calibration image containing known spatial coordinate reference points, and the pixel coordinates of each reference point in the calibration image are identified and extracted. Based on the pixel coordinates and the corresponding known spatial coordinates, the imaging area of ​​the sealing surface is divided into multiple continuous imaging sub-regions; for each imaging sub-region, the geometric transformation parameters of the sub-region from the image pixel coordinate system to the sealing surface physical coordinate system are calculated using the corresponding point pairs of the pixel coordinates and known spatial coordinates on its boundary; all imaging sub-regions and their corresponding geometric transformation parameters are collected to construct a local distortion correction parameter lookup table. Based on the center pixel coordinates of each detection box in the refined detection box set, the local distortion correction parameter lookup table is queried to obtain the corresponding local geometric transformation parameters as local correction parameters. Based on the local correction parameters, for each detection box in the refined detection box set, its vertex pixel coordinates and size are read, and the corresponding local correction parameters are called to construct a geometric transformation relationship; the geometric transformation relationship is used to perform coordinate mapping transformation on the vertex pixel coordinates and size to obtain the precise position and size of the detection box in the physical coordinate system of the sealing surface; Based on the precise position and size of each detection frame on the physical plane, the final defect detection data is generated by summarizing the physical position and physical size values.

5. A data processing system for image recognition of minute defects on the sealing surface of a ferrule connector, wherein the system implements the method as described in any one of claims 1 to 3, characterized in that, include: The preprocessing module is used to preprocess the original image of the ferrule connector surface obtained by magnification 150 times to obtain an enhanced image; The intelligent detection module is used to construct a feature extraction and detection model based on the attention-enhanced YOLOv8 network, to extract defect features and perform preliminary localization and identification on the enhanced image, and to obtain preliminary detection predictions including candidate boxes and categories. The feature extraction and detection model includes a backbone network, an optimized neck network, and a detection head. The optimization module is used to perform post-processing optimization on the preliminary detection prediction to obtain a refined detection box set; The geometric calibration module is used to image known spatial coordinate reference points calibrated on the physical plane of the sealing surface through a 150x imaging process, acquire calibration images, and extract the pixel coordinates of each reference point. Based on the mapping relationship between the known spatial coordinates and pixel coordinates, a local distortion correction parameter lookup table is established for the imaging area of ​​the sealing surface. According to the center pixel coordinates of each detection frame in the refined detection frame set, the local distortion correction parameter lookup table is queried to obtain the corresponding local correction parameters. Using the local correction parameters, coordinate mapping transformation is performed on the vertex pixel coordinates and dimensions of each detection frame to obtain the precise position and size of each detection frame on the physical plane. Based on the precise position and size of each detection frame on the physical plane, the final defect detection data is obtained by summarizing.

6. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 3.

Images