High-precision image detection method for micro drill blade surface based on deep learning

Abstract

The application discloses a high-precision image detection method for micro drill blade surface based on deep learning, comprising the following steps: S1, collecting visible light images and structured light images of the micro drill blade surface, and completing image preprocessing; S2, performing spatial alignment on the images, executing cross-modal fusion, and generating a feature fusion tensor; S3, inputting the feature fusion tensor into a multi-scale residual backbone network to extract a layered semantic feature set; S4, inputting the semantic feature set into three task branches of defect detection, region segmentation and type classification, and outputting corresponding prediction results; S5, calculating a multi-task loss function, dynamically adjusting the task branch weight, and optimizing the feature sharing structure; S6, generating a detection report according to the prediction results, and outputting defect coordinates, boundary contours, type labels and confidence values. The application realizes multi-modal fusion, high-precision identification and structured output of the micro drill blade surface, and significantly improves the accuracy, efficiency and automation level of defect detection.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a high-precision image detection method for micro-drill cutting surfaces based on deep learning. Background Technology

[0002] In fields such as high-density circuit board processing, precision hole machining, and micro-part manufacturing, micro-drills are key cutting tools, and the integrity of their cutting edge directly determines the machining quality and tool life. Currently, industrial sites mainly rely on manual visual inspection or traditional image algorithms to detect defects on the micro-drill cutting edge. Manual methods are limited by operator experience and suffer from problems such as strong subjectivity, low efficiency, and high false detection rate. On the other hand, traditional vision algorithms based on edge detection and grayscale analysis are difficult to accurately identify complex backgrounds, micro-cracks, and highly reflective surfaces.

[0003] Existing technologies attempt to automatically classify micro-drill images using convolutional neural networks, but most are based on single visible light images, ignoring the three-dimensional structural features of the cutting edge surface morphology. This leads to a significant drop in recognition performance when cracks are blurred or cutting edge damage is not significant. Furthermore, most existing models only achieve defect detection, failing to jointly implement cutting edge segmentation and defect type identification. They lack a complete multi-task learning architecture and cannot meet the comprehensive requirements of industrial online inspection for detection accuracy, efficiency, and completeness of detection information.

[0004] Furthermore, micro-drill cutting edge images are easily affected by changes in lighting, metal reflections, and micro-texture interference during acquisition. Image preprocessing algorithms often use fixed-parameter models and lack adaptive preprocessing procedures for micro-drilling conditions, leading to the loss of original feature information and affecting the model's discrimination ability. Meanwhile, multi-task deep learning networks suffer from inter-task interference issues in feature-sharing structure design, easily resulting in negative optimization where improved accuracy in one task leads to decreased performance in another.

[0005] Therefore, how to provide a high-precision image detection method for micro-drill cutting surfaces based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a high-precision image detection method for micro-drill cutting surfaces based on deep learning. This invention fully integrates visible light image and structured light image information, constructs a multi-scale residual backbone network and a multi-task branch structure, and describes in detail the entire process from image preprocessing, cross-modal fusion, feature extraction, defect localization to result output. It has the advantages of high detection accuracy, comprehensive recognition, and strong adaptability to complex industrial scenarios.

[0007] The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to embodiments of the present invention includes the following steps:

[0008] S1. Acquire visible light and structured light images of the micro-drill cutting edge and perform preprocessing;

[0009] S2. Spatial alignment is performed on the preprocessed visible light image and structured light image, and a cross-modal fusion operation based on channel feature matching is performed to align the information of the corresponding pixel regions at the channel level to obtain the feature fusion tensor.

[0010] S3. Input the feature fusion tensor into the backbone neural network. The backbone neural network uses a multi-scale residual structure to extract features from the fused image feature tensor and outputs a hierarchical semantic feature set.

[0011] S4. Input the hierarchical semantic feature set into three task branch networks, which are used to perform cutting surface defect detection, cutting surface region segmentation and defect type classification, respectively. Each task branch network extracts a feature subset and outputs the corresponding prediction result.

[0012] S5. Calculate the multi-task loss function jointly based on the feature subset, adjust the weight parameters of each task branch network, and optimize the feature sharing structure between the backbone neural network and each task branch network.

[0013] S6. Generate an inspection report for the micro-drill cutting edge based on the prediction results, and mark the defect location coordinates, area boundary contour, defect type label and corresponding confidence value.

[0014] Optionally, the visible light image refers to a two-dimensional image of the micro-drill cutting surface acquired by an industrial camera under natural lighting conditions, and the structured light image refers to a three-dimensional morphological image of the micro-drill cutting surface acquired using a structured light projection device under specific coded lighting conditions.

[0015] Optionally, the preprocessing includes image distortion correction, grayscale normalization, pixel-level alignment, boundary cropping, contrast stretching, edge sharpening, and noise filtering.

[0016] Optionally, S2 specifically includes:

[0017] S21. Set the preprocessed visible light image as a two-dimensional image and the structured light image as a three-dimensional image containing depth grayscale information, wherein the image coordinates include horizontal pixel positions and vertical pixel positions, and the structured light image also contains the grayscale value of each pixel.

[0018] S22. Use affine transformation to register the spatial coordinates of the two-dimensional and three-dimensional images, perform the transformations respectively, ensure that the corresponding pixels are consistent in spatial position, and output a pair of images with complete pixel alignment.

[0019] S23. Extract multiple channel feature vectors from the two-dimensional image and the three-dimensional image respectively, and determine the optimal channel correspondence by calculating the feature similarity between different channels. The feature similarity is calculated based on the direction and magnitude of the channel features.

[0020] S24. Perform a splicing and fusion operation on the channel features with high matching degree according to the channel dimension to generate a feature fusion tensor containing multi-source image information.

[0021] Optionally, the optimal channel correspondence is a channel matching relationship established based on the similarity matrix between the channel feature vectors of the visible light image and the channel feature vectors of the structured light image. A channel matching score matrix is ​​constructed using the similarity calculation results. A maximum matching strategy is executed in the channel matching score matrix to select the channel pair with the highest similarity score as the matching result. A set of channel index pairs is output, and the channel dimensions of the visible light image and the structured light image are fused and arranged according to the set of channel index pairs.

[0022] Optionally, S3 specifically includes:

[0023] S31. Input the feature fusion tensor into the backbone neural network. The three dimensions of the feature tensor correspond to the horizontal pixel position, vertical pixel position and fusion channel dimension of the image, respectively.

[0024] S32. The backbone neural network is set up with a multi-scale structure composed of multiple residual units. Each residual unit receives the feature output of the previous unit in sequence, performs convolution transformation, batch normalization and nonlinear activation function combination operation, and adds the input features to the output features through identity jump connection to form a short connection structure.

[0025] S33. Perform downsampling processing on the feature map output by each residual unit to extract semantic features at different receptive field scales and form a multi-level feature map set.

[0026] S34. The feature maps output by all residual units are concatenated along the channel dimension to form a hierarchical semantic feature set, and the hierarchical semantic feature set is passed to the task branch network.

[0027] Optionally, the hierarchical semantic feature set refers to a tensor set formed by arranging feature maps output from multiple scale levels in sequence according to semantic granularity, with each layer of feature maps corresponding to image semantic information under different receptive fields.

[0028] Optionally, S4 specifically includes:

[0029] S41. Represent the hierarchical semantic feature set as a three-dimensional tensor, and input it in parallel along the channel dimension to the defect detection branch network, the blade region segmentation branch network and the defect type classification branch network. Each branch extracts an independent feature subset for task computation.

[0030] S42. The defect detection branch network extracts a subset of defect features from the three-dimensional tensor, obtains a low-dimensional feature tensor through a channel-compressed convolutional structure, and outputs the probability value of whether each pixel is a defect by channel-weighted summation.

[0031] S43. The blade surface region segmentation branch network extracts a subset of blade surface features from the three-dimensional tensor, performs multi-scale upsampling operation on the subset of blade surface features, generates upsampled feature maps at three scales, splices all scale feature maps in the channel dimension to form a fusion tensor, and outputs the region category distribution of each pixel through a classification convolutional layer.

[0032] S44. The defect type classification branch network extracts a subset of type features from the three-dimensional tensor, performs full-image average pooling on the subset of type features, extracts channel-level global feature vectors, and inputs them into a fully connected neural network to calculate the probability distribution of each defect type and output the defect classification prediction result.

[0033] Optionally, S5 specifically includes:

[0034] S51. Based on the output results of the three task branches of defect detection, blade region segmentation and defect type classification, calculate the loss function for the corresponding task. The defect detection task uses pixel-level binary cross-entropy loss, the region segmentation task uses multi-class cross-entropy loss, and the defect classification task uses single-label multi-class cross-entropy loss. Each loss function is calculated pixel-by-pixel or category-by-category based on the true label and the prediction result.

[0035] S52. Assign adjustable weight coefficients to each task branch. Construct a joint multi-task loss function by weighting the loss functions of the three tasks. The sum of all weight coefficients is 1. The weight values ​​are adaptively adjusted according to the task gradient distribution during training to ensure that the learning effect of each task is balanced.

[0036] S53. Backpropagation is performed based on the joint multi-task loss function, while optimizing the parameters of the backbone neural network and the task branch network. The feature transfer structure maintains the generality of the underlying layer and implements the separation and sharing strategy of task-specific features in the middle and high layers. The learning efficiency of the entire network structure is optimized through the end-to-end parameter update mechanism.

[0037] Optionally, S6 specifically includes:

[0038] S61. The pixel-level defect probability output by the defect detection task branch is binarized, and the high-confidence pixel region is extracted as a defect region mask by combining the set threshold. The minimum bounding rectangle of the boundary of all independent defect regions is obtained by connected component analysis, and the two-dimensional coordinate information of the corresponding position is extracted as the defect position coordinate.

[0039] S62. Perform an intersection operation between the segmentation probability prediction value output by the blade region segmentation task branch and the defect region mask to further extract the structural category information of the defect region, and use the contour tracking algorithm to perform polygon fitting operation on the outer contour of each defect region to output the closed region boundary contour composed of a continuous set of boundary points.

[0040] S63. Based on the defect category prediction vector output by the defect type classification task branch, select the maximum probability index value to correspond to the defect label set as the defect type label, and use the maximum probability index value as the confidence value, and combine it with the defect location coordinates and boundary contour to form a structured description vector.

[0041] S64. Write the structured description vectors of all defects into the detection report text file according to the preset format. The report content includes: defect number, location coordinates, structural region attribution, boundary contour point set, defect type label and corresponding confidence value, and is indexed and sorted according to image number.

[0042] The beneficial effects of this invention are:

[0043] First, by introducing a cross-modal fusion mechanism between structured light images and visible light images, this invention constructs a joint feature tensor with channel feature alignment, which significantly enhances the perception of the micro-morphology and texture details of the cutting edge. This solves the problem of information loss in high-reflectivity and boundary-blurred scenes by a single image modality, and improves the robustness of cutting edge defect recognition and fine-grained feature extraction capabilities.

[0044] Secondly, this invention designs a multi-scale residual structure in the backbone network, which, combined with three task branch networks, forms a joint optimization architecture, enabling the simultaneous execution of three tasks: defect detection, region segmentation, and defect type classification. The multi-task feature sharing mechanism enhances the model's expressive power under complex tasks, effectively avoiding gradient conflicts and information redundancy between tasks, thus achieving a balance between training efficiency and overall detection performance.

[0045] Finally, based on the prediction results, this invention constructs a structured inspection output process, which automatically generates an inspection report containing defect location, boundary contour, defect type and confidence level. This meets the actual needs of industrial inspection for visualization, traceability and quantification, and has a high degree of automation and engineering deployment friendliness, providing an intelligent solution for micro-drill tool condition monitoring. Attached Figure Description

[0046] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0047] Figure 1 This is a flowchart of the high-precision image detection method for micro-drill cutting surfaces based on deep learning proposed in this invention;

[0048] Figure 2 This is a flowchart of the cross-modal image feature fusion sub-process of the high-precision image detection method for micro-drill cutting surfaces based on deep learning proposed in this invention.

[0049] Figure 3 This is a multi-task branch structure diagram of the high-precision image detection method for micro-drill cutting surfaces based on deep learning proposed in this invention. Detailed Implementation

[0050] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0051] refer to Figure 1-3 A high-precision image detection method for micro-drill cutting surfaces based on deep learning includes the following steps:

[0052] S1. Acquire visible light and structured light images of the micro-drill cutting edge and perform preprocessing;

[0053] S2. Spatial alignment is performed on the preprocessed visible light image and structured light image, and a cross-modal fusion operation based on channel feature matching is performed to align the information of the corresponding pixel regions at the channel level to obtain the feature fusion tensor.

[0054] S3. Input the feature fusion tensor into the backbone neural network. The backbone neural network uses a multi-scale residual structure to extract features from the fused image feature tensor and outputs a hierarchical semantic feature set.

[0055] S4. Input the hierarchical semantic feature set into three task branch networks, which are used to perform cutting surface defect detection, cutting surface region segmentation and defect type classification, respectively. Each task branch network extracts a feature subset and outputs the corresponding prediction result.

[0056] S5. Calculate the multi-task loss function jointly based on the feature subset, adjust the weight parameters of each task branch network, and optimize the feature sharing structure between the backbone neural network and each task branch network.

[0057] S6. Generate an inspection report for the micro-drill cutting edge based on the prediction results, and mark the defect location coordinates, area boundary contour, defect type label and corresponding confidence value.

[0058] This invention constructs a complete six-step deep learning detection process, realizing multimodal acquisition of micro-drill cutting edge images, fusion feature extraction, multi-task recognition, and automatic report output. It improves the accuracy of defect recognition, the coordination of task execution, and the automation of result output, and is suitable for high-precision tool quality assessment in complex industrial inspection environments.

[0059] In this embodiment, the visible light image refers to a two-dimensional image of the micro-drill cutting edge area captured by an industrial-grade color area array camera under ambient natural light or stable white light illumination conditions, at a fixed angle and constant focal length. The image resolution is greater than 2 million pixels, the image format is RGB three-channel structure, and the image frames are transmitted to the processing unit in real time via a gigabit Ethernet interface. The structured light image refers to a sequence of reflected interference images acquired by a high dynamic range image acquisition device under controlled illumination angles after the micro-drill cutting edge is illuminated by a structured light coding projection device with a regular striped light field. The depth value of each pixel in the image sequence is calculated by a spatial structured light triangulation algorithm and mapped to a grayscale image, ultimately generating a three-dimensional morphology image reflecting the microscopic contour changes of the micro-drill cutting edge. The three-dimensional morphology image is stored in grayscale single-channel format and synchronized with the visible light image number.

[0060] This invention combines visible light and structured light images, integrating two-dimensional texture and three-dimensional shape information, so that the blade surface can still be accurately characterized under conditions such as reflection, dirt, or blurred outline, thus improving the ability to identify micro-cracks and minor chipping.

[0061] In this embodiment, the image preprocessing steps include the following seven consecutive processing flows: First, image distortion correction is performed, and the visible light image is remapped at the pixel level for radial and tangential distortion based on camera calibration parameters; second, grayscale normalization is performed, and all pixel grayscale values ​​are uniformly adjusted to the closed interval [0,1] using a linear transformation method; third, a sub-pixel level alignment algorithm based on multi-feature template matching is used to perform matching registration operations on the visible light image and the structured light image in translation, rotation, and scale space; subsequently, the image boundary region is cropped proportionally to eliminate projection boundary errors and background interference regions; after cropping, the image contrast is stretched using a linear piecewise function to improve the structural clarity of the intermediate grayscale region; then, the image edge information is enhanced using a Laplacian sharpening convolution kernel to highlight the contour boundaries; finally, the image is noise filtered using a combined bilateral filtering and median filtering method to preserve edge details and remove high-frequency artifacts, thereby enhancing image quality.

[0062] This invention enhances the clarity and stability of input images by establishing a systematic image preprocessing workflow, effectively suppresses noise interference, and unifies image scale and brightness conditions, providing a standardized input basis for subsequent feature extraction and model training.

[0063] In this embodiment, S2 specifically includes:

[0064] S21. Record the preprocessed visible light image as a two-dimensional image. Structured light images are recorded as three-dimensional images. ,in Indicates the horizontal pixel position of the image. Indicates the vertical pixel position of the image. Represents the grayscale value of a structured light image channel;

[0065] S22, Regarding the image and Pixel-level spatial registration is performed using an affine transformation function. Perform spatial alignment operations to obtain aligned image pairs. and The Defined as:

[0066] ;

[0067] in, The linear transformation coefficients of the two-dimensional transformation matrix. These are translation parameters;

[0068] S23. Align the image... The input is fed into the channel feature matching unit to extract the channel feature sets of the visible light image respectively. and structured light image channel feature set The matching degree between channels is calculated using the cosine similarity function:

[0069] ;

[0070] in, Indicates the first The eigenvector of the visible light channel and the eigenvalue of the visible light channel The similarity of the feature vectors of each structured light channel. Indicates the first One visible light channel feature vector, Indicates the first There are 1 structured light channel feature vectors, where || is the vector magnitude and represents the modulus operation;

[0071] S24. Establish the optimal channel correspondence based on the similarity of channel feature vectors, and concatenate and fuse the matched feature vectors along the channel dimension to construct a feature fusion tensor:

[0072] ;

[0073] in, Represents the feature fusion tensor. Indicates the channel dimension index. This is the index of the selected channel in the correspondence.

[0074] This invention improves the coupling degree of multi-source image features and enhances image expressiveness by performing pixel-level spatial alignment, channel-level feature matching, and cross-modal fusion operations, effectively improving feature extraction performance in low-contrast regions and asymmetric structures.

[0075] In this embodiment, the optimal channel correspondence is a channel matching relationship established based on the similarity matrix between the feature vectors of the visible light image channel and the feature vectors of the structured light image channel. The similarity calculation results are used to construct a channel matching score matrix. The maximum matching strategy is executed in the channel matching score matrix, and the channel pair with the highest similarity score is selected as the matching result. The channel index pair set is output, and the channel dimensions of the visible light image and the structured light image are fused and arranged according to the channel index pair set.

[0076] This invention employs a maximum matching strategy based on the similarity matrix to achieve channel-level fusion sorting, enabling the fusion features to have matching consistency in structural expression, improving the complementary expression quality between different image channels, and providing highly consistent input for the subsequent backbone network to extract unified semantics.

[0077] In this embodiment, S3 specifically includes:

[0078] S31. Obtain the feature fusion tensor ,in Indicates the horizontal pixel position of the image. Indicates the vertical pixel position of the image. The channel dimension index is used to input the feature fusion tensor into the backbone neural network for feature processing.

[0079] S32. A multi-scale residual structure is set in the backbone neural network, wherein the multi-scale residual structure is composed of... It consists of several residual cascaded units, each of which receives the feature tensor output by the previous unit and performs the following operations:

[0080] Define the residual input as The output is ,in Indicates the first Each residual unit number, For the input channel dimension, Output channel dimension;

[0081] Each residual unit performs a composite operation of convolution transformation, batch normalization, and nonlinear activation, calculated as follows:

[0082] ;

[0083] in, For the first The convolution kernel weights in each residual unit This represents the convolution operation. This indicates a batch normalization operation. Represents the ReLU activation function;

[0084] S33. Perform downsampling on the feature map output by each residual unit to extract semantic features at different scales and form a multi-level feature map set.

[0085] S34. The feature maps output by all residual units are concatenated along the channel dimension to form a hierarchical semantic feature set, and the hierarchical semantic feature set is passed to the task branch network.

[0086] This invention extracts semantic feature information from different receptive fields through multi-scale residual structures, realizing hierarchical modeling from low-level texture details to high-level semantic representation, enhancing the robustness of the model to complex morphological changes of the cutting edge, and improving the discriminative power of deep feature expression.

[0087] In this embodiment, the hierarchical semantic feature set refers to a multi-level semantic tensor set formed by convolving the input feature fusion tensor at different scales through multiple residual units to extract feature maps corresponding to edge texture, scale structure and global contour respectively, and stacking them in order from small to large scale.

[0088] This invention constructs a hierarchical semantic feature set, ensuring the complete transmission of multi-scale features in subsequent tasks, further enhancing the network's ability to jointly perceive the fine-grained structure of the cutting edge and the global contour, and providing more comprehensive feature support for task branches.

[0089] In this embodiment, S4 specifically includes:

[0090] S41. The semantic feature set is defined as a tensor. ,in Represents the horizontal pixel coordinates of the image. Represents the vertical pixel coordinates of the image. Indicates the channel dimension index, Simultaneously transmit to the three task branch networks;

[0091] S42. In the defect detection task branch, from the tensor Extracting defect feature subsets The input is fed into the feature compression convolution unit to generate a compressed tensor. Execute the following pixel-level prediction function:

[0092] ;

[0093] in, Indicates pixel position Defect probability prediction value at the location, In the compression tensor, the first Pixel values ​​of the channel, Indicates the number of input feature channels. Indicates the number of channels after compression. For the first The convolution kernel coefficients of the channels, The bias constant is Use the Sigmoid activation function;

[0094] S43. In the task branch of dividing the cutting edge region, from the tensor Extracting the blade surface feature subset Perform three-scale spatial upsampling operations to generate upsampled feature tensors respectively. , , ,in , , The channel dimensions, representing the three scales respectively, are combined as follows:

[0095] ;

[0096] in, , indicating the dimension of the fusion channel;

[0097] Will Input is fed into a classification convolutional layer, and the output is a predicted segmentation probability value. ,in Indicates a region category index;

[0098] S44. In the defect type classification task branch, from tensor Extracting type feature subsets Perform a channel-level global averaging operation to calculate the global feature vector:

[0099] ;

[0100] in, Indicates the channel number. , These are the image height and width, respectively. Indicates the first The global average value of the channels is ultimately used to generate a defect category prediction vector through a fully connected structure:

[0101] ;

[0102] in, This indicates the predicted defect category distribution. This is the weight matrix of the fully connected layer. For the bias vector of the fully connected layer, This represents the activation function.

[0103] This invention achieves simultaneous execution of defect detection, region segmentation, and type classification by constructing a three-branch parallel multi-task learning structure. This not only improves the overall computational efficiency but also enhances the accuracy and stability of the prediction results of each task through task cooperation.

[0104] In this embodiment, S5 specifically includes:

[0105] S51. Obtain the prediction outputs of the three task branch networks respectively. Let the defect detection output be... Region segmentation output is Defect type classification output is Calculate the single-task loss function for each of the three factors and their corresponding real labels:

[0106] The defect detection loss function is defined as pixel-level binary cross-entropy loss:

[0107] ;

[0108] in, This is a genuine defect label. This is the predicted defect probability value. These are the image height and width, respectively. Represents a logarithmic function;

[0109] The region segmentation loss function is defined as multi-class cross-entropy:

[0110] ;

[0111] in, For the true label of region segmentation, The predicted value of the segmentation probability. Indicates the number of categories of the cutting edge region;

[0112] The defect type classification loss function is defined as the single-label multi-class cross-entropy:

[0113] ;

[0114] in, For the first The true label of class defects To predict probabilities, Indicates the number of defect types;

[0115] S52, Introducing Task Weighting Coefficients , , The joint multi-task loss function is defined as follows:

[0116] ;

[0117] in, Each coefficient is dynamically adjusted in each round of training based on the magnitude of the task gradient;

[0118] S53. Based on the joint loss function Perform backpropagation and gradient descent operations on all parameters of the backbone neural network and the three task branch networks to update the parameter set of the backbone neural network. With task branch parameter set Simultaneously optimize the feature sharing structure between channels, so that each task branch can share low-level structural features while maintaining differences in high-level structures.

[0119] This invention sets up a multi-task loss function and its dynamic weighting mechanism, optimizes the collaborative training process between tasks, improves the learning ability of the network under complex multi-objective tasks, and enhances the overall model's adaptive adjustment ability between different tasks through an end-to-end parameter update mechanism.

[0120] In this embodiment, S6 specifically includes:

[0121] S61. The pixel-level defect probability output by the defect detection task branch is binarized, and the high-confidence pixel region is extracted as a defect region mask by combining the set threshold. The minimum bounding rectangle of the boundary of all independent defect regions is obtained by connected component analysis, and the two-dimensional coordinate information of the corresponding position is extracted as the defect position coordinate.

[0122] S62. Perform an intersection operation between the segmentation probability prediction value output by the blade region segmentation task branch and the defect region mask to further extract the structural category information of the defect region, and use the contour tracking algorithm to perform polygon fitting operation on the outer contour of each defect region to output the closed region boundary contour composed of a continuous set of boundary points.

[0123] S63. Based on the defect category prediction vector output by the defect type classification task branch, select the maximum probability index value to correspond to the defect label set as the defect type label, and use the maximum probability index value as the confidence value, and combine it with the defect location coordinates and boundary contour to form a structured description vector.

[0124] S64. Write the structured description vectors of all defects into the detection report text file according to the preset format. The report content includes: defect number, location coordinates, structural region attribution, boundary contour point set, defect type label and corresponding confidence value, and is indexed and sorted according to image number.

[0125] This invention, based on a structured output test report format, enables standardized description and visual expression of defect information. It has advantages such as traceability of test results, archiving of data, and automatic report generation, significantly improving the operability and practicality of defect detection in industrial settings.

[0126] Example 1:

[0127] To verify the feasibility of this invention in practice, it was applied to a micro-drill tool quality inspection production line to perform high-precision defect detection and quality assessment on the cutting edges of different batches of micro-drills. In this scenario, the micro-drill tools used are mainly used for high-precision circuit board hole machining. The tool diameter is between 0.1mm and 0.5mm, and their cutting edge structure is complex, the boundaries are weak, and defects such as micro-cracks, chipping, and wear are prone to occur. Traditional image recognition methods have problems such as high edge detection miss rate, inaccurate classification, and unclear contour extraction. Moreover, manual visual inspection has low efficiency and poor consistency, making it difficult to meet the high-standard quality inspection requirements for large-volume tool shipments.

[0128] The detection system constructed in this invention is deployed in the cutting edge detection stage. It uses an industrial camera and structured light acquisition equipment to simultaneously acquire visible light images and three-dimensional topographic images of each micro-drill tool. After acquisition, the images undergo unified preprocessing, including distortion correction, normalization, edge enhancement, and noise filtering, to improve overall image clarity and contrast. Subsequently, cross-modal image fusion is achieved through spatial registration and channel feature alignment mechanisms to generate a fused feature tensor. This fused tensor is then input into the backbone neural network for multi-scale feature extraction, and three branches of the network are used to perform defect region detection, cutting edge region segmentation, and defect type classification, respectively.

[0129] During operation, the system trained and validated a dataset of 14,200 images targeting five typical defect types: microcracks, chipping, wear, burrs, and mixed defects. The images were enhanced with varying lighting and angles to ensure model generalization ability. To verify the performance advantages of this invention, two comparative methods were introduced: Method A, a traditional image processing + SVM classification model, and Method B, a single-modal deep convolutional detection model. The experiments used a unified hardware platform and evaluation metrics, including defect recognition accuracy, contour segmentation accuracy, average detection time, repeatability consistency, and false detection rate.

[0130] The results show that the present invention outperforms the comparative methods in several key performance aspects. Regarding defect detection accuracy, traditional method A averages only 82.4%, method B improves to 91.7%, while the present invention reaches 97.6%. In terms of contour segmentation accuracy, the present invention achieves an average IoU of 0.874, significantly better than the comparative scheme. The average inspection time per tool is 0.41 seconds, meeting the requirements of actual production lines. In 200 repeated inspection experiments, the present invention maintained a detection consistency rate of 99.2% on the same samples, far exceeding the approximately 85% level of manual visual inspection. In the identification of small defects (diameter < 50 μm), the detection rate of the present invention is 93.4%, an improvement of more than 10 percentage points compared to method B, verifying its advantages in microstructure identification.

[0131] Based on the above performance evaluation indicators, the following are the key performance comparison data of this invention in the intelligent detection scenario of micro-drill cutting surfaces:

[0132] Table 1 Performance Comparison of Micro-Drill Cutting Edge Detection Methods

[0133]

[0134] As can be seen from the above implementation results, the present invention is significantly superior to traditional methods in terms of detection accuracy, fine-grained recognition capability and computational efficiency. At the same time, it enhances the engineering adaptability of the detection report through structured result output, effectively solving the key problems of "high false negative rate, single recognition and coarse output" of traditional detection methods, and has the ability to be widely promoted and applied in complex manufacturing scenarios.

[0135] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A high-precision image detection method for micro-drill cutting surfaces based on deep learning, characterized in that, Includes the following steps: S1. Acquire visible light and structured light images of the micro-drill cutting edge and perform preprocessing; S2. Spatial alignment is performed on the preprocessed visible light image and structured light image, and a cross-modal fusion operation based on channel feature matching is performed to align the information of the corresponding pixel regions at the channel level to obtain the feature fusion tensor. S3. Input the feature fusion tensor into the backbone neural network. The backbone neural network uses a multi-scale residual structure to extract features from the fused image feature tensor and outputs a hierarchical semantic feature set. S4. Input the hierarchical semantic feature set into three task branch networks, which are used to perform cutting surface defect detection, cutting surface region segmentation and defect type classification, respectively. Each task branch network extracts a feature subset and outputs the corresponding prediction result. S5. Calculate the multi-task loss function jointly based on the feature subset, adjust the weight parameters of each task branch network, and optimize the feature sharing structure between the backbone neural network and each task branch network. S6. Generate an inspection report for the micro-drill cutting edge based on the prediction results, and mark the defect location coordinates, area boundary contour, defect type label and corresponding confidence value.

2. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 1, characterized in that, The visible light image refers to a two-dimensional image of the micro-drill cutting surface acquired by an industrial camera under natural lighting conditions, while the structured light image refers to a three-dimensional morphological image of the micro-drill cutting surface acquired using a structured light projection device under specific coded lighting conditions.

3. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 1, characterized in that, The preprocessing includes image distortion correction, grayscale normalization, pixel-level alignment, boundary cropping, contrast stretching, edge sharpening, and noise filtering.

4. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 1, characterized in that, S2 specifically includes: S21. Set the preprocessed visible light image as a two-dimensional image and the structured light image as a three-dimensional image containing depth grayscale information, wherein the image coordinates include horizontal pixel positions and vertical pixel positions, and the structured light image also contains the grayscale value of each pixel. S22. Use affine transformation to register the spatial coordinates of the two-dimensional and three-dimensional images, perform the transformations respectively, ensure that the corresponding pixels are consistent in spatial position, and output a pair of images with complete pixel alignment. S23. Extract multiple channel feature vectors from the two-dimensional image and the three-dimensional image respectively, and determine the optimal channel correspondence by calculating the feature similarity between different channels. The feature similarity is calculated based on the direction and magnitude of the channel features. S24. Perform a splicing and fusion operation on the channel features with high matching degree according to the channel dimension to generate a feature fusion tensor containing multi-source image information.

5. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 4, characterized in that, The optimal channel correspondence is a channel matching relationship established based on the similarity matrix between the channel feature vectors of the visible light image and the channel feature vectors of the structured light image. The similarity calculation results are used to construct a channel matching score matrix. The maximum matching strategy is executed in the channel matching score matrix, and the channel pair with the highest similarity score is selected as the matching result. The channel index pair set is output, and the channel dimensions of the visible light image and the structured light image are fused and arranged according to the channel index pair set.

6. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 1, characterized in that, S3 specifically includes: S31. Input the feature fusion tensor into the backbone neural network. The three dimensions of the feature tensor correspond to the horizontal pixel position, vertical pixel position and fusion channel dimension of the image, respectively. S32. The backbone neural network is set up with a multi-scale structure composed of multiple residual units. Each residual unit receives the feature output of the previous unit in sequence, performs convolution transformation, batch normalization and nonlinear activation function combination operation, and adds the input features to the output features through identity jump connection to form a short connection structure. S33. Perform downsampling processing on the feature map output by each residual unit to extract semantic features at different receptive field scales and form a multi-level feature map set. S34. The feature maps output by all residual units are concatenated along the channel dimension to form a hierarchical semantic feature set, and the hierarchical semantic feature set is passed to the task branch network.

7. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 6, characterized in that, The hierarchical semantic feature set refers to a tensor set formed by arranging feature maps output from multiple scale levels in sequence according to semantic granularity. Each layer of feature maps corresponds to the semantic information of the image under different receptive fields.

8. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 1, characterized in that, S4 specifically includes: S41. Represent the hierarchical semantic feature set as a three-dimensional tensor, and input it in parallel along the channel dimension to the defect detection branch network, the blade region segmentation branch network and the defect type classification branch network. Each branch extracts an independent feature subset for task computation. S42. The defect detection branch network extracts a subset of defect features from the three-dimensional tensor, obtains a low-dimensional feature tensor through a channel-compressed convolutional structure, and outputs the probability value of whether each pixel is a defect by channel-weighted summation. S43. The blade surface region segmentation branch network extracts a subset of blade surface features from the three-dimensional tensor, performs multi-scale upsampling operation on the subset of blade surface features, generates upsampled feature maps at three scales, splices all scale feature maps in the channel dimension to form a fusion tensor, and outputs the region category distribution of each pixel through a classification convolutional layer. S44. The defect type classification branch network extracts a subset of type features from the three-dimensional tensor, performs full-image average pooling on the subset of type features, extracts channel-level global feature vectors, and inputs them into a fully connected neural network to calculate the probability distribution of each defect type and output the defect classification prediction result.

9. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 1, characterized in that, S5 specifically includes: S51. Based on the output results of the three task branches of defect detection, blade region segmentation and defect type classification, calculate the loss function for the corresponding task. The defect detection task uses pixel-level binary cross-entropy loss, the region segmentation task uses multi-class cross-entropy loss, and the defect classification task uses single-label multi-class cross-entropy loss. Each loss function is calculated pixel-by-pixel or category-by-category based on the true label and the prediction result. S52. Assign adjustable weight coefficients to each task branch. Construct a joint multi-task loss function by weighting the loss functions of the three tasks. The sum of all weight coefficients is 1. The weight values ​​are adaptively adjusted according to the task gradient distribution during training to ensure that the learning effect of each task is balanced. S53. Backpropagation is performed based on the joint multi-task loss function, while optimizing the parameters of the backbone neural network and the task branch network. The feature transfer structure maintains the generality of the underlying layer and implements the separation and sharing strategy of task-specific features in the middle and high layers. The learning efficiency of the entire network structure is optimized through the end-to-end parameter update mechanism.

10. The high-precision image detection method for micro-drill cutting surfaces based on deep learning according to claim 1, characterized in that, S6 specifically includes: S61. The pixel-level defect probability output by the defect detection task branch is binarized, and the high-confidence pixel region is extracted as a defect region mask by combining the set threshold. The minimum bounding rectangle of the boundary of all independent defect regions is obtained by connected component analysis, and the two-dimensional coordinate information of the corresponding position is extracted as the defect position coordinate. S62. Perform an intersection operation between the segmentation probability prediction value output by the blade region segmentation task branch and the defect region mask to further extract the structural category information of the defect region, and use the contour tracking algorithm to perform polygon fitting operation on the outer contour of each defect region to output the closed region boundary contour composed of a continuous set of boundary points. S63. Based on the defect category prediction vector output by the defect type classification task branch, select the maximum probability index value to correspond to the defect label set as the defect type label, and use the maximum probability index value as the confidence value, and combine it with the defect location coordinates and boundary contour to form a structured description vector. S64. Write the structured description vectors of all defects into the detection report text file according to the preset format. The report content includes: defect number, location coordinates, structural region attribution, boundary contour point set, defect type label and corresponding confidence value, and is indexed and sorted according to image number.

Images