Fine linear defect segmentation method

Abstract

The invention relates to the technical field of image processing and computer vision, and discloses a fine linear defect segmentation method. The method comprises the following steps: carrying out serial detail enhancement processing and nonlinear contrast amplification processing on a to-be-processed image to obtain a pre-enhanced image; inputting the pre-enhanced image into a pre-constructed and trained deep learning image segmentation network, wherein the deep learning image segmentation network outputs a binary segmentation mask image corresponding to the fine linear defects; according to the method, through two stages of pre-enhancement processing in series connection, fine and light linear defects which are originally difficult to recognize can become relatively significant in the image, and the enhanced image provides stronger defect features which are easier to learn for deep learning of the image segmentation network; therefore, the problem that the deep learning image segmentation network is difficult to converge or poor in convergence effect when processing such defects is effectively solved, and stable convergence of the deep learning image segmentation network can be realized when the enhanced image preprocessed by the method is used for training.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and computer vision, and in particular to a method for segmenting subtle linear defects. Background Art

[0002] In numerous fields, including industrial inspection, materials science, and medical imaging, accurate and automated detection and segmentation of subtle linear defects such as shallow scratches, microcracks, and fibers within products or samples is crucial. However, these defects are often characterized by small width, low contrast with the background, and irregular shapes, posing significant challenges to automated segmentation.

[0003] Existing image segmentation methods primarily include traditional image processing methods and deep learning-based segmentation network methods. Traditional image processing methods, primarily including threshold segmentation, edge detection, and morphological operations, are somewhat effective for defects with distinct features and high contrast. However, they often lack sensitivity for detecting and segmenting subtle, faint linear defects, leading to missed or false detections. Furthermore, traditional image processing methods require complex parameter adjustments and exhibit poor robustness, making them unsuitable for large-scale applications.

[0004] Deep learning-based segmentation network methods, primarily semantic segmentation networks such as U-Net, SegNet, and DeepLab, have achieved remarkable success in image segmentation in recent years. However, when these existing deep learning-based segmentation network methods are directly applied to images containing subtle defects such as fine, faint lines, these networks often face training failures or poor convergence.

[0005] The reasons are:

[0006] 1. Weak features: Fine linear defects account for a small proportion of pixels in the image, and their features are not prominent, making it difficult for the network to learn effective discriminative features.

[0007] 2. Gradient vanishing / exploding: In deep networks, weak defect signals may gradually disappear during the back-propagation process, resulting in the inability to effectively update network parameters.

[0008] 3. Sample imbalance: There may be a serious imbalance between the number of defective pixels and background pixels, further increasing the difficulty of training.

[0009] The above reasons lead to the technical bottleneck of low segmentation accuracy and ineffective network training when the existing standard segmentation network processes such subtle and faint linear defects, and often fails to obtain satisfactory results.

[0010] Therefore, a fine linear defect segmentation method is urgently needed to overcome the above defects. Summary of the Invention

[0011] The object of the present invention is to provide a method for segmenting fine linear defects to solve or at least partially solve the technical problems existing in the prior art.

[0012] To achieve this object, the present invention adopts the following technical solutions:

[0013] The present invention provides a method for segmenting fine linear defects, which is suitable for performing defect segmentation on an image to be processed that has fine linear defects. The method for segmenting fine linear defects comprises the following steps:

[0014] S1, performing a series detail enhancement process and a nonlinear contrast amplification process on the image to be processed to obtain a pre-enhanced image;

[0015] S2. Input the pre-enhanced image into a pre-built and trained deep learning image segmentation network, and the deep learning image segmentation network outputs a binary segmentation mask image of the subtle linear defect.

[0016] Preferably, the step S1 specifically includes:

[0017] S11, performing detail enhancement processing based on local information difference on the image to be processed to obtain a first-stage enhanced image;

[0018] S12: Perform nonlinear contrast amplification processing based on self-multiplication of pixel values ​​on the first-stage enhanced image to obtain the pre-enhanced image.

[0019] Specifically, the step S11 includes:

[0020] S111, calculating the image to be processed I orig The local mean image I of (x, y) mean (x, y);

[0021] S112, calculating the image to be processed I orig (x, y) and the local mean image I mean The difference between (x, y) is used to obtain the first pixel-level difference image I diff (x, y);

[0022] S113, the first pixel level difference image I diff After (x, y) is multiplied by the preset enhancement factor k, weighted amplification is performed to obtain the second pixel-level difference image I weighted_diff (x, y);

[0023] S114: the second pixel level difference image I weighted_diff (x, y) and the image to be processed I orig(x, y) is processed by pixel-level superposition to obtain the first result image I stage1_pre (x, y);

[0024] S115: the first result image I stage1_pre (x, y) is subjected to grayscale value clipping to obtain the first stage enhanced image I stage1_out (x, y), the first stage enhanced image I stage1_out The pixel value of (x, y) is within the preset pixel value valid range.

[0025] Specifically, the step S111 includes:

[0026] Through the mean filter f with preset mask width W and mask height H mean (Image, W, H) calculates the image to be processed I orig The local mean image I of (x, y) mean (x, y), the calculation formula is:

[0027] I mean (x, y) = f mean (I orig (x, y), W, H).

[0028] Preferably, the step S12 specifically includes:

[0029] S121, enhancing the first stage image I stage1_out (x, y) performs pixel-level self-value multiplication processing to obtain a second result image;

[0030] S122: Perform grayscale clipping on the second result image to obtain the pre-enhanced image I. enhanced (x, y), the pre-enhanced image I enhanced The pixel value of (x, y) is within the preset pixel value valid range.

[0031] Specifically, the first stage enhanced image I is obtained by the following formula: stage1_out (x, y) performs pixel-level self-value multiplication processing:

[0032] I stage2_pre (x, y) = I stage1_out (x, y)·I stage1_out (x, y)·Mult+Add,

[0033] Among them, I stage2_pre (x, y) is the second result image, Mult is the multiplication coefficient, and Add is the addition offset.

[0034] Specifically, the first result image I is clipped by the clipping function clip(v, L, U) stage1_pre (x, y) is subjected to grayscale value clipping to obtain the first stage enhanced image I stage1_out (x, y), the first stage enhanced image I stage1_out The pixel value of (x, y) is within the preset pixel value valid range [L, U];

[0035] The second result image I is clipped by the clipping function clip(v, L, U) stage2_pre (x, y) is subjected to grayscale value clipping to obtain the pre-enhanced image I enhanced (x, y), the pre-enhanced image I enhanced The pixel value of (x, y) is within the preset pixel value valid range [L, U];

[0036] Wherein, the clipping function clip(v, L, U) is defined as:

[0037]

[0038] Preferably, the step S2 specifically includes:

[0039] S21, the pre-enhanced image I enhanced (x, y) input to a pre-built and trained deep learning image segmentation network In the deep learning image segmentation network According to the pre-enhanced image I enhanced (x, y) output segmentation mask image M pred (x, y), where θ is the deep learning image segmentation network The network parameters of the deep learning image segmentation network is a convolutional neural network architecture for segmentation tasks, the segmentation mask image M pred The formula for (x, y) is defined as:

[0040]

[0041] S22, the segmentation mask image M is processed by the following function pred (x, y) is subjected to binary segmentation processing to obtain the binary segmentation mask image M binary (x, y):

[0042]

[0043] Among them, T seg is a preset threshold, the binary segmentation mask image M binaryWhen (x, y) is 1, the binary segmentation mask image M binary (x, y) represents the fine linear defect part, and the binary segmentation mask image M binary When (x, y) is 0, the binary segmentation mask image M binary (x, y) represents the background part.

[0044] Preferably, the fine linear defect segmentation method further comprises the following steps:

[0045] After the subtle linear defects are expanded and marked, the deep learning image segmentation network is trained using the expanded marked subtle current defects. Perform auxiliary iterative training.

[0046] Specifically, the deep learning image segmentation network is set The training data set is The training dataset The real defect mask in is M gt_orig (x, y), the deep learning image segmentation network is trained by the following steps Perform auxiliary iterative training:

[0047] 1. Strategic extension of initial labels: The real defect mask M in gt_orig Marking the fine linear defect at (x, y), and appropriately extending the defect area boundary of the fine linear defect outward so that the range covered by the defect area boundary of the fine linear defect obtained by marking is slightly larger than the actual physical boundary of the fine linear defect;

[0048] 2. Iterative label refinement and retraining based on prediction results:

[0049] Let the current iteration round be t and the network parameter be θ t , the training data set is Among them I enhanced (i) represents the i-th pre-enhanced image, represents its training label at the tth iteration, and N is the number of training samples;

[0050] Step a (model training and prediction):

[0051] Use the current training dataset D t Training a deep learning image segmentation network Get the network parameters θ of the current stage t .

[0052] The model is then used to train each pre-enhanced image I enhanced(i) Perform prediction to obtain the predicted defect segmentation mask

[0053]

[0054] Step b (generate pseudo labels):

[0055] The predicted defect segmentation mask Convert to binary pseudo labels through thresholding

[0056]

[0057] Step c (correction and refinement):

[0058] The pseudo labels generated in step b As a reference, refer to the corresponding pre-enhanced image Perform quick checks and corrections to remove obvious prediction errors and adjust the label boundaries closer to the actual defects to obtain the corrected refined labels Let the correction rule be the correction function The revised refined label

[0059]

[0060] Step d (model retraining / fine-tuning):

[0061] Use the refined tags from step c Construct a new training dataset Based on this updated training dataset Deep Learning Image Segmentation Network For further training or in θ t Fine-tune based on the updated network parameters θ t+1 ;

[0062] Step e (iteration):

[0063] Update t to t+1, repeat steps a to d, and perform multiple rounds of iterative optimization until the segmentation effect reaches the predetermined performance index on an independent validation set.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1. Significantly improve the visibility and contrast of subtle linear defects: Through a two-stage pre-enhancement process (detail enhancement and nonlinear contrast amplification) in series, subtle and faint linear defects that were originally difficult to identify can be made relatively prominent in the image;

[0066] 2. Improve the training convergence of deep learning segmentation networks: The enhanced images provide the deep learning image segmentation network with stronger and easier-to-learn defect features, effectively solving the problem of deep learning image segmentation networks having difficulty converging or having poor convergence when processing such defects. Experimental verification shows that deep learning image segmentation networks that originally could not converge or had low segmentation accuracy can achieve stable convergence when trained using the enhanced images preprocessed by the method of the present invention.

[0067] 3. Improved segmentation accuracy for subtle defects: Due to the enhanced features of subtle linear defects and combined with iterative label refinement, the deep learning segmentation network can more accurately identify and locate the boundaries of subtle linear defects, thereby achieving higher segmentation accuracy and effectively reducing the recall rate;

[0068] 4. Enhanced robustness of the method: The iterative refinement process in the auxiliary training strategy helps improve the adaptability of the deep learning segmentation network to subtle linear defects of different shapes and backgrounds;

[0069] 5. Reduce extreme reliance on initial annotation quality: Through label expansion and iterative refinement, the shortcomings of initial manual annotation in perfectly outlining subtle defects can be compensated to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 This is a flow chart of a method for segmenting fine linear defects provided by an embodiment of the present invention.

[0072] Figure 2 This is a flowchart of the execution of step S1 in the method for segmenting fine linear defects provided by an embodiment of the present invention.

[0073] Figure 3 This is a flowchart of the execution of step S2 in the method for segmenting fine linear defects provided by an embodiment of the present invention.

[0074] Figure 4 Schematic diagram of an image to be processed in an experiment provided by an embodiment of the present invention.

[0075] Figure 5 yes Figure 4 Schematic diagram after processing in step S11.

[0076] Figure 6 yes Figure 5 Schematic diagram after processing in step S12.

[0077] Figure 7 yes Figure 6 Schematic diagram after processing in step S2. DETAILED DESCRIPTION

[0078] In order to explain in detail the possible application scenarios, technical principles, specific solutions that can be implemented, and the purpose and effects of this application, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of this application and are therefore only examples and are not intended to limit the scope of protection of this application.

[0079] See also Figure 1 The present invention provides a method for segmenting fine linear defects, which is suitable for segmenting defects in an image to be processed with fine linear defects. The fine linear defects here include but are not limited to shallow scratches, microcracks, fibers and other defects.

[0080] Next, this embodiment will combine how to apply the method of the present invention to automatically segment fine scratches on the surface of a micro lens of a 3C product, so as to better explain the fine linear defect segmentation method of the present invention in detail.

[0081] See also Figure 1-Figure 7 The method for segmenting fine linear defects comprises the following steps:

[0082] S1. Performing a series detail enhancement process and a nonlinear contrast amplification process on the image to be processed to obtain a pre-enhanced image.

[0083] S2. Input the pre-enhanced image into a pre-built and trained deep learning image segmentation network, and the deep learning image segmentation network outputs a binary segmentation mask image of the subtle linear defect.

[0084] The image to be processed can be obtained by capturing the surface of a 3C micro-lens using an industrial camera. Generally, when a 3C micro-lens surface has fine linear defects, the image to be processed will contain shallow scratches, micro-cracks, fibers, and other defects with a small width and low contrast against the background. These defects can be manually identified, thereby confirming that the surface of the 3C micro-lens is a defective product. Of course, the present invention can also be applied to other products or materials with fine linear defects, and the application of the present invention is not limited here.

[0085] It is worth noting that the present invention mainly performs defect segmentation on images to be processed with subtle linear defects. It can also perform segmentation on images to be processed with defects other than subtle linear defects. Of course, images to be processed that do not have subtle linear defects or defects other than subtle linear defects can be identified and screened out manually or by image recognition methods to reduce segmentation costs.

[0086] Preferably, the step S1 specifically includes:

[0087] S11 , performing detail enhancement processing based on local information difference on the image to be processed to obtain a first-stage enhanced image.

[0088] S12: Perform nonlinear contrast amplification processing based on self-multiplication of pixel values ​​on the first-stage enhanced image to obtain the pre-enhanced image.

[0089] The detail enhancement processing based on local information difference and the nonlinear contrast amplification processing based on the self-multiplication of pixel values ​​performed here constitute a two-stage pre-enhancement processing connected in series for each image to be processed, so as to make the defects more prominent relative to the background and facilitate subsequent segmentation and training processing.

[0090] Specifically, the step S11 includes:

[0091] S111, calculating the image to be processed I orig The local mean image I of (x, y) mean (x, y).

[0092] Specifically, the step S111 includes:

[0093] Through the mean filter f with preset mask width W and mask height H mean (Image, W, H) calculates the image to be processed I orig The local mean image I of (x, y) mean (x, y), the calculation formula is:

[0094] I mean (x, y) = f mean (I orig (x, y), W, H).

[0095] If the mean filter f mean The preset mask width W of (Image, W, H) is 57, and the preset mask height H is 7, then the mean filter f mean (Image, W, H) is a mean filter of 57 times 57. Of course, the preset mask width W and mask height H of the mean filter need to be selected according to the actual image type and are not limited here.

[0096] S112, calculating the image to be processed I orig (x, y) and the local mean image I mean The difference between (x, y) is used to obtain the first pixel-level difference image I diff (x, y), the calculation formula is:

[0097] I diff (x, y) = I orig (x, y)-I mean (x, y),

[0098] By I orig (x, y) and I mean The pixel values ​​of (x, y) are subtracted one by one to obtain the image to be processed I orig Each pixel of (x, y) corresponds to the local mean image I mean I of (x, y) diff (x, y).

[0099] S113, the first pixel level difference image I diff After (x, y) is multiplied by the preset enhancement factor k, weighted amplification is performed. The above calculation obtains the second pixel-level difference image I weighted_diff (x, y), the calculation formula is as follows:

[0100] I weig ted_diff (x, y) = k·I diff (x, y) = k·(I orig (x, y)-I mean (x, y)),

[0101] By doing so, the high-frequency details of the fine linear defects can be highlighted. For example, if the enhancement factor k is set to 1.2, the image to be processed I can be obtained. orig (x, y) the second pixel level difference value of each pixel, thereby obtaining the second pixel level difference image I weighted_diff (x, y).

[0102] S114: the second pixel level difference image I weighted_diff (x, y) and the image to be processed I orig (x, y) is processed by pixel-level superposition to obtain the first result image I stage1_pre (x, y), the calculation formula is as follows:

[0103] I stage1_pre (x, y) = I orig (x, y) + I weig ted_diff (x, y),

[0104] Substituting and sorting, we get:

[0105] I stage1_pre (x, y) = (1 + k)·I orig (x, y)-k·I mean (x, y).

[0106] S115: the first result image I stage1_pre (x, y) is subjected to grayscale value clipping to obtain the first stage enhanced image I stage1_out (x, y), the first stage enhanced image I stage1_out The pixel value of (x, y) is within the preset pixel value valid range, and the processing effect is as follows Figure 5 shown.

[0107] Specifically, the first result image I is clipped by the clipping function clip(v, L, U) stage1_pre (x, y) is subjected to grayscale value clipping to obtain the first stage enhanced image I stage1_out (x, y), the first stage enhanced image I stage1_out The pixel value of (x, y) is within the preset pixel value valid range [L, U]. Taking an 8-bit grayscale image as an example, L is 0 and U is 255.

[0108] Wherein, the clipping function clip(v, L, U) is defined as:

[0109]

[0110] At this time, the first stage enhanced image I stage1_out The formula for (x, y) is as follows:

[0111] I stage1_out (x, y) = clip(I stage1_pre (x, y), L, U),

[0112] Through the above steps, the image to be processed I can be obtained orig (x, y) The first stage enhanced image I after the first stage pre-enhancement stage1_out (x, y), the first stage enhanced image I stage1_out The pixel value of (x, y) is clipped to the range of [0, 255] to meet the requirements of subsequent operations.

[0113] Preferably, the step S12 specifically includes:

[0114] S121, enhancing the first stage image I stage1_out (x, y) is multiplied by its own value at the pixel level to obtain a second result image.

[0115] S122: Perform grayscale clipping on the second result image to obtain the pre-enhanced image I. enhanced (x, y), the pre-enhanced image I enhanced The pixel value of (x, y) is within the preset pixel value valid range.

[0116] Specifically, the first stage enhanced image I is obtained by the following formula: stage1_out (x, y) performs pixel-level self-value multiplication processing:

[0117] I stage2_pre (x, y) = I stage1_out (x, y)·I stage1_out (x, y)·Mult+Add,

[0118] Among them, I stage2_pre (x, y) is the second result image, Mult is the multiplication coefficient, and Add is the addition offset.

[0119] If the multiplication coefficient Mult is set to 0.02 and the addition offset Add is set to 0, the image to be processed I can be calculated by the above formula: orig The second result image I corresponding to (x, y) stage2_pre (x, y).

[0120] Specifically, the second result image I is clipped by the clipping function clip(v, L, U) stage2_pre (x, y) is subjected to grayscale value clipping to obtain the pre-enhanced image I enhanced (x, y), the pre-enhanced image I enhanced The pixel value of (x, y) is within the preset pixel value valid range [L, U], where L is 0 and U is 255;

[0121] Wherein, the clipping function clip(v, L, U) is defined as:

[0122]

[0123] At this time, the pre-enhanced image I enhanced The formula for (x, y) is as follows:

[0124] I enhanced (x, y) = clip(I stage2_pre (x, y), L, U),

[0125] Through the above steps, the image to be processed I can be obtained orig (x, y) Pre-enhanced image I after the second stage of pre-enhancement enhanced (x, y), this pre-enhanced image I enhancedThe pixel value of (x, y) is clipped to the range of [0, 255] to meet the requirements of subsequent operations. The processing effect is as follows: Figure 6 shown.

[0126] Preferably, the step S2 specifically includes:

[0127] S21, the pre-enhanced image I enhanced (x, y) input to a pre-built and trained deep learning image segmentation network In the deep learning image segmentation network According to the pre-enhanced image I enhanced (x, y) output segmentation mask image M pred (x, y), where θ is the deep learning image segmentation network The network parameters of the deep learning image segmentation network is a convolutional neural network architecture for segmentation tasks, the segmentation mask image M pred The formula for (x, y) is defined as:

[0128]

[0129] It is understandable that the deep learning image segmentation network here It can be a network with a convolutional neural network architecture such as U-Net, SegNet, Deeplab, etc. for segmentation tasks.

[0130] S22, the segmentation mask image M is processed by the following function pred (x, y) is subjected to binary segmentation processing to obtain the binary segmentation mask image M binary (x, y):

[0131]

[0132] Among them, T seg is a preset threshold, the binary segmentation mask image M binary (x, y) is 1 When the binary segmentation mask image M binary (x, y) represents the fine linear defect part, and the binary segmentation mask image M binary When (x, y) is 0, the binary segmentation mask image M binary (x, y) represents the background part, and its processing effect is as follows Figure 7 shown.

[0133] Preferably, the fine linear defect segmentation method further comprises the following steps:

[0134] After the subtle linear defects are expanded and marked, the deep learning image segmentation network is trained using the expanded marked subtle current defects. Perform auxiliary iterative training.

[0135] Specifically, the deep learning image segmentation network is set The training data set is The training dataset is The pre-prepared package contains N pre-enhanced images and its corresponding label training dataset.

[0136] The training dataset The real defect mask in is M gt_orig (x, y), the deep learning image segmentation network is trained by the following steps Perform auxiliary iterative training:

[0137] 1. Strategic extension of initial labels: The real defect mask M in gt_orig The fine linear defect in (x, y) is marked, and the defect area boundary of the fine linear defect is moderately expanded outward so that the range covered by the defect area boundary of the fine linear defect obtained by marking is slightly larger than the actual physical boundary of the fine linear defect.

[0138] It is understandable that since such fine linear defects can be identified by the naked eye but are not easy to be identified by machines, their positions are mainly marked manually. When marking, the boundaries of the fine linear defect marks need to be consciously extended by 7 to 10 pixels on both sides to obtain the initial training label M. gt_expanded (x, y), which is the initial training label M here gt_expanded (x, y) is the corresponding real defect mask M gt_orig Extended annotation of subtle linear defects in (x, y).

[0139] 2. Iterative label refinement and retraining based on prediction results:

[0140] Let the current iteration round be t and the network parameter be θ t , the training data set is Among them I enhanced (i) represents the i-th pre-enhanced image, It represents the training label at the tth iteration, and N is the number of training samples.

[0141] In this embodiment, if U-Net is used as the deep learning image segmentation network Set the maximum number of iterations Titer = 3.

[0142] Initial training (t=0): using the dataset Train the U-Net model and obtain the parameter θ0.

[0143] Iteration round (t=1 to Titer).

[0144] Step a (model training and prediction):

[0145] Use the current training dataset D t Training a deep learning image segmentation network Get the network parameters θ of the current stage t .

[0146] Then use this model to train each image I enhanced (i) Perform prediction to obtain the predicted defect segmentation mask

[0147]

[0148] Specifically, using the current model θ t-1 For all training images Make predictions and get

[0149] Step b (generate pseudo labels):

[0150] The predicted defect segmentation mask Convert to binary pseudo labels through thresholding

[0151]

[0152] Specifically, Thresholding is performed to obtain pseudo labels

[0153] Step c (correction and refinement):

[0154] The pseudo labels generated in step b As a reference, refer to the corresponding pre-enhanced image Perform quick checks and corrections to remove obvious prediction errors and adjust the label boundaries closer to the actual defects to obtain the corrected refined labels Let the correction rule be the correction function The revised refined label

[0155]

[0156] Specifically, by manually Make corrections, remove obvious errors, and make the defect boundaries closer to the actual situation to obtain refined labels

[0157] Step d (model retraining / fine-tuning):

[0158] Use the refined tags from step c Construct a new training dataset Based on this updated training dataset Deep Learning Image Segmentation Network For further training or in θ t Fine-tune based on the updated network parameters θ t+1 .

[0159] Specifically, using the updated dataset Retrain the U-Net model to obtain new model parameters θ t .

[0160] Step e (iteration):

[0161] Update t to t+1, repeat steps a to d, and perform multiple rounds of iterative optimization until the segmentation effect reaches the predetermined performance index on an independent validation set. The performance index here refers to the segmentation error of subtle linear defects being less than a certain preset value. This means that the deep learning image segmentation network that has undergone multiple rounds of iterative optimization is considered to be The segmentation accuracy can meet the actual segmentation requirements.

[0162] After multiple experimental verifications, the final U-Net model (with parameters θ) after the Titer round of iterative training optimization was Titer ) is used to segment the fine scratches of the new micro lens image that has been pre-enhanced. Input pre-enhanced image I enhanced_new (x, y), model output prediction mask M pred_new (x, y), and then the final binary segmentation result M is obtained by thresholding binary_new (x, y).

[0163] Experimental verification results: Compared to existing techniques that directly use the processed image for U-Net training, which often fails to converge despite long training cycles and multiple parameter adjustments, this embodiment significantly enhances the characteristics of subtle scratches through a specific, two-stage pre-enhancement process, enabling the U-Net network to more easily learn and identify these defects. Furthermore, this embodiment combines initial label expansion with subsequent iterative label refinement, making the training process more stable and converging. This significantly improves both final segmentation accuracy and recall. This method can effectively and accurately segment extremely subtle scratches that were previously difficult to detect or had poor segmentation results.

[0164] It should be noted that this embodiment is based on Figure 4 An 8-bit grayscale image is shown as the image to be processed. The image to be processed is composed of multiple pixels, each with independent xy coordinates. Of course, other types of images can be converted to grayscale images and then segmented using the method described in this invention. The number of bits in the grayscale image can be flexibly set according to actual needs and is not limited here.

[0165] Combine Figure 1-Figure 7 , the present invention has the following beneficial effects:

[0166] 1. Significantly improve the visibility and contrast of subtle linear defects: Through a two-stage pre-enhancement process (detail enhancement and nonlinear contrast amplification) in series, subtle and faint linear defects that were originally difficult to identify can be made relatively prominent in the image;

[0167] 2. Improve the training convergence of deep learning segmentation networks: The enhanced images provide the deep learning image segmentation network with stronger and easier-to-learn defect features, effectively solving the problem of deep learning image segmentation networks having difficulty converging or having poor convergence when processing such defects. Experimental verification shows that deep learning image segmentation networks that originally could not converge or had low segmentation accuracy can achieve stable convergence when trained using the enhanced images preprocessed by the method of the present invention.

[0168] 3. Improved segmentation accuracy for subtle defects: Due to the enhanced features of subtle linear defects and combined with iterative label refinement, the deep learning segmentation network can more accurately identify and locate the boundaries of subtle linear defects, thereby achieving higher segmentation accuracy and effectively reducing the recall rate;

[0169] 4. Enhanced robustness of the method: The iterative refinement process in the auxiliary training strategy helps improve the adaptability of the deep learning segmentation network to subtle linear defects of different shapes and backgrounds;

[0170] 5. Reduce extreme reliance on initial annotation quality: Through label expansion and iterative refinement, the shortcomings of initial manual annotation in perfectly outlining subtle defects can be compensated to a certain extent.

[0171] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for segmenting fine linear defects, suitable for segmenting a to-be-processed image with fine linear defects, characterized in that: The method for segmenting fine linear defects comprises the following steps: Performing a series of detail enhancement processing and nonlinear contrast amplification processing on the image to be processed to obtain a pre-enhanced image; The pre-enhanced image is input into a pre-built and trained deep learning image segmentation network, and the deep learning image segmentation network outputs a binary segmentation mask image of the image to be processed.

2. The fine linear defect segmentation method according to claim 1, wherein: The performing of a series detail enhancement process and a nonlinear contrast amplification process on the image to be processed to obtain a pre-enhanced image specifically includes: Performing detail enhancement processing based on local information difference on the image to be processed to obtain a first-stage enhanced image; The first-stage enhanced image is subjected to nonlinear contrast amplification processing based on self-multiplication of pixel values ​​to obtain the pre-enhanced image.

3. The fine linear defect segmentation method according to claim 2, wherein: The performing detail enhancement processing based on local information difference on the image to be processed to obtain a first-stage enhanced image specifically includes: Calculate the image to be processed I orig The local mean image I of (x,y) mean (x,y); Calculate the image to be processed I orig (x,y) and local mean image I mean The difference between (x, y) is used to obtain the first pixel-level difference image I diff (x,y); The first pixel level difference image I diff After multiplying (x, y) by the preset enhancement factor k, weighted amplification is performed to obtain the second pixel-level difference image I weighted_diff (x,y); The second pixel level difference image I weighted_diff (x, y) and the image to be processed I orig (x, y) is processed at the pixel level to obtain the first result image I stage1_pre (x,y); For the first result image I stage1_pre (x, y) is clipped with grayscale value to obtain the first stage enhanced image I stage1_out (x, y), the first stage enhanced image I stage1_out The pixel value of (x, y) is within the preset pixel value valid range.

4. The fine linear defect segmentation method according to claim 3, wherein: The calculation of the image to be processed I orig The local mean image I of (x,y) mean (x,y), specifically including: Through the mean filter f with preset mask width W and mask height H mean (Image, W, H) calculates the image to be processed I orig The local mean image I of (x,y) mean (x,y), the calculation formula is: I mean (x,y)=f mean (I orig (x,y),W,H)。 5. The fine linear defect segmentation method according to claim 3, wherein: The performing of nonlinear contrast amplification processing based on self-multiplication of pixel values ​​on the first-stage enhanced image to obtain the pre-enhanced image specifically includes: For the first stage enhanced image I stage1_out (x, y) performs pixel-level self-value multiplication processing to obtain a second result image; The grayscale value clipping process is performed on the second result image to obtain the pre-enhanced image I enhanced (x, y), the pre-enhanced image I enhanced The pixel value of (x, y) is within the preset pixel value valid range.

6. The fine linear defect segmentation method according to claim 5, wherein: The first stage enhanced image I is calculated by the following formula stage1_out (x,y) performs pixel-level self-value multiplication processing: I stage2_pre (x,y)=I stage1_out (x,y)·I stage1_out (x,y)·Mult+Add, Among them, I stage2_pre (x, y) is the second result image, Mult is the multiplication coefficient, and Add is the addition offset.

7. The fine linear defect segmentation method according to claim 5, wherein: The first result image I is clipped by the clipping function clip(v,L,U) stage1_pre (x, y) is clipped with grayscale value to obtain the first stage enhanced image I stage1_out (x, y), the first stage enhanced image I stage1_out The pixel value of (x,y) is within the preset pixel value valid range [L,U]; The second result image I is clipped by the clipping function clip(v,L,U) stage2_pre (x, y) is subjected to grayscale value clipping to obtain the pre-enhanced image I enhanced (x, y), the pre-enhanced image I enhancnd The pixel value of (x,y) is within the preset pixel value valid range [L,U]; Wherein, the clipping function clip(v,L,U) is defined as:

8. The fine linear defect segmentation method according to claim 1, wherein: Inputting the pre-enhanced image into a pre-built and trained deep learning image segmentation network, wherein the deep learning image segmentation network outputs a binary segmentation mask image of the subtle linear defect, specifically includes: The pre-enhanced image I enhanced (x,y) input to a pre-built and trained deep learning image segmentation network In the deep learning image segmentation network According to the pre-enhanced image I enhanced (x,y) output segmentation mask image M pred (x, y), where θ is the deep learning image segmentation network The network parameters of the deep learning image segmentation network is a convolutional neural network architecture for segmentation tasks, the segmentation mask image M pred The formula for (x,y) is defined as: The segmentation mask image M is processed by the following function pred (x, y) is subjected to binary segmentation processing to obtain the binary segmentation mask image M binary (x,y): Among them, T seg is a preset threshold, the binary segmentation mask image M bunary When (x, y) is 1, the binary segmentation mask image M binary (x, y) represents the fine linear defect part, and the binary segmentation mask image M binary When (x, y) is 0, the binary segmentation mask image M binary (x,y) represents the background part.

9. The fine linear defect segmentation method according to claim 8, wherein: The method for segmenting fine linear defects further comprises the following steps: After the subtle linear defects are expanded and marked, the deep learning image segmentation network is trained using the expanded marked subtle current defects. Perform auxiliary iterative training.

10. The method for segmenting fine linear defects according to claim 9, wherein The deep learning image segmentation network The training data set is The training dataset The real defect mask in is M gt_orig (x, y), the deep learning image segmentation network is trained by the following steps Perform auxiliary iterative training:

1. Strategic extension of initial labels: The real defect mask M in gt_orig Mark the fine linear defect at (x, y), and appropriately expand the defect area boundary of the fine linear defect outward so that the range covered by the defect area boundary of the fine linear defect obtained by marking is slightly larger than the actual physical boundary of the fine linear defect; 2. Iterative label refinement and retraining based on prediction results: Let the current iteration round be t and the network parameter be θ t , the training data set is Among them I enhanced (i) represents the i-th pre-enhanced image, represents its training label at the tth iteration, and N is the number of training samples; Step a (model training and prediction): Use the current training dataset D t Training a deep learning image segmentation network Get the network parameters θ of the current stage t . The model is then used to train each pre-enhanced image I enhanced (i) Perform prediction to obtain the predicted defect segmentation mask Step b (generate pseudo labels): The predicted defect segmentation mask The threshold is converted into a binary pseudo label by the following formula Step c (correction and refinement): The pseudo labels generated in step b As a reference, refer to the corresponding pre-enhanced image Perform quick checks and corrections to remove obvious prediction errors and adjust the label boundaries closer to the actual defects to obtain the corrected refined labels Let the correction rule be the correction function The revised refined label for: Step d (model retraining / fine-tuning): Use the refined tags from step c Construct a new training dataset Based on this updated training dataset Deep Learning Image Segmentation Network For further training or in θ t Fine-tune based on the updated network parameters θ t+1 ; Step e (iteration): Update t to t+1, repeat steps a to d, and perform multiple rounds of iterative optimization until the segmentation effect reaches the predetermined performance index on an independent validation set.

Images