A multi-view industrial semantic segmentation labeling method and system for weak feature defects

Abstract

The application discloses a kind of multi-view industrial semantic segmentation labeling methods and systems for weak feature defects, collect multi-view surface image sample set and organize loading according to view angle directory;By guiding surface edge region of interest extraction algorithm, obtain region of interest working image;By weak feature enhancement algorithm, generate review enhancement map;After region of interest working image grayscale normalization, input semantic preliminary draft model based on semantic segmentation network, generate editable defect pre-labeling mask;In interactive interface, in combination with review enhancement map, through brush, rubber and ridge line tracking intelligent selection, manually correct pre-labeling mask and determine defect or normal;The correction result is written in the way of separating draft from final draft in labeling work area, and export the same name region of interest image and binary mask pair.The application improves the labeling efficiency of weak feature defects and the consistency of mask boundary, enhances the recognizability of low-contrast cracks.

Description

Technical Field

[0001] This invention relates to the field of industrial visual defect annotation technology, specifically a multi-view industrial semantic segmentation and annotation method and system for weak feature defects. Background Technology

[0002] In precision manufacturing and surface quality inspection scenarios, product defects often manifest as subtle features such as low-contrast cracks, minor scratches, and localized texture anomalies. These defects occupy a very small proportion of the target area in the entire image captured by industrial cameras, have blurred edges, and low differentiation from the background, yet they directly determine the recall rate of the final defect detection model and the risk of missed detection on the production line. Therefore, high-quality, reproducible semantic segmentation annotation has become an indispensable foundational element in the closed-loop of industrial visual quality inspection.

[0003] Currently, the industrial sector widely uses general-purpose semantic segmentation annotation tools (such as LabelMe and CVAT) for defect annotation. However, these tools are inherently designed for natural scenes or salient targets, and their annotation paradigms have the following inherent limitations:

[0004] First, there is a lack of perceptual aids for subtle features. Annotators need to search for defect targets from scratch on the original whole image. For low-contrast cracks or fine scratches, they are very likely to miss them due to visual fatigue. Moreover, the initial sketch lacks reference, resulting in extremely low initial annotation efficiency.

[0005] Second, there is a lack of intelligent drawing methods for slender and curved boundaries. Existing tools only provide polygonal or freehand brushes, requiring annotators to manually trace the edges point by point. This is not only time-consuming but also makes it difficult to ensure the continuity of the boundaries. Especially for weak cracks with varying directions and widths, the drawing process is highly subjective and the mask consistency is poor.

[0006] Third, there is a lack of multi-view data organization and spatial alignment capabilities. Industrial inspection often requires the collection of multiple views of a product (such as left view, right view, and top view), but general tools do not manage data according to viewpoints, nor can they automatically focus and geometrically correct the region of interest (ROI) of the original image. This results in inconsistencies between the labeled mask and the coordinates of the working image, and the labeled products need to undergo additional post-processing before they can be used for model training, affecting delivery efficiency and data quality.

[0007] Fourth, there is a lack of an efficient closed-loop annotation process. Existing solutions mostly rely on manual full-scale annotation of each image, lacking an iterative mechanism of "AI pre-annotation - manual refinement - separation of draft and final version". It is difficult to achieve process traceability and result standardization while ensuring annotation accuracy, and cannot meet the dual requirements of production lines for annotation efficiency and quality consistency.

[0008] In summary, existing general-purpose annotation tools suffer from a series of problems when dealing with subtle feature defects and multi-perspective industrial scenarios, such as difficulty in defect search, slow outlining, unstable boundaries, data misalignment, and uncontrollable processes. There is an urgent need for a dedicated annotation method and system adapted to this scenario to improve annotation efficiency, mask quality, and delivery availability. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and propose a multi-view industrial semantic segmentation and annotation method and system for weak feature defects, so as to solve the problems of insufficient support for weak features, low efficiency of drawing from scratch, and unstable boundaries of existing general annotation tools.

[0010] To achieve the above objectives, the technical solution specifically adopted by the present invention is as follows:

[0011] A multi-view industrial semantic segmentation and annotation method for weak feature defects includes:

[0012] Step 1: Collect a multi-view surface image sample set of industrial products;

[0013] Step 2: Using a guided surface edge region of interest extraction algorithm, the region of interest is automatically extracted from the sample to obtain a working image of the region of interest;

[0014] Step 3: Using a weak feature enhancement algorithm, weak feature enhancement is performed on the working image of the region of interest, and a kernel enhancement map is generated through illumination correction and local contrast enhancement.

[0015] Step 4: Input the working image of the region of interest into the weak feature defect semantic draft model based on the semantic segmentation network to generate an editable defect pre-annotation mask; the pre-annotation mask is only used as a revocable draft.

[0016] Step 5: Load the working image of the region of interest and the pre-annotated mask in the interactive annotation interface. Combined with the enhanced display based on the kernel enhancement map, manually correct the pre-annotated mask using a pen, eraser, and intelligent selection, and determine whether the current sample is defective or normal. The intelligent selection is achieved by automatically tracking and snapping to bright ridges on the kernel enhancement map using a ridge tracking algorithm.

[0017] Step 6: Write the revised results into the annotation workspace in a way that separates the automatic saving of the draft from the confirmation of the final draft, and simultaneously export the region of interest image with the same name and the binary mask pair to form deliverable training data.

[0018] Preferably, the sample set includes at least one complete view from the left, right and top views, and is organized by view directory and loaded into the annotation workspace.

[0019] Preferably, in step 2, the method for extracting the region of interest working image is as follows: when the input is a full image from the camera, the region of interest on the product surface is automatically extracted based on edge guidance and geometric constraints and corrected to a unified working coordinate, thereby outputting the region of interest working image; when the input is a cropped region of interest image, it is verified and used as the region of interest working image.

[0020] Preferably, in step 2, when inputting the entire image from the camera, the algorithm for extracting the region of interest at the surface edge specifically includes:

[0021] Identify the input form of the current sample, wherein the input form includes the full camera image and the cropped region of interest image;

[0022] When the input is a full image from the camera, the algorithm for extracting the region of interest (ROI) on the surface edge is called according to the viewpoint: the image is smoothed and edge detected, and morphological closing operation and constrained adaptive edge fitting are combined to obtain the quadrilateral region on the product surface. The homography transformation from the source image to the working image is estimated, and perspective correction and cropping are performed on the source image based on the homography transformation to output the working image of the ROI.

[0023] When the input is a cropped region of interest image, a unit transformation pass-through is used, and after verification, the input is used as the working image of the region of interest.

[0024] When the input format cannot be safely determined, the sample is set to a pending confirmation state.

[0025] Preferably, the weak feature enhancement algorithm specifically includes:

[0026] Convert the working image of the region of interest into a grayscale image and estimate the background illumination using Gaussian blur;

[0027] Illumination correction is performed based on the background illumination to eliminate uneven illumination;

[0028] The image after illumination correction is subjected to positional stretching and local contrast enhancement;

[0029] The image after local contrast enhancement is subjected to bilateral filtering for noise reduction and desharpening, and then further local contrast enhancement is performed to obtain the kernel enhancement image.

[0030] Preferably, the illumination correction is achieved by subtracting the Gaussian blur estimate of the background illumination from the original grayscale image and adding the median of the background illumination.

[0031] Preferably, the quantile stretching is a dynamic range stretching performed by determining the stretching interval according to preset lower and upper quantiles; the local contrast enhancement adopts contrast-limited adaptive histogram equalization.

[0032] Preferably, the pre-labeled mask generation step specifically includes:

[0033] The original working image of the region of interest is used as the input to the semantic draft model, and gray-level normalization is performed on the regions prone to weak features.

[0034] The normalized image is input into a semantic segmentation network based on the SegFormer-B2 architecture, and a defect probability map is obtained after activation function.

[0035] The defect probability map is binarized, and the binarization result is filtered by connected component filtering to remove connected components with an area smaller than a preset threshold, thereby obtaining a defect pre-labeled mask.

[0036] The defect pre-annotation mask is combined with the current manual mask and then loaded into the canvas of the interactive annotation interface as an undoable edit. When the pre-annotation mask is empty, the current sample is not automatically determined to be normal.

[0037] Preferably, the intelligent selection specifically includes:

[0038] On the enhanced map, seed points are determined based on the ridge line responses near the locations clicked by the annotator.

[0039] The ridge tracing algorithm is used to perform bidirectional tracing along the bright ridge direction, and mask strokes are generated after attempting to cross the short gaps.

[0040] The ridge response is calculated as follows: for the candidate pixel position and the tracking direction, the brightness difference on the kernel enhancement map along the two sides of the position perpendicular to the tracking direction is calculated respectively, and the smaller value of the brightness difference on both sides is taken as the ridge score of the position in that direction.

[0041] Preferably, the method further includes: during the interactive correction step, the annotator can switch between viewing the enhanced review image and the original working image of the region of interest.

[0042] Preferably, the result derivation step specifically includes:

[0043] When editing occurs on the canvas in the interactive annotation interface, the draft is automatically saved according to the anti-shake timing.

[0044] When the annotator confirms a defect or makes a valid decision, the final draft is submitted and the project version information is updated.

[0045] Output the region of interest image with the same name and the binary mask in pairs to the results directory.

[0046] This invention also provides a multi-view industrial semantic segmentation and annotation system for weak feature defects, comprising:

[0047] The sample loading module is used to open the multi-view image data directory, construct the annotation workspace, and enumerate the samples to be annotated;

[0048] The Region of Interest (ROI) processing module is used to extract the ROI from the full camera image or a cropped image and generate a working image by using a guided surface edge ROI extraction algorithm.

[0049] The enhanced display module is used to generate a weak feature kernel enhancement map through a weak feature enhancement algorithm;

[0050] The AI ​​draft module is used to call a semantic draft model with weak feature defects based on a semantic segmentation network to generate an editable pre-annotated mask;

[0051] The interactive correction module provides intelligent selection of brushes, erasers, and ridge-tracking algorithms, as well as interaction between defective and normal decision-making.

[0052] The persistence and export module is used to save drafts, submit final drafts, and export the region of interest image and binary mask pair.

[0053] This invention has the following characteristics and beneficial effects:

[0054] This invention organizes and loads multi-view images by catalog, and uses a guided surface edge ROI extraction algorithm to adaptively process the full camera image or cropped ROI image, confining the annotation attention to the effective surface area of ​​the product and effectively eliminating background interference. At the same time, it unifies the coordinates of the source image and the working image through homography transformation, so that the annotation mask is strictly aligned with the product surface, avoiding the extra post-processing caused by cropping and coordinate mismatch in general tools, and significantly improving the delivery usability of the annotation product.

[0055] This invention utilizes the V21 weak feature enhancement algorithm to sequentially perform background illumination estimation and correction, quantile dynamic range stretching, CLAHE local contrast enhancement, bilateral filtering denoising, and desharpening on the ROI working image. This significantly amplifies the grayscale difference between low-contrast crack ridges and the background, making previously difficult-to-discern weak features clearly discernible on the verification enhancement image. This effectively reduces the visual search difficulty and fatigue of annotators, thereby reducing the risk of missed annotations from the source.

[0056] This invention introduces a weak feature defect semantic draft model based on the SegFormer-B2 semantic segmentation network into the annotation process. Taking the original ROI working image as input, it generates a defect pre-annotation mask after normalizing the weak feature-prone areas. In particular, the pre-annotation mask is positioned as a "revocable draft", and an empty draft is not automatically equivalent to a normal sample. This not only leverages the advantage of the Transformer architecture's large receptive field for contextual awareness of weak features, but also uses manual review as a fallback. This transforms the annotator from the traditional mode of "full-image search and zero-based drawing" to "checking and local correction", greatly improving annotation efficiency.

[0057] This invention provides an intelligent selection tool based on a ridge tracing algorithm in the interactive correction process. It determines seed points on the verification enhancement map based on the ridge response and tracks bidirectionally along the bright ridge direction. At the same time, it automatically crosses and completes short gaps, enabling slender and curved crack boundaries to generate continuous strokes in an adsorption manner. This method effectively solves the industry pain point that general brush tools are difficult to draw weak crack boundaries with varying directions and uneven widths, and significantly improves the boundary continuity and annotation consistency of defect masks.

[0058] This invention employs a two-level storage mechanism that separates automatic draft saving from final confirmation. During canvas editing, drafts are automatically saved in a de-shaking sequence, while the final draft is submitted and version information is updated upon decision confirmation, ensuring the traceability and data security of the annotation process. Simultaneously, images of ROIs with the same name are output in pairs with binary masks, ensuring that the annotation results are strictly aligned with the working images in the coordinate system. This allows the data to be directly used for training deep learning detection models without additional post-processing, meeting the standardized and reproducible delivery requirements of industrial production lines for annotated data.

[0059] The various steps of the technical solution of this invention are mutually synergistic and supportive. ROI extraction provides a clean working image for the enhancement algorithm and AI draft, the V21 enhancement image provides a clear ridgeline input base map for intelligent selection, the AI ​​draft provides an efficient starting point for manual correction, and the separation of draft / final draft and paired export ensures process controllability and result usability. Together, they form a complete technical closed loop, which overcomes a series of interrelated defects of general annotation tools in multi-view industrial scenarios with weak feature defects, and achieves better collaborative technical effects. Attached Figure Description

[0060] Figure 1 The main flowchart of a multi-view industrial semantic segmentation and annotation method for weak feature defects provided by the present invention is shown below.

[0061] Figure 2 The annotation system module structure diagram provided by this invention;

[0062] Figure 3The flowchart of ROI constraints and AI pre-labeling provided by this invention;

[0063] Figure 4 The flowchart for interactive refinement and submission confirmation provided by this invention;

[0064] Figure 5 This is a schematic diagram illustrating the effect of weak feature annotation provided by the present invention. Detailed Implementation

[0065] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0066] Example 1

[0067] This embodiment provides a multi-view industrial semantic segmentation and annotation method for weak feature defects, such as Figure 1 , 3 and Figure 4 As shown, it includes:

[0068] Step 1: Collect a multi-view surface image sample set of industrial products. The sample set should contain at least one complete view from the left, right and top views. After organizing the images according to the view directory, load them into the annotation work area.

[0069] In this embodiment, taking the right view verification subset as an example, the system scans the samples to be labeled and establishes a working area index; the source image remains read-only, and the labeled products are written to an independent working area to avoid contaminating the original data.

[0070] Step 2: Using the guided surface edge region of interest (ROI) extraction algorithm, the region of interest is automatically extracted from the sample to obtain the working image of the region of interest.

[0071] Specifically, in this embodiment, it is necessary to first identify the input form of the current sample, where the input form includes the full camera image and the cropped ROI image.

[0072] In this embodiment, the parameters of the input form are defined as follows:

[0073] Full-resolution camera image: This refers to the raw, full-resolution image directly acquired by an industrial area scan or line scan camera. Its size is determined by the camera sensor resolution, typically 2448×2048 or 4096×3072 pixels. The image format is a standard 8-bit grayscale image or a 24-bit color image (converted to grayscale during processing). This type of image usually contains irrelevant information such as the product surface area, background, fixtures, and trays.

[0074] Cropped ROI image: This refers to an image that has been segmented from the original full camera image after preprocessing steps (such as mechanical positioning or software cropping), containing only the effective surface area of ​​the product. Its format is also a standard 8-bit grayscale image or a 24-bit color image. Furthermore, when the input is a full camera image, the region of interest on the product surface is automatically extracted based on edge guidance and geometric constraints, corrected to a unified working coordinate system, and then output as a working image of the region of interest.

[0075] The algorithm for extracting regions of interest (ROIs) on the surface edges is called according to the viewpoint: Image smoothing and edge detection are performed, combined with morphological closing operations and constrained adaptive edge fitting. Taking the right viewpoint as an example, Gaussian algorithm is first used for background estimation and illumination correction, followed by Canny edge detection with a low threshold of 25 and a high threshold of 85. Two closing operations are performed using structuring elements to connect boundaries broken due to uneven illumination. Then, candidate slopes are enumerated within the slope constraint range. Candidate edge points are projected onto the intercept histogram and weighted by edge strength. The slope and intercept with the highest score are obtained by superimposing position priors and using them as the consensus line. Outliers are iteratively removed within the neighborhood of the consensus line, and the final line is fitted using least squares to obtain the quadrilateral region on the product surface. The four corner points are obtained by pairwise intersection of the four sides. These corner points are verified by geometric constraints such as area ratio, aspect ratio, and convexity. The longest side of the quadrilateral is taken as the width W, and the longest short side as the height H, corresponding to the... Use direct linear transformation to list equations point by point. Then, perform singular value decomposition on A, rearrange the right singular vectors corresponding to the smallest singular value into a 3×3 matrix, and estimate the homography transformation H from the source image to the working image. Iterate through each pixel of the working image. Use the inverse of H to find its coordinates in the source graph. If the calculation result is not an integer, the grayscale is retrieved using bilinear or cubic interpolation. If the result exceeds the boundaries of the source image, it is filled with black. This corrects perspective distortion into a rectangle and crops the background outside the product surface, thus performing perspective correction and cropping on the source image and outputting the working image of the region of interest. ;

[0076] When the input is a cropped region of interest image, a unit transformation pass-through is used, and after verification, the input is used as the working image of the region of interest.

[0077] When the input shape cannot be safely determined, the sample is placed in a pending confirmation state, prompting for manual confirmation before proceeding to subsequent annotation. Step 3: Using the weak feature enhancement algorithm (V21), weak feature enhancement is performed on the working image of the region of interest, and a verification enhancement map is generated through illumination correction and local contrast enhancement.

[0078] Specifically, weak feature enhancement algorithms include:

[0079] The working image of the region of interest is converted into grayscale image I, and the background illumination is estimated using Gaussian blur. ;

[0080] Illumination correction is performed based on background illumination to eliminate uneven illumination. The illumination correction method is as follows:

[0081] ;

[0082] in The corrected image grayscale value; This indicates the original input image in pixel coordinates. The grayscale value at that location; Indicates the entire image Perform Gaussian low-pass filtering and standard deviation as After Gaussian kernel convolution, The value at; express The median of all pixels in the entire image.

[0083] Image after illumination correction Quantitative stretching and CLAHE local contrast enhancement are performed. Quantitative stretching involves dynamically stretching within a range determined by preset lower and upper quantiles. Set the pixel value to 0. The pixel value is set to 255. CLAHE local contrast enhancement employs contrast-limited adaptive histogram equalization.

[0084] The image with enhanced local contrast is subjected to bilateral filtering for denoising and desharpening, followed by further local contrast enhancement to obtain a kernel-enhanced image. For each pixel ,Think center, The neighborhood of the diameter The inner bilateral filtering denoising formula is:

[0085]

[0086] in Represents spatial weights, distance The closer the two are, the greater their weight. Represents grayscale weight, and The more similar it is, the greater its weight. How far away should the neighboring pixels be included in the calculation? The larger it is, the blurrier it becomes; To determine if multiple images constitute the same edge, The larger the value, the less distinct the edges become, and the closer it is to ordinary Gaussian blur. It is the diameter of the neighborhood window. The denominator is the entire summation result, i.e., the normalization factor. .

[0087] The desharpening mask is first applied using GaussianBlur, and then using addWeighted:

[0088]

[0089] Substituting the parameters, we get:

[0090]

[0091] in It's the Gaussian blur version. It's high-frequency detail.

[0092] Step 4: Input the working image of the region of interest into the weak feature defect semantic draft model based on the semantic segmentation network SegFormer-B2 to generate an editable defect pre-annotation mask; the pre-annotation mask is only used as a revocable draft.

[0093] The pre-labeled mask generation step specifically includes:

[0094] Working image with the original region of interest As input to the initial semantic model, grayscale normalization is performed on regions prone to weak features. ;

[0095] It should be noted that weak feature-prone areas refer to local sub-regions within specific processing areas, stress concentration areas, or historically high-incidence areas of industrial products, determined based on prior knowledge of the product's manufacturing process, where the probability of defect occurrence is significantly higher than other locations on the surface. For example, for machined metal workpieces, edges, corners, areas around locating holes, and tool retraction marks are considered weak feature-prone areas (such as microcracks and burrs); for workpieces with surface coatings, their rounded corners or grooved areas are considered weak feature-prone areas. Gray-level normalization is performed on these areas, i.e., calculating the mean and standard deviation of gray levels within the sub-region, and then performing gray-level normalization on each pixel within the region. This operation standardizes the feature distribution to improve the detection sensitivity of the semantic segmentation network in this region.

[0096] The normalized image is input into a semantic segmentation network based on the SegFormer-B2 architecture. The defect probability map P is obtained by applying the sigmoid activation function;

[0097] Furthermore, a threshold is set in this embodiment. The defect probability map P is binarized, and the binarization result is filtered by connected component filtering to remove connected components with an area smaller than a preset threshold, thus obtaining the defect pre-annotation mask. ;

[0098] Finally, the defect pre-annotation mask is applied. Compared to current manual masks according to After merging, the image is loaded onto the canvas as a single, undoable edit; when the pre-annotated mask... If the value is empty, the current sample will not be automatically determined to be normal.

[0099] In this embodiment, the semantic draft provides high-recall proposals for weak features such as low-contrast diagonal cracks, transforming "finding defects and drawing outlines" into "verification and local correction".

[0100] In this embodiment, the training process of the semantic segmentation network fθ based on the SegFormer-B2 architecture is as follows:

[0101] Training dataset construction: Collect industrial surface images of the same type as the product to be labeled, including various subtle defects. Pixel-level precise annotations are performed on the training images based on historical experience, generating corresponding binary masks as ground truth for training. In the binary mask, defect areas are represented by 1, and the background by 0. The training, validation, and test sets are divided in approximately an 8:1:1 ratio.

[0102] Network initialization: The semantic segmentation network is initialized with parameters using SegFormer-B2 weights pre-trained on the ImageNet-1K dataset.

[0103] Training parameter settings: Stochastic gradient descent (SGD) with momentum is used as the optimizer, with a momentum coefficient of 0.9 and a weight decay coefficient of 0.0005. The initial learning rate is set to 0.001, and a multinomial learning rate decay strategy with power=0.9 is adopted. The number of training epochs is set to 200, and the batch size is set to 8 based on the graphics card's memory. The input image size is uniformly scaled to 512×512 pixels.

[0104] Loss function: A combined loss function, namely binary cross-entropy loss, is used. ) and Dice Loss The weighted sum of the loss functions is given by: , where α and β are balance coefficients, which are set to 1 and 0.5 respectively in this embodiment.

[0105] Inference Process: The network output layer is followed by a sigmoid activation function, mapping the predicted value of each pixel to the [0,1] interval, representing the probability that the pixel belongs to a defect region. During the inference phase, a classification threshold τ=0.99 is set, and the probability is... Figure 2The value is converted into the final predicted mask. Step 5: Load the working image of the region of interest and the pre-annotated mask in the interactive annotation interface, and combine the enhanced display based on the kernel enhancement map to manually correct the pre-annotated mask by using a pen, eraser and intelligent selection, and determine whether the current sample is defective or normal; the intelligent selection is to automatically track and snap to the bright ridges on the kernel enhancement map by the ridge tracking algorithm.

[0106] Specifically, intelligent selection includes:

[0107] In the review enhancement diagram Above, seed points are determined based on the ridge line responses near the clicked location by the annotator;

[0108] The algorithm tracks the bright ridge bidirectionally and attempts to cross it at short gaps to generate mask strokes.

[0109] The method for generating mask strokes is as follows: attempt to cross dark gaps within a preset number of steps, generate snap strokes, and incorporate them into the mask.

[0110] The ridge response is calculated as follows: for each candidate pixel position and tracking direction, the brightness difference on the kernel enhancement map along both sides of that position perpendicular to the tracking direction is calculated, and the smaller of the two brightness differences is taken as the ridge score for that position in that direction. .

[0111] Finally, the current sample is confirmed to be defective or normal, and the interactive decision is completed. In the interactive correction step, the annotator can switch between viewing the enhanced image and the original region of interest working image.

[0112] Understandably, in this embodiment, intelligent selection is used to complete the continuity of the slender crack boundary; Figure 5 This document provides a schematic diagram illustrating the process from original image, ROI, enhancement review, AI draft, intelligent selection to final mask. Figure 5 As can be seen, the original image has low contrast between the subtle defects and the background, and the edges are blurry, making direct annotation very difficult. After region of interest (ROI) extraction, the defect area is converted to a front-aligned view. Further enhancement processing of the ROI image significantly widens the grayscale difference between the defect and the background, making the edges clearer and facilitating manual review and adjustment. Feeding the normalized grayscale image into a trained semantic segmentation network yields the initial segmentation result from the AI ​​output. To address the undersegmentation and oversegmentation issues caused by the blurred boundaries of subtle defects, this embodiment utilizes ridge line scoring to select seed pixels, accurately outlining the defect target based on the initial AI draft, and then manually correcting it to generate the final mask, thus achieving automatic and high-precision segmentation and annotation of these weak-feature defects.

[0113] Step 6: Write the revised results into the annotation workspace in a way that separates the automatic saving of the draft from the confirmation of the final draft, and simultaneously export the region of interest image with the same name and the binary mask pair to form deliverable training data.

[0114] Specifically, when editing occurs on the canvas of the interactive annotation interface, the draft is automatically saved according to the anti-shake timing sequence;

[0115] When the annotator confirms a defect or makes a valid decision, the final draft is submitted and the project version information is updated.

[0116] The region of interest image with the same name is paired with a binary mask and output to the results directory to form deliverable training data.

[0117] In this embodiment, separating the draft and the final version ensures that the process is traceable; exporting in pairs ensures that the mask and the working image are strictly aligned, which facilitates the training of the subsequent detection model.

[0118] like Figure 5 As shown, all compliant camera images from the right perspective will be processed image by image using an edge ROI extraction algorithm based on the prior calibration of the viewpoint. The product surface correction will be automatically mapped to a standard rectangle in a unified working coordinate system. Images with insufficient edge confidence are marked as pending confirmation to prevent incorrect cropping. The position and size of the quadrilaterals in each image are adaptively calculated using gradient weighted voting of the edge points in that image; parameters such as σ, Canny threshold, search band, and slope upper limit are shared by the entire group.

[0119] See you in the interactive refinement stage Figure 5 The initial AI results showed high recall for low-contrast diagonal cracks. If the features at both ends of the crack were weakened or there were gaps in the outline, the annotator could use ridge tracing to snap the image together, and then use a brush and eraser to make pixel-level adjustments. After completion, the ROI image and binary mask were exported in pairs.

[0120] Example 2

[0121] This embodiment provides a multi-view industrial semantic segmentation and annotation system for implementing the weak feature defect-oriented system of Embodiment 1, such as... Figure 2 As shown, it includes:

[0122] The sample loading module is used to open the multi-view image data directory, construct the annotation workspace, and enumerate the samples to be annotated;

[0123] The Region of Interest (ROI) processing module is used to extract the ROI from the full camera image or a cropped image and generate a working image by using a guided surface edge ROI extraction algorithm.

[0124] The enhanced display module is used to generate a weak feature kernel enhancement map through a weak feature enhancement algorithm;

[0125] The AI ​​draft module is used to call a semantic draft model with weak feature defects based on a semantic segmentation network to generate an editable pre-annotated mask;

[0126] The interactive correction module provides intelligent selection of brushes, erasers, and ridge-tracking algorithms, as well as interaction between defective and normal decision-making.

[0127] The persistence and export module is used to save drafts, submit final drafts, and export the region of interest image and binary mask pair.

[0128] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-view industrial semantic segmentation and annotation method for weak feature defects, characterized in that, include: Step 1: Collect a multi-view surface image sample set of industrial products; Step 2: Using a guided surface edge region of interest extraction algorithm, the region of interest is automatically extracted from the sample to obtain a working image of the region of interest; Step 3: Using a weak feature enhancement algorithm, weak feature enhancement is performed on the working image of the region of interest, and a kernel enhancement map is generated through illumination correction and local contrast enhancement. Step 4: Input the working image of the region of interest into the weak feature defect semantic draft model based on semantic segmentation network to generate an editable defect pre-annotation mask; The pre-annotation mask is only intended as a revocable draft; Step 5: Interactive correction step. Load the working image of the region of interest and the pre-annotated mask in the interactive annotation interface. Combined with the enhanced display based on the verification enhancement map, manually correct the pre-annotated mask using the pen, eraser and intelligent selection, and determine whether the current sample is defective or normal. The intelligent selection is achieved by using a ridge line tracking algorithm to automatically track and absorb bright ridges on the kernel enhancement map into strokes. Step 6: Result Export Step. Write the corrected results into the annotation workspace in a way that separates the automatic saving of the draft from the final confirmation. Simultaneously export the region of interest image with the same name and the binary mask pair to form deliverable training data.

2. The method according to claim 1, characterized in that, The sample set contains at least one complete viewpoint from the left, right, and top views, and is organized by viewpoint catalog before being loaded into the annotation workspace.

3. The method according to claim 1, characterized in that, In step 2, the method for extracting the working image of the region of interest is as follows: when the input is a full image from the camera, the region of interest on the product surface is automatically extracted based on edge guidance and geometric constraints and corrected to a unified working coordinate to output the working image of the region of interest; when the input is a cropped region of interest image, it is used as the working image of the region of interest.

4. The method according to claim 3, characterized in that, In step 2, when inputting the entire image from the camera, the algorithm for extracting the region of interest at the surface edge specifically includes: Identify the input form of the current sample, wherein the input form includes the full camera image and the cropped region of interest image; When the input is a full image from the camera, the algorithm for extracting the region of interest (ROI) on the surface edge is called according to the viewpoint: the image is smoothed and edge detected, and morphological closing operation and constrained adaptive edge fitting are combined to obtain the quadrilateral region on the product surface. The homography transformation from the source image to the working image is estimated, and perspective correction and cropping are performed on the source image based on the homography transformation to output the working image of the ROI. When the input is a cropped region of interest image, a unit transformation passthrough is used to treat the input as the working image of the region of interest. When the input format cannot be safely determined, the sample is set to a pending confirmation state.

5. The method according to claim 1, characterized in that, The weak feature enhancement algorithm specifically includes: Convert the working image of the region of interest into a grayscale image and estimate the background illumination using Gaussian blur; Illumination correction is performed based on the background illumination to eliminate uneven illumination; The image after illumination correction is subjected to positional stretching and local contrast enhancement; The image after local contrast enhancement is subjected to bilateral filtering for noise reduction and desharpening, and then further local contrast enhancement is performed to obtain the kernel enhancement image.

6. The method according to claim 5, characterized in that, The illumination correction is achieved by subtracting the Gaussian blur estimate of the background illumination from the original grayscale image and then adding the median of the background illumination.

7. The method according to claim 5, characterized in that, The quantile stretching is a dynamic range stretching process determined by a preset lower and upper quantile; the local contrast enhancement adopts a limited contrast adaptive histogram equalization.

8. The method according to claim 1, characterized in that, The pre-labeled mask generation step specifically includes: The original working image of the region of interest is used as the input to the semantic draft model, and gray-level normalization is performed on the regions prone to weak features. The normalized image is input into a semantic segmentation network based on the SegFormer-B2 architecture, and a defect probability map is obtained after activation function. The defect probability map is binarized, and the binarization result is filtered by connected component filtering to remove connected components with an area smaller than a preset threshold, thereby obtaining a defect pre-labeled mask. The defect pre-annotation mask is combined with the current manual mask and then loaded into the canvas of the interactive annotation interface as an undoable edit. When the pre-annotation mask is empty, the current sample is not automatically determined to be normal.

9. The method according to claim 1, characterized in that, The intelligent selection specifically includes: On the enhanced map, seed points are determined based on the ridge scores near the locations clicked by the annotator. The ridge tracing algorithm is used to perform bidirectional tracing along the bright ridge direction, and mask strokes are generated after attempting to cross the short gaps. The ridge score is calculated as follows: for the candidate pixel position and the tracking direction, the brightness difference on the kernel enhancement map along the two sides of the position perpendicular to the tracking direction is calculated respectively, and the smaller value of the brightness difference on both sides is taken as the ridge score of the position in that direction.

10. The method according to claim 1, characterized in that, Also includes: In the interactive correction step, the annotator can switch between viewing the enhanced review image and the original working image of the region of interest.

11. The method according to claim 1, characterized in that, The specific steps for deriving the results include: When editing occurs on the canvas in the interactive annotation interface, the draft is automatically saved according to the anti-shake timing. When the annotator confirms a defect or makes a valid decision, the final draft is submitted and the project version information is updated. Output the region of interest image with the same name and the binary mask in pairs to the results directory.

12. A multi-view industrial semantic segmentation and annotation system for implementing the method as described in any one of claims 1 to 11, characterized in that, include: The sample loading module is used to open the multi-view image data directory, construct the annotation workspace, and enumerate the samples to be annotated; The Region of Interest (ROI) processing module is used to extract the ROI from the full camera image or a cropped image and generate a working image by using a guided surface edge ROI extraction algorithm. The enhanced display module is used to generate a weak feature kernel enhancement map through a weak feature enhancement algorithm; The AI ​​draft module is used to call a semantic draft model with weak feature defects based on a semantic segmentation network to generate an editable pre-annotated mask; The interactive correction module provides intelligent selection of brushes, erasers, and ridge-tracking algorithms, as well as interaction between defective and normal decision-making. The persistence and export module is used to save drafts, submit final drafts, and export the region of interest image and binary mask pair.

Images