A few-shot defect detection method and system based on a visual language large model

Abstract

This application provides a method and system for few-sample defect detection based on a large visual language model, relating to the field of computer vision technology. The method includes: acquiring the original image of the workpiece to be inspected and extracting a first feature; generating semantic cues based on the image and / or prior information using a large visual language model, and converting them into second features that can be aligned with the first feature, and performing cross-modal fusion to obtain fused features; performing subspace projection on the fused features in the neighborhood of candidate locations, determining anisotropy measures based on local statistics, and estimating the number of endmembers, distinguishing between in-bag and out-of-bag operations under this measure, and determining the number of active endmembers in conjunction with cross-scale stability criteria; adjusting the decoding channels and thresholds according to the number of active endmembers, and feeding the results back to the fusion process; decoding the adjusted fused features and outputting a pixel-level defect segmentation map; thus improving the localization accuracy and segmentation stability of weak feature defects under the condition of limited defect samples.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a method and system for detecting defects with few samples based on a large visual language model. Background Technology

[0002] Currently, AI-based defect detection technologies mainly include three categories: object detection, semantic segmentation, and anomaly detection. Object detection and semantic segmentation rely on large-scale labeled sample training. In industrial scenarios, the scarcity of defect samples and the high cost of labeling lead to insufficient training samples and limited model performance. Although anomaly detection methods only require normal samples for training, their output is mostly heatmaps, making it difficult to achieve pixel-level accurate segmentation. They are particularly ineffective at detecting weak feature defects, such as fine cracks and texture perturbations.

[0003] In recent years, few-shot or zero-shot segmentation methods have begun to incorporate cue mechanisms to alleviate the problem of insufficient training samples. However, existing schemes often combine cue mechanisms with visual features in a coarse manner, frequently only involving categories or coarse-grained regions, making it difficult to achieve fine alignment within candidate regions. Furthermore, the lack of stable geometric criteria makes them prone to misclassification under complex textures and anisotropic conditions.

[0004] Therefore, in the scenario of few-sample defect detection, how to achieve fine alignment between visual features and semantic cues within the candidate region, and how to establish a stable geometric criterion through anisotropy measurement and endmember number estimation, in order to improve the localization accuracy and segmentation stability of weak feature defects, has become a technical problem to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a method and system for detecting defects using a small number of samples based on a large visual language model.

[0006] Firstly, this application provides a few-sample defect detection method based on a large visual language model, including:

[0007] Acquire the original image of the workpiece to be inspected; perform feature extraction on the original image to obtain the first feature characterizing the local structure and global semantics;

[0008] Based on the original image and / or prior information, a large visual language model is used to generate semantic prompts representing suspected defects; the semantic prompts are converted into second features representing the semantic attributes of suspected defects and aligned dimensionally with the first feature; the first feature and the second feature are fused across modalities to obtain fused features; the semantic prompts include: candidate location information;

[0009] Within the neighborhood of at least one candidate location determined by the semantic cues, the fused features are subspace-projected to obtain a cue-aligned embedding, and an anisotropy measure is determined based on the local statistics of the cue-aligned embedding.

[0010] Endmember number estimation is performed under the anisotropy metric, and intra-packet and / or extra-packet discrimination is performed. The number of active endmembers is determined based on the cross-scale stability criterion. The decoding channel and decision threshold are adjusted according to the number of active endmembers, and the adjustment results are fed back to the cross-modal fusion to constrain attention allocation.

[0011] The adjusted fusion features are decoded to output a pixel-level defect segmentation map of the original image.

[0012] Optionally, the semantic prompts may further include: defect category indication and morphological constraint information;

[0013] The semantic hints generated to represent suspected defects include:

[0014] The original image and prior information are provided as conditional information to the visual language model to generate semantic prompts for indicating suspected defects.

[0015] Based on the camera calibration parameters, the positional elements in the semantic prompts are mapped to at least one candidate position in the image coordinate system;

[0016] Based on the prior information, the category indication and morphological constraints in the semantic prompt are constrained or corrected so that the category indication and morphological constraints meet the numerical or logical limitations of the prior information.

[0017] Optionally, converting the semantic cue into a second feature that characterizes the suspected defect semantic attributes and is dimensionally alignable with the first feature includes:

[0018] The semantic prompts are embedded to obtain semantic vectors;

[0019] The semantic vector is converted into an alignment vector that is consistent with the first feature in the channel dimension using a mapping function;

[0020] Based on the candidate location information in the semantic prompts, a semantic weight map aligned with the image coordinates is generated, and the alignment vector is normalized and scale-aligned.

[0021] The semantic weight map and the alignment vector are combined according to the positional correspondence to form the second feature.

[0022] Optionally, the cross-modal fusion of the first feature and the second feature to obtain the fused feature includes:

[0023] At each candidate position determined by the candidate position information, a local normalized coordinate system is established based on the dominant direction, length and width in the shape constraint information, and translation, rotation and scale normalization are performed on the local region corresponding to the first feature.

[0024] Within the local normative coordinate system, at least one structural core is selected from a preset adjustable structural core group according to the defect category indication, and the structural response of the first feature is calculated using the dominant direction, width, and curvature as parameters to obtain a structural response map;

[0025] The second feature is weighted according to its position using the structural response map, and then residually combined with the first feature to form the fused feature.

[0026] Optionally, the cross-modal fusion of the first feature and the second feature to obtain the fused feature further includes:

[0027] Based on the sparse attention mask where the dominant direction and curvature construction direction in the morphological constraint information are consistent, cross-modal connections are established only in the strip-shaped neighborhood along the prediction center line.

[0028] When the defect category is indicated as a texture perturbation class, narrowband suppression is performed on the first feature in the frequency domain along the dominant direction to obtain a background suppression map;

[0029] Under the constraint of the sparse attention mask, the cross-modal attention of the first feature and the second feature is calculated, and the obtained attention result is weighted and synthesized with the background suppression map according to the position correspondence to form the fused feature.

[0030] Optionally, the step of subspatially projecting the fused features into the neighborhood of at least one candidate location determined by the semantic cues to obtain a cue-aligned embedding includes:

[0031] Within the neighborhood of each candidate position, a set of basis vectors is selected from the vector set of the second feature in that neighborhood according to a weight threshold and a linear independence criterion to span a cue-aligned subspace.

[0032] The representation of the fused feature in the neighborhood is covariance normalized to establish local alignment coordinates;

[0033] Based on the dominant direction determined by the morphological constraint information, an oblique projection operator with directional constraints is constructed to decompose the fusion feature and obtain the parallel components and anisotropic orthogonal residuals relative to the basis vector group.

[0034] The decomposition process is repeated at two or more neighborhood scales, and a stable scale is selected based on the stability criterion that the relative change in the explanation rate of the parallel components is not greater than a preset threshold.

[0035] Optionally, obtaining the cue alignment embedding further includes:

[0036] Based on the affine transformation set defined by the morphological constraint information, an alignment search is performed on the fusion features at the stable scale to determine the affine transformation that minimizes the metric of the anisotropic orthogonal residual.

[0037] The parallel portion of the basis vector group after the affine transformation is mapped back to the original coordinate system by the inverse transformation to obtain the shape-consistent component.

[0038] The parallel components, the morphologically consistent components, and the anisotropic orthogonal residuals are combined according to their positional correspondence and a preset weight to form the prompt alignment embedding.

[0039] Optionally, the neighborhood of the candidate location is a curved tubular neighborhood adaptively determined based on semantic cues and anisotropy metrics. The method for determining the neighborhood of the candidate location includes:

[0040] Starting from the candidate position, a dominant direction field is constructed based on the dominant direction determined by the morphological constraint information and combined with the principal axis of the anisotropy metric. The centerline is obtained by integrating along the dominant direction field.

[0041] The range of values ​​for the cross-sectional radius is determined based on the camera calibration parameters and the width parameter in the shape constraint information, and an anisotropic elliptical cross section is constructed in the normal direction of the centerline using the anisotropy metric as the weight.

[0042] The process is expanded segment by segment along the center line until the interpretation rate of the parallel components is lower than a preset threshold or the number of active endmembers no longer increases. Regions with brightness or saturation exceeding the threshold are then removed to obtain the neighborhood.

[0043] Optionally, the step of performing endmember number estimation and in-packet and / or out-of-packet discrimination under the anisotropy measure, and determining the number of active endmembers based on the cross-scale stability criterion, includes:

[0044] The neighborhood is divided into multiple cross sections along the centerline of the curved tubular neighborhood according to the arc length, and the cue alignment embedding in each cross section is standardized according to the anisotropic metric under the stable scale to obtain anisotropic standard coordinates.

[0045] Within each cross section, an extreme value search is performed based on the principal axis direction of the anisotropy metric to determine multiple candidate endmember points. The candidate endmembers of adjacent cross sections are matched according to arc length proximity and feature similarity to form candidate endmember trajectories. The number of continuous segments of the candidate endmember trajectories is not less than a preset threshold.

[0046] In the curved coordinate system of the curved tubular neighborhood, an affine packet is constructed for the candidate endmembers of each cross section, and the affine packets of adjacent cross sections are transmitted in parallel along the center line and the envelope is calculated to obtain the curved affine packet.

[0047] When the alignment embedding can be combined by affine curves to represent candidate endmember trajectories within the package, it is determined to be within the package if the reconstruction error does not exceed a preset error threshold; otherwise, it is determined to be outside the package.

[0048] The minimum number of candidate endmember trajectories that satisfy the cross-scale relative change rate of packet volume threshold is used as the endmember number estimation result, and the number of candidate endmember trajectories that simultaneously satisfy the sustained fragment number threshold and the relative change rate of packet volume threshold is determined as the number of active endmembers.

[0049] Secondly, this application provides a few-sample defect detection system based on a large visual language model, including:

[0050] The acquisition module is used to acquire the original image of the workpiece to be inspected; and to extract features from the original image to obtain a first feature that characterizes the local structure and global semantics.

[0051] The fusion module, based on the original image and / or prior information, uses a large visual language model to generate semantic prompts representing suspected defects; converts the semantic prompts into second features representing the semantic attributes of suspected defects and dimensionally alignable with the first feature; performs cross-modal fusion of the first feature and the second feature to obtain fused features; the semantic prompts include: candidate location information;

[0052] The processing module performs subspace projection on the fused features within the neighborhood of at least one candidate position determined by the semantic cues to obtain a cue-aligned embedding, and determines an anisotropy measure based on the local statistics of the cue-aligned embedding.

[0053] The discrimination module performs endmember number estimation and intra-packet and / or extra-packet discrimination under the anisotropy metric, and determines the number of active endmembers based on the cross-scale stability criterion; it adjusts the decoding channel and the decision threshold according to the number of active endmembers, and feeds the adjustment result back to the cross-modal fusion to constrain attention allocation;

[0054] The output module is used to decode the adjusted fusion features and output a pixel-level defect segmentation map of the original image.

[0055] Compared with existing technologies, this application introduces semantic prompts generated by a large visual language model, and the semantic prompts include at least candidate location information, enabling the model to still operate effectively under conditions where defect samples are scarce. The candidate location information narrows the detection range to the suspected area, effectively reducing the interference of full-image search, improving the localization accuracy of weak feature defects, and overcoming the shortcomings of existing anomaly detection methods that can only provide coarse-grained heatmaps and cannot locate defects at the pixel level.

[0056] Compared to existing few-shot segmentation methods that suffer from loose integration of cues and visual features, this application performs subspace projection on the fused features within the neighborhood of the candidate location and determines anisotropy measures based on local statistics. This feature design enables fine alignment of visual features and semantic cues in local geometric space, thereby better distinguishing defect features from complex texture backgrounds and improving discrimination capabilities under complex industrial surface conditions.

[0057] Furthermore, this application introduces endmember number estimation and in-packet / out-of-packet discrimination under anisotropy measurement, and combines a cross-scale stability criterion to determine the number of active endmembers, feeding this result back to the decoder for channel and threshold adjustment. This mechanism establishes a stable geometric criterion in low-sample scenarios, avoiding overfitting or noise interference under single scale and single criterion, resulting in more robust segmentation output. Therefore, this application can significantly reduce false positives and false negatives in weak feature defect detection, achieving higher localization accuracy and segmentation stability. Attached Figure Description

[0058] Figure 1 A flowchart illustrating a few-sample defect detection method based on a large visual language model, provided for embodiments of this application;

[0059] Figure 2 This is a schematic diagram of a small-sample defect detection system structure provided in an embodiment of this application;

[0060] Figure 3 A flowchart illustrating a method for generating semantic hints representing suspected defects, provided as an embodiment of this application;

[0061] Figure 4 This is a schematic diagram of a few-sample defect detection system based on a large visual language model, provided as an embodiment of this application. Detailed Implementation

[0062] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0063] See Figure 1The flowchart shown is a few-sample defect detection method based on a large visual language model provided in an embodiment of this application, including the following steps:

[0064] S101: Acquire the original image of the workpiece to be inspected; perform feature extraction on the original image to obtain the first feature representing the local structure and global semantics;

[0065] S102: Based on the original image and / or prior information, generate semantic prompts representing suspected defects using a large visual language model; convert the semantic prompts into second features representing the semantic attributes of suspected defects and dimensionally alignable with the first feature; perform cross-modal fusion of the first feature and the second feature to obtain fused features; the semantic prompts include: candidate location information;

[0066] S103: In the neighborhood of at least one candidate position determined by the semantic cues, the fused features are subspace-projected to obtain the cues-aligned embedding, and an anisotropy measure is determined based on the local statistics of the cues-aligned embedding.

[0067] S104: Perform endmember number estimation and intra-packet and / or extra-packet discrimination under the anisotropy metric, and determine the number of active endmembers based on the cross-scale stability criterion; adjust the decoding channel and decision threshold according to the number of active endmembers, and feed the adjustment result back to the cross-modal fusion to constrain attention allocation;

[0068] S105: Decode the adjusted fusion features and output a pixel-level defect segmentation map of the original image.

[0069] Regarding the above S101:

[0070] In one embodiment, the inspection platform includes an industrial camera, an imaging lens, a controllable illumination unit, and a trigger control unit. The industrial camera has a resolution of 2MP to 12MP, with a pixel size preferably in the range of 2.0μm to 5.5μm. The lens's field of view covers the defect-prone areas of the inspected workpiece, and the working distance is set according to the production line cycle time and installation conditions. The illumination method can employ coaxial light, ring-shaped oblique light, strip-shaped side light, or a combination of multiple light paths, and polarizers can be configured in highly reflective material scenarios to suppress specular highlights. The camera acquires data synchronously with the conveyor line encoder via external triggering.

[0071] To ensure that subsequent physical dimensions are referenced, camera calibration is performed before operation: camera intrinsic and distortion parameters are measured using a checkerboard or dot matrix calibration board; if necessary, extrinsic camera parameters are determined in conjunction with workpiece positioning references. Calibration results are used for subsequent pixel-to-physical size conversion and geometric correction. Acquired raw images are stored in a buffer, and each frame undergoes quality checks, including sharpness thresholds, saturation pixel ratio thresholds, and motion blur thresholds; frames failing to meet these thresholds trigger re-acquisition or an alarm.

[0072] In another embodiment, for roll-to-roll processes or continuous strip materials, the imaging system uses a linear array camera and a strip light source to achieve line-triggered sampling with line speed as a reference, and records the line speed and line delay parameters in the acquisition log for subsequent prior information retrieval.

[0073] In one embodiment, the image encoder consists of an imaging consistency preprocessing module, a multi-scale local structure subnetwork, and a global semantic subnetwork, and outputs a first feature in the feature convergence unit.

[0074] In practice, distortion correction and geometric correction are performed on the original image; brightness and white balance are normalized, and threshold truncation or reflection decomposition is applied to the bright saturated areas to reduce specular components; when the texture noise is strong, edge-preserving filtering or frequency domain band-stop suppression can be used to weaken the periodic background. After preprocessing, a normalized image is obtained, and its size can be uniformly set to H×W, for example, 2048×2048, for subsequent batch processing.

[0075] Furthermore, using the normalized image as input, pyramid feature maps {L2, L3, L4, L5} are constructed (with strides of 4, 8, 16, and 32 pixels, respectively). Each layer extracts edge, corner, texture, and fine crack responses through learnable convolutional blocks; in fine crack scenarios, adjustable directional operators can be added to shallow layers to enhance the response of 1-3 pixel wide linear targets. The number of output channels for each layer can be set to C2-C5, for example, 64, 128, 256, and 512.

[0076] A global semantic subnetwork captures long-range dependencies and overall morphological semantics. This subnetwork employs a self-attention-based encoding structure, establishing cross-regional associations through learnable positional encoding, thereby aggregating context across a large field of view. Its output is spatially aligned with the shallow layers of the local structural pyramid, and its channel count matches that of subsequent convergent units.

[0077] Multi-scale local structural features and global semantic features are aligned and fused in both scale and channel dimensions. The alignment process includes upsampling or downsampling to a uniform scale, channel linear mapping, and normalization. Subsequently, lateral connectivity and adaptive weight fusion are used to form a first feature that simultaneously contains both fine structural cues and overall semantic cues. The first feature can be represented as F1∈R^{H / 4×W / 4×C} (e.g., C=256), achieving a balance between spatial resolution and expressive power, facilitating subsequent cross-modal fusion with the second feature corresponding to semantic cues.

[0078] Where H and W represent the pixel height and width (in pixels) of the normalized input image. For example, samples of different resolutions are uniformly scaled to H×W=2048×2048 for batch processing and feature alignment; R^{H / 4×W / 4×C} represents a real-valued three-dimensional tensor (or feature map stack), with a spatial dimension of H / 4×W / 4 and a channel dimension of C; F1 represents the first feature, i.e., the visual feature tensor output by the image encoder and used for subsequent fusion with semantic cues; H / 4×W / 4 represents the feature map scale with a spatial downsampling factor of 4 (corresponding to the encoder's total stride = 4). The purpose of choosing a 1 / 4 scale is to balance computational cost and spatial resolution, ensuring sufficient pixel density for subsequent fine alignment and projection in the candidate neighborhood. If other backbone networks are used, they can also be 1 / 8, 1 / 16, etc.; {L2,L3,L4,L5}: refers to the feature layers whose spatial step size (relative to the original image) of each layer of the multi-scale pyramid is 4, 8, 16, and 32 respectively. For example, L2 corresponds to a 1 / 4 scale and L5 corresponds to a 1 / 32 scale.

[0079] In another alternative embodiment, to adapt to low-sample scenarios, a parameter-efficient domain adaptive unit can be introduced into the image encoder. For example, a learnable low-rank adaptation layer can be added only at the bottleneck layer or lateral connection to perform lightweight updates on small-scale industrial data. The update range is limited to the selected sub-module of the encoder without changing the overall structure, thereby improving the structural sensitivity to specific materials and processes without relying on large-scale annotation.

[0080] In another embodiment, for fabrics, roller prints or sprayed stripes with periodic textures, an optional frequency domain suppression branch is added between preprocessing and shallow structure extraction to narrowband attenuate the main frequency and several harmonics, and the suppressed residual components are input in parallel with the original shallow features into the local structure sub-network to reduce the interference of background periodicity on the first feature.

[0081] Regarding S102 above:

[0082] In one embodiment, after acquiring the original image, the system gathers prior information synchronized with the image. This prior information may include: intrinsic and extrinsic parameters, camera calibration parameters such as pixel size, imaging parameters such as exposure, gain, and lighting mode, motion parameters such as linear velocity and line delay, dimensional / tolerance constraints, and historical defect statistics. The original image and the prior information are then provided as conditional inputs to the visual language model to generate a semantic prompt draft for the current workpiece.

[0083] To facilitate subsequent processing, the semantic prompt draft is structured into prompt entries, which must at least contain candidate location information and may further include defect category indications and morphological constraint information. Candidate location information can be represented by a center point, bounding box, or coarse mask. When an entry uses a relative location description, such as "top left 1 / 3," or a physical size description, it is converted to pixel position and pixel scale in the image coordinate system based on camera calibration parameters. For cases with multiple candidate locations, entries are sorted and redundancies are removed based on their confidence level and consistency with prior knowledge, retaining several high-scoring candidate locations for subsequent processes.

[0084] In one embodiment, the system first embeds the text / tags of the structured prompt items to obtain a semantic vector. In scenarios that include category indicators and morphological parameters, such as dominant direction, length, width, and curvature, these parameters can be encoded according to a preset template and concatenated with the semantic vector to form a unified prompt vector. Subsequently, the prompt vector is projected onto an alignment vector with the same number of first feature channels through linear mapping or a single lightweight convolution.

[0085] A semantic weight map aligned with image coordinates is generated based on candidate location information: when a candidate location is represented by a center point, a weight distribution that decays with distance can be constructed near that point; when represented by a bounding box or coarse mask, it can be normalized to a [0,1] weight; for multiple candidate locations, their weights can be weighted and superimposed according to the confidence level of each item. The semantic weight map is upsampled / downsampled and aligned according to the spatial scale of the first feature, and the alignment vector is normalized in magnitude and aligned in scale; finally, the semantic weight map and the alignment vector are combined according to pixel position to obtain the second feature. This second feature matches the first feature in the channel dimension and remains consistent with the candidate location in the spatial dimension, which facilitates subsequent fine-grained processing in the neighborhood of the candidate location.

[0086] In one embodiment, to obtain fused features, the system first performs scale and channel consistency checks on the first and second features, and then adopts a position-selective fusion strategy: using a semantic weight map as a spatial gate, the fusion is limited to mainly occurring in the neighborhood of the candidate position; fusion weights are calculated in this neighborhood, and the fusion weights can be obtained by standardizing the pixel-wise similarity between the first and second features; the first feature is weighted according to the fusion weights, and the second feature is injected in a gated manner to form a preliminary fusion result; then the preliminary fusion result is combined with the first feature by residual analysis and normalized to obtain the fused features.

[0087] In another embodiment, cross-modal fusion can be achieved using local mask attention: using the semantic weight graph as a mask, cross-modal connections are established only in the neighborhood of the candidate position, local attention calculation is performed on the first feature and the second feature, and the attention output is added to the residual of the first feature to form the fused feature.

[0088] When the visual language large model generates multiple cue items at once, the system can perform the above transformation and fusion process on each item one by one, or aggregate them in the channel dimension after batch stacking. When only coarse-grained regional cue items can be obtained, the coarse cue morphology can be refined into candidate positions and neighborhoods first, and then processed according to the above steps.

[0089] Regarding the above S103:

[0090] For each candidate location provided by the semantic prompt, the system constructs a local neighborhood on a spatial scale aligned with the first and second features. The neighborhood selection is adaptively determined based on the dominant direction and width parameter in the morphological constraints: when a dominant direction exists, the neighborhood uses an elliptical window stretched along that direction, with its major axis aligned with the dominant direction and its minor axis set according to the width parameter or the spatial decay curve of the semantic weight; when there is no dominant direction, an isotropic circular or square window can be used. The neighborhood size can be dynamically adjusted based on the semantic confidence of the candidate location, background complexity, or pixel resolution to balance detail preservation and computational stability.

[0091] Subsequently, feature vectors for all pixel locations within the neighborhood are extracted from the fused features, and amplitude normalization and contrast equalization are performed. Abnormal pixels with saturation or severe reflection are removed to obtain a normalized feature set for subsequent projection.

[0092] Within the same neighborhood, the system selects several representative semantic vectors as a basis vector set from the second feature or its semantic prototype set, according to the semantic weight threshold and the similarity redundancy removal criteria. The upper limit of the number of this basis vector set can be configured, for example, no more than a few, to span the "cue alignment subspace" to carry the semantic components that should appear in the neighborhood.

[0093] When semantic cues include morphological constraints such as dominant direction, length, width, or curvature, the system employs a solution strategy with directional constraints. This strategy decomposes the fused features at each location within the neighborhood into two parts: one part is the "aligned portion" consistent with the semantic base, and the other part is the "anisotropic residual" that is inconsistent with the semantic base but amplified and focused on near the dominant direction. The key to this solution strategy is to minimize the overall error while giving higher priority to components consistent with the dominant direction, preventing excessive suppression of weak signals such as fine cracks and banded defects extending along specific directions. When semantic cues do not contain directional information, the system degenerates into a decomposition method without directional bias.

[0094] To balance semantic consistency and geometric deviation, the system normalizes and combines the "alignment strength" (which can be understood as the combined weight or similarity relative to each semantic basis) and "anisotropic residual strength" (which can be understood as the energy or inconsistency index of the residual in the principal axis direction of the neighborhood) of each pixel location to obtain the cue alignment embedding vector for that location. The embedding vectors of all locations in the neighborhood are arranged spatially to form a "cue alignment embedding map".

[0095] This embedding not only reflects the degree of pixel response to semantic cues, but also characterizes the abnormal trend of inconsistency with cues in a low-dimensional way, which facilitates the stable evaluation of subsequent geometric criteria.

[0096] Within each neighborhood, the system extracts local statistics based on the cue-aligned embedding graph, constructing a single scalar anisotropy measure as the geometric basis for subsequent endmember number estimation and in-packet / out-of-packet discrimination. This measure can be obtained by standardizing and weighting the following components:

[0097] Stretching index: Based on the ratio of the extent of expansion of the feature distribution in the main direction to that in the secondary direction within the neighborhood, it characterizes whether the distribution is significantly stretched along a certain direction;

[0098] Directional consistency index: compare the degree of consistency between the main direction of the neighborhood and the dominant direction of the semantic prompt. The higher the consistency, the better the neighborhood matches the morphological prior of the prompt.

[0099] Compactness index: Based on the overall dispersion or volume of characteristic distribution within the neighborhood, it distinguishes between dense defect clusters and diffuse background strays.

[0100] After quantile normalization or logarithmic compression, the aforementioned components are combined with predetermined weights to form a single anisotropy measure. This measure possesses three properties: first, it is sensitive to the dominant direction, highlighting weak targets extending along that direction; second, it is less affected by scale variations, making it easy to use in conjunction with cross-scale stability criteria; and third, its numerical value is monotonic and intuitive, facilitating the setting of thresholds or sorting rules in subsequent steps.

[0101] Understandably, the neighborhood shape, size, and adaptive rules can be configured into multiple pre-set plans based on different workpiece materials, texture complexity, and camera resolution, and the system can switch according to the scene;

[0102] It is understood that the selection of semantic basis vector groups can adopt a general process of threshold screening combined with similarity redundancy removal, or alternative schemes based on cluster centers or sparse selection can be given in other embodiments of the specification.

[0103] Regarding S104 above:

[0104] The cue-aligned embedding within the neighborhood of a candidate location can be understood as a set of points with directional preferences in the feature space. If judged directly using only a single threshold or a single class channel, it's easy to mistake subtle textures in the background for defects. The idea behind endmember number estimation is to first find the extreme representatives that can "expand" the boundary of the point set, and then use these representatives to construct a minimal yet sufficient "explanatory frame." When there are too few representatives, many points cannot be explained; when there are too many representatives, the frame expands, encompassing the background as well. Gradually introducing candidate endmembers while simultaneously monitoring whether "it truly improves the explanation" and "whether the frame over-expands" strikes a balance between these two opposing goals, thus obtaining the most suitable initial value for the number of endmembers within the neighborhood.

[0105] Here, candidate endmembers represent a set of keypoints selected from the embedding distribution within the neighborhood of the candidate location that "best represent the shape boundary"; endmember number estimation refers to determining the minimum but sufficient number of these representative points. Intuitively, the more endmembers, the wider the framework; the fewer endmembers, the weaker the explanatory power.

[0106] Furthermore, weak feature defects often extend along a dominant direction, such as fine cracks and brush marks; background textures, on the other hand, are often dispersed in multiple directions. Anisotropy measurement essentially assesses the contrast between the degree of clustering along the dominant direction and the overall dispersion. It is used both to select candidate endmembers (preferably falling in the stretched direction) and to adjust the threshold stringency (the stronger the directionality, the more critical the representative point must be to truly be "in place"). This significantly reduces the probability of directionally inconsistent noise points being included in the frame.

[0107] Furthermore, changes in neighborhood size alter the data distribution: at too small a scale, the point set is dominated by occasional noise; at too large a scale, details are obscured by the background. Cross-scale stability examines whether the number of endmembers, interpretable range, and in-packet proportions remain relatively stable as the neighborhood is slowly enlarged or shrunk. If multiple adjacent scales yield similar results, it indicates that structural defect morphology has been captured, rather than accidental noise or streaks. Using this as the ultimate criterion for the number of active endmembers can eliminate spurious judgments based on fluctuations.

[0108] The number of active endmembers essentially tells the decoder how many semantic components are actually effective in the current neighborhood. Based on this, the decoder can activate only the corresponding category channels, raise the entry threshold for other channels, and differentiate the spatial threshold between in-packet and out-of-packet operations.

[0109] On the other hand, writing back the packet / outside direction and the end-member direction to the fusion stage can restrict semantic injection to propagate only along the trusted direction, preventing prompts from "illuminating" irrelevant areas. This forms a closed-loop correction from geometric criteria to semantic reasoning.

[0110] In this context, the interpretation framework composed of candidate endmembers is used as a reference. If the embedding at each position within the neighborhood can be reasonably combined from the framework, it is considered to be within the package; otherwise, it is considered outside the package. Within the package means that it is interpreted by the current structure, while outside the package means that it is inconsistent with the current structure or is questionable.

[0111] In specific implementation, after completing the cue-aligned embedding and anisotropy metric acquisition within the candidate location neighborhood, this embodiment performs endmember number estimation and in-packet / out-of-packet discrimination within the same neighborhood. First, the system performs directional retrieval of the embedding distribution along the principal direction indicated by the anisotropy metric and its normal, extracting several significant candidate endmember points whose spacing meets the minimum separation requirement; for regions with complex textures or high noise, a density redundancy removal is performed to control the candidate size. Subsequently, a stepwise modeling approach is adopted, introducing candidate endmembers sequentially according to their salience: for each candidate introduced, it is checked whether the reconstruction error of the embedding within the neighborhood is substantially reduced and whether the envelope range maintains a controllable growth; only when both satisfy the preset criteria is it accepted. Pixels that can be reasonably covered by the current endmember set are marked as in-packet, otherwise they are marked as out-of-packet. To accommodate the distribution differences under different materials and imaging conditions, the error and envelope criteria are tightened when the anisotropy metric is strong, and moderately relaxed when it is weak, in order to reduce the risk of false rejection of weak features.

[0112] To avoid randomness caused by a single scale, the system repeats the estimation and discrimination process at multiple neighboring scales, comparing the relative changes in the number of endmembers, envelope range, and intra-packet proportion between adjacent scales. When all the above indicators are within the threshold range across consecutive scales, the endmember set is considered stable, and the set with lower complexity and better intra-packet coverage is taken as the result; its endmember count is recorded as the active endmember count, and the corresponding intra-packet / out-of-packet mask and confidence map are output. This cross-scale stability criterion can suppress the interference of local extrema and background texture on the judgment, making the determination of the endmember count repeatable and objective.

[0113] Based on the number of active endmembers, a lightweight adaptive scheduling is implemented during the decoding stage. The system designates the category channel corresponding to the active endmember as the effective channel, while the remaining channels enter a restricted mode, retaining only the basic residual contribution to avoid introducing irrelevant semantics. Simultaneously, the pixel-level decision threshold is spatially adjusted according to the in-packet / out-of-packet mask, with the threshold appropriately lowered within the packet to improve the detection rate and raised outside the packet to suppress false detections. For neighborhoods with fewer active endmembers and higher stability levels, the scores of the effective channels can be further temperature-scaled or segmented stretched to enhance the discriminative power between channels and reduce boundary-type false positives.

[0114] To form a semantically and geometrically consistent closed-loop constraint, the above results are synchronously written back to the cross-modal fusion process. During fusion, the semantic injection weights of out-of-package regions are reduced or cross-modal connections are weakened to suppress excessive responses introduced by cues at the source. In-package regions, however, a higher injection intensity is retained, and when a dominant direction or morphological constraint exists, local attention adjacency relationships are established only along the direction that aligns with it, weakening significantly deviating connections. The write-back and decoding are adaptively executed in small iterative steps until the neighborhood prediction converges or the maximum number of iterations is reached, before moving to the next candidate position. When multiple candidate positions exist in the same image, they can be processed in parallel, and conflict resolution and consistency synthesis are performed at the decoding output.

[0115] In this way, the determination of the number of endmembers no longer depends on fixed experience, but is dynamically obtained under the joint constraints of anisotropy measurement and cross-scale stability, and directly drives the coordinated adjustment of the decoding and fusion ends to improve the localization accuracy and segmentation stability of weak feature defects under the condition of few samples.

[0116] Regarding the above S105:

[0117] After determining the number of endmembers, channel gating, and threshold scheduling, the system decodes the adjusted fused features to obtain pixel-level defect segmentation maps. A typical implementation involves a lightweight decoder upsampling the fused features level by level and performing jump-joining fusion with the shallow features of the encoder to recover spatial details; subsequently, a segmentation score map for each pixel is generated through a classification prediction layer. During decoding, the aforementioned effective channel set is used, and scores are calculated only for the activated category channels; simultaneously, the in-packet / out-of-packet masks and spatialization thresholds are read, and corresponding decision thresholds and minimum connected component area constraints are applied to the in-packet and out-of-packet regions respectively, thereby obtaining initial masks for binary or multi-class categories.

[0118] To improve boundary quality, the system can perform lightweight boundary refinement or guided filtering after mask generation. For highly reflective or saturated regions, local corrections can be made according to the aforementioned visibility rules. If the same image contains multiple candidate locations, the system prioritizes and merges masks generated from different candidates based on stability level, intra-packet coverage, or endmember consistency to avoid duplicate annotations and gap omissions. Finally, the segmentation mask is restored to the original image coordinate system according to the camera calibration information, and the corresponding confidence map, vectorized contour, or minimum bounding box and other derived results are output for display on the quality inspection terminal or for use in downstream processes.

[0119] It is understandable that the above decoding process is not limited to a specific network structure. Any equivalent implementation that can complete step-by-step upsampling, channel constraints and spatial thresholding can achieve the same technical effect.

[0120] For example, see Figure 2This is a schematic diagram of a few-sample defect detection system structure provided in an embodiment of this application, wherein the overall structure can be divided into a visual branch and a semantic branch:

[0121] 1) The visual branch uses an image encoder to extract image features from the original image.

[0122] 2) The language branch first inputs the Generative Pre-trained Transformer (GPT) application programming interface (API), namely GPTAPI, to generate language cues. These cues are then processed by the cues encoder to generate language features. After feature extraction, the image and language features are fused using a feature fusion module. Finally, a mask decoder is used to decode the features and generate a detection map.

[0123] Optionally, the semantic prompts may further include: defect category indication and morphological constraint information;

[0124] join Figure 3 The flowchart below illustrates a method for generating semantic hints representing suspected defects, as provided in an embodiment of this application. The method includes steps S201 to S203, wherein:

[0125] S201: Provide the original image and prior information as conditional information to the visual language big model to generate semantic prompts for indicating suspected defects;

[0126] S202: Based on the camera calibration parameters, map the location elements in the semantic prompt to at least one candidate location in the image coordinate system;

[0127] S203: Based on the prior information, constrain or correct the category indication and morphological constraints in the semantic prompt so that the category indication and morphological constraints meet the numerical or logical limitations of the prior information.

[0128] This optional implementation aims to address the problems of overly coarse prompts, semantic drift, or difficulty in direct application caused by large visual language models under limited sample conditions. By introducing joint constraints of images and prior information, natural language or label-based outputs are regularized into computable, comparable, and verifiable structured semantic prompts, and a unified mapping from relative descriptions to image coordinate systems is completed. Furthermore, prior rules such as process, material, and dimensional tolerances are used to constrain or correct the consistency of category indicators and morphological parameters, thereby converging the candidate search space from the source, improving the reliability of candidate positions and morphological priors, and providing stable inputs for subsequent candidate neighborhood projection, anisotropy measurement, and endmember estimation, thus improving the localization accuracy and segmentation stability of weak feature defects.

[0129] In one embodiment, the system takes the original image as the main input and simultaneously retrieves prior information such as camera calibration data, imaging parameters, linear velocity and line delay, nominal product dimensions and tolerances, and historical defect statistics. These elements are then provided as conditional information to the visual language model to generate a semantic prompt draft for the current workpiece. Subsequently, the system organizes the draft into items to form a structured prompt set.

[0130] Each prompt provides at least candidate location information, and supplements it with defect category indication and morphological constraint information as needed. Candidate location information can be expressed in one of three ways: center point, bounding box, or coarse mask; category indication refers to the coding of the established classification system; morphological constraint information can include one or more of the following: dominant direction, length, width, curvature, aspect ratio, or connectivity, and is accompanied by an initial confidence level.

[0131] To unify the position description to the image coordinate system, the system converts the relative orientation or physical size into pixel position and pixel scale based on the camera calibration results; under continuous motion conditions, it combines linear velocity and line delay to achieve line shift compensation and correct the position error caused by motion.

[0132] It is important to note that for areas with high reflectivity, saturation, or occlusion, the system generates a visibility mask to correct or directly eliminate candidate execution positions that fall into the invisible area, thus avoiding the introduction of unreliable hints in unobservable areas.

[0133] Next, the system constrains and corrects the prompt entries according to a priori consistency rule base. This rule base includes the legal correspondence between materials and categories, the correlation constraints between process steps and frequently occurring parts, the range constraints between nominal dimensions and minimum inspectable dimensions, and the judgment rules for consistency with the process direction. If the category and material in an entry do not match, the weight is reduced or promoted to a higher-level category; if the morphological parameters exceed the tolerance, they are truncated to the allowable range; if the dominant direction deviates significantly from the process reference direction, it is marked as suspicious and its participation weight is reduced.

[0134] After completing the consistency processing, the system constructs a comprehensive score based on location mapping accuracy, visibility, prior consistency, and model confidence. It then sorts and merging the entries, eliminates overlapping bounding boxes and nearest neighbor center points, and finally retains several candidates with higher scores as input for subsequent steps.

[0135] In another embodiment, when the visual language large model only produces coarse-grained regions or natural language descriptions, the system first converts the description into a loose region mask, and then refines it into one or more specific candidate locations according to the above-mentioned process of location mapping, visibility correction and prior consistency constraints, and completes the category indication and morphological constraint fields.

[0136] In this way, semantic prompts are transformed from generalized descriptions into a structured, coordinate-based set of prompts that is consistent with priors. This can directly drive feature projection and geometric analysis within the candidate neighborhood, significantly reducing the risk of semantic drift and false detection.

[0137] Optionally, converting the semantic cue into a second feature that characterizes the suspected defect semantic attributes and is dimensionally alignable with the first feature includes:

[0138] The semantic prompts are embedded to obtain semantic vectors;

[0139] The semantic vector is converted into an alignment vector that is consistent with the first feature in the channel dimension using a mapping function;

[0140] Based on the candidate location information in the semantic prompts, a semantic weight map aligned with the image coordinates is generated, and the alignment vector is normalized and scale-aligned.

[0141] The semantic weight map and the alignment vector are combined according to the positional correspondence to form the second feature.

[0142] This optional implementation aims to normalize the semantic cues output by the large visual language model into a second feature that carries both suspected defective semantic attributes and is strictly aligned with the first feature in terms of channel and space. It addresses the problem of "semantic incomputability" through semantic embedding and channel mapping, and solves the problem of "spatial grounding" through a weighted graph based on candidate location information. This provides stable input for subsequent cross-modal fusion and candidate neighborhood geometric analysis, reducing false positives and false negatives caused by semantic drift.

[0143] In practical implementation, the system first embeds the structured semantic prompts. The defect categories, morphological constraints, and prior labels involved in the prompts are encoded according to a fixed template and then fed into a lightweight encoder or a lookup table-based embedding module, outputting a fixed-length semantic vector. At the output end, amplitude normalization and bias correction are performed to suppress the impact of extreme values ​​on the mapping stability.

[0144] For example, the indication of fine cracks in ceramic glaze may include elements such as "crack", "length three to five millimeters", "dominant direction along the conveying direction", and "width less than two pixels"; the indication of pitting corrosion in aluminum profiles may include elements such as "pitting corrosion", "diameter less than 0.5 millimeters", and "confidence reduction in high-reflectivity areas on the surface"; the indication of weft ripples in fabrics may include elements such as "texture disturbance" and "period range consistent with process reference".

[0145] The semantic vector is mapped to an aligned vector with the number of channels of the first feature through a linear transformation or a one-to-one convolution. Layer normalization or amplitude normalization is then applied at the output to ensure that its numerical distribution is on the same order of magnitude as the first feature. To enhance industrial domain adaptability, this mapping layer can perform lightweight updates on a small amount of domain data. The update scope is limited to the mapping layer itself and does not change the main structure of the encoder.

[0146] For example, in a brushed metal panel scene, to reduce the impact of specular highlights on subsequent blending, a short amplitude compression module can be added after the mapping layer to control the relative gain of the alignment vector in the high-reflectivity region.

[0147] The system generates a semantic weight map that is consistent with the image coordinates based on the candidate location information and aligns it with the spatial scale of the first feature.

[0148] For example, when the candidate location is the center point, a weight distribution with a high center value and a smooth decay outwards is generated around that point; if the indication includes a dominant direction and width, the weight distribution is stretched into a thin strip along the dominant direction and narrowed laterally to match the width constraint. When used for detecting fine cracks in ceramics, this strip-shaped distribution can better cover the crack direction.

[0149] For example, when the candidate location is a bounding box, the weight is higher inside the box, resulting in a smooth transition at the edge and a rapid decay outside the box. This is useful for scenarios involving spraying debris or stains, as it preserves a necessary transition zone at the boundary, reducing artifacts caused by hard edges.

[0150] For example, when the candidate position is a coarse mask, the weights are uniformly higher inside the mask, and then gradually decrease at the edges based on confidence and shape constraints. This facilitates the subsequent implementation of local attention within the mask area when dealing with large-area texture perturbations in fabrics.

[0151] In highly reflective materials or saturated regions, the system introduces a visibility mask to lower the weights; in roll-to-roll operation, the system combines linear velocity and line delay information to perform line shift compensation on the weight positions to avoid alignment errors caused by movement.

[0152] Furthermore, the system broadcasts the alignment vector to a spatial grid consistent with the first feature according to its position, and modulates it pixel by pixel by the semantic weight map to obtain a tensor that simultaneously possesses semantic attributes and spatial orientation, which serves as the second feature.

[0153] When there are multiple candidate locations, corresponding weight maps and alignment vectors are generated respectively. They are then weighted and synthesized according to the confidence of the entries, and then uniformly normalized to form a single second feature for cross-modal fusion.

[0154] For example, when both "pitting" and "fine crack" warnings appear simultaneously in the same metal plate image, the system generates weight maps and alignment vectors for the two types of warnings respectively, first forming partial second features within their respective candidate ranges, and then synthesizing them according to category confidence to avoid mutual interference; the same partial synthesis strategy can also be used in scenarios where PCB silkscreen scratches and solder pad stains coexist.

[0155] In this way, semantic prompts are stably transformed into second features that are consistent with the first feature channel and strictly aligned with the image coordinates. This second feature retains both the semantic information of category and morphology, and clearly limits the scope of influence to the spatial neighborhood of the candidate region. It provides a directly callable tensor input for subsequent cross-modal fusion, candidate neighborhood subspace projection, and anisotropy measurement calculations, and is applicable to various industrial surface inspection scenarios such as ceramic glaze, anodized metal, textile fabric, and PCB screen printing.

[0156] Optionally, the cross-modal fusion of the first feature and the second feature to obtain the fused feature includes:

[0157] At each candidate position determined by the candidate position information, a local normalized coordinate system is established based on the dominant direction, length and width in the shape constraint information, and translation, rotation and scale normalization are performed on the local region corresponding to the first feature.

[0158] Within the local normative coordinate system, at least one structural core is selected from a preset adjustable structural core group according to the defect category indication, and the structural response of the first feature is calculated using the dominant direction, width, and curvature as parameters to obtain a structural response map;

[0159] The second feature is weighted according to its position using the structural response map, and then residually combined with the first feature to form the fused feature.

[0160] This alternative implementation addresses the problem of weak features, elongated or banded defects being submerged in complex backgrounds. It proposes establishing a local canonical coordinate system at candidate locations and using an adjustable structural kernel to calculate the directional, scaled structural response of the first feature. This response is then used to apply position-corresponding weight modulation to the second feature, and finally, the results are combined with the residuals of the first feature. Through a chain of "coordinate alignment - structural amplification - semantic modulation - robust fusion," semantic cues are transformed into geometrically selective gains, effectively suppressing background clutter inconsistent with the shape and improving the fusion result's ability to distinguish different morphologies such as fine cracks, narrow-band scratches, and pitting clumps.

[0161] In practice, for each candidate location, the system reads the dominant direction, length, and width from the morphological constraints and extracts the local region centered on that candidate from the first feature. Then, three normalization steps are performed:

[0162] Translation normalization moves the candidate center to the local coordinate origin, which facilitates subsequent kernel alignment and response calculation.

[0163] Rotation normalization aligns the vertical axis of the local coordinate system with the dominant direction and makes the horizontal axis orthogonal to it, ensuring that the structural core operates in the sense of "alignment".

[0164] Scale normalization scales local areas to a uniform reference scale according to the length and width in the shape constraints, preventing mismatch between the core and the target under different working conditions.

[0165] For example, in the scenario of fine cracks on ceramic glaze, the dominant direction is usually consistent with the direction of the crack and the width is small; in the scenario of scratches on anodized aluminum plates, the dominant direction is often similar to the direction of process movement and the width is slightly larger than the fine crack; in the scenario of weft ripples on fabrics, the length is significantly greater than the width and the direction is in a stable relationship with the weaving reference.

[0166] Furthermore, within the specified coordinate system, based on the defect category indication and morphological parameters, the system selects at least one kernel from a preset structural kernel group to calculate the structural response of the first feature, thereby obtaining a structural response map. The kernel group includes, but is not limited to, the following three types:

[0167] A linear, slender core is used to enhance the cracks and extremely narrow scratches. This core exhibits significant longitudinal passband and transverse suppression characteristics, with a continuous longitudinal response and rapid transverse decay. The sensitive distance can be controlled by length and width parameters.

[0168] A strip core is used for strip defects of moderate width. This core retains limited passbands in both the longitudinal and transverse directions. The transverse bandwidth is consistent with the width in the shape constraint, while the longitudinal bandwidth is limited by length or connectivity requirements.

[0169] The clumping core is used for pitting corrosion, slag formation, and other near-clumpy targets. This core enhances the localized dense area with a centrosymmetric design and provides a smooth transition to the surrounding gradient.

[0170] For example, linear, slender cores are preferred when detecting microcracks in glass substrates; clumpy cores are preferred when detecting spray coating slag; and strip-shaped cores are preferred when detecting narrow-band scratches on brushed metal surfaces. If the shape constraint includes curvature information, the system makes slight turns according to the given curvature as the core advances longitudinally to adapt to the slightly bent defect trajectory.

[0171] Furthermore, the system performs one-to-one spatial modulation of the structural response map and the second feature within a normalized coordinate system: pixels with high responses have their local weights increased, while pixels with low responses have their injection intensity reduced. After modulation, the semantic enhancement result is residually combined with the first feature, and the amplitude and contrast are renormalized to obtain the fused feature. The residual path ensures that the stable expression of the visual branch is preserved even when the structural response is insufficient or the indication is uncertain, avoiding false detections caused by excessive semantic amplification.

[0172] For example, in textile fabrics, if weft ripples and warp textures interweave, the structural response will take a high value in the thin strips that are in the same direction as the dominant direction, and a low value in the texture area that is not in the same direction; the second feature is thus enhanced only in the thin strip area, and the final fusion result is only sensitive to the "same direction" disturbance, thereby suppressing the pseudo signal that is the same as the weaving texture but in the opposite direction.

[0173] Furthermore, when the semantic prompt is missing the width parameter, the system uses a two-level kernel for adaptive estimation: first, a wider kernel is used for coarse detection, and then a narrower kernel is used for fine detection. The intersection of the two is taken as the effective enhancement band. When there is uncertainty in the dominant direction, the system sets several discrete deflection angles near the dominant direction, calculates the structural response respectively, uses the maximum response principle to determine the enhancement direction, and records the deflection angle size as an uncertainty index for subsequent anisotropy measurement and endmember estimation reference.

[0174] For example, in the case of minor scratches on PCB silkscreen, the directional fluctuation between manufacturing batches is often within ten degrees. This approach can compensate for missed detections caused by directional drift without significantly increasing the amount of computation.

[0175] In this way, by establishing a standardized coordinate system and unifying the scale, the structural kernel and the target geometry are aligned in direction and size, avoiding insufficient response caused by "kernel mismatch". Through the selection of an adjustable structural kernel and parameterized response, the details of "morphological consistency" are physically amplified before fusion. By modulating the second feature with the structural response and performing residual back-fusion, semantic information is grounded in spatially verifiable geometric support, while maintaining the robustness of the visual base. This process can be reused in various industrial surface scenarios, enabling enhancement of extremely narrow targets such as fine cracks, as well as targeted amplification of clumps and narrow-band targets, providing high-quality input for subsequent subspace projection and geometric criterion construction of candidate neighborhoods.

[0176] Optionally, the cross-modal fusion of the first feature and the second feature to obtain the fused feature further includes:

[0177] Based on the sparse attention mask where the dominant direction and curvature construction direction in the morphological constraint information are consistent, cross-modal connections are established only in the strip-shaped neighborhood along the prediction center line.

[0178] When the defect category is indicated as a texture perturbation class, narrowband suppression is performed on the first feature in the frequency domain along the dominant direction to obtain a background suppression map;

[0179] Under the constraint of the sparse attention mask, the cross-modal attention of the first feature and the second feature is calculated, and the obtained attention result is weighted and synthesized with the background suppression map according to the position correspondence to form the fused feature.

[0180] This optional implementation addresses two types of easily confused scenarios: one is fine cracks or banded scratches extending along the dominant direction, with clear geometric orientation but complex surrounding background, often "diluted" by global attention; the other is texture perturbations with obvious periodic components, such as weft ripples in fabrics, equidistant stripes in brushed metal, and granular stripes after spraying, which often mask weak defects with strong periodic signals. Therefore, before cross-modal fusion, a sparse attention mask is constructed with the dominant direction and curvature as constraints, establishing connections only along the predicted center line of the banded neighborhood, thus reducing interactions in irrelevant directions. In texture perturbation scenarios, narrowband frequency domain suppression along the dominant direction is performed on the first feature to generate a background suppression map, removing interference from the dominant frequency and several neighboring frequency bands. Then, cross-modal attention is calculated under mask constraints and synthesized with the background suppression map by position weight, thereby obtaining a fusion feature that balances directional consistency and texture suppression, significantly improving the signal-to-noise ratio and separability of weak targets.

[0181] In practical implementation, at each candidate location, the system generates a local predicted centerline based on the dominant direction and width in the morphological constraints, and slowly updates the direction according to the curvature to fit the slight bend. A strip-shaped neighborhood is constructed with the centerline as the central axis, the bandwidth of which is given by the width parameter in the morphological constraints, and the edges are smoothly decayed to reduce hard edge effects. Subsequently, cross-modal connections are only established within this strip-shaped neighborhood; connections outside the neighborhood are not constructed, thus forming a sparse attention mask with consistent orientation. This mask can be configured with sparsity rate and bandwidth upper limit, facilitating control of computational load and the probability of incorrect edge connections on rough surfaces or under high-reflectivity processes.

[0182] For example, in a scene depicting fine cracks in ceramic glaze, the center line deflects slowly along the crack direction, and the bandwidth is set to two to three times the crack width to ensure coverage of the main crack while suppressing textures on both sides. In a scene depicting strip-shaped scratches on an anodized aluminum plate, the center line aligns with the scratch trail, and the bandwidth is slightly narrower to avoid including the brushed background in the connecting edges.

[0183] When the defect category is indicated as texture perturbation, the system extracts local stripes aligned with the dominant direction around the candidate location, performs directional spectral analysis on the first feature to determine the dominant frequency and several neighboring frequency bands, and generates a background suppression map. The suppression map assigns lower gain to the dominant frequency and its nearest neighboring frequency bands, maintains normal gain in the remaining frequency bands, and uses a gradual transition at the boundaries to avoid ringing. For high-reflectivity or saturated regions, the suppression intensity is moderately reduced to preserve visible information. Finally, the suppression map is mapped back to the spatial domain and multiplied with the first feature according to their corresponding positions to obtain the first feature after background suppression.

[0184] For example, in the weft-direction wavy fabric scenario, the dominant direction is aligned with the warp and weft references, and the dominant frequency is determined by the periodic peak position obtained from spline fitting, with the suppression bandwidth covering the main peak and the weak peaks on both sides. In the sprayed texture disturbance scenario, if there is a bimodal period, the suppression map dynamically covers the two main peak intervals, ensuring that the interference of particle strips on subsequent segmentation is sufficiently weakened.

[0185] Under the constraint of the aforementioned sparse mask, the system computes cross-modal attention between the first and second features, establishing cross-modal interaction only within the strip-shaped neighborhood, while maintaining the original residual channel outside the neighborhood. The attention result represents the consistency strength between semantic cues and visual evidence. Subsequently, the attention result and the background suppression map are weighted and synthesized according to their corresponding positions: the attention contribution is reduced in the suppressed region, and the attention contribution is maintained or moderately increased in the unsuppressed region, resulting in a fused feature with consistent direction and clean texture. To ensure stability, the system uses slowly varying weights at the edges of the strip-shaped neighborhood and performs weighted fusion according to candidate confidence at the overlapping locations of multiple candidate positions, performing non-maximum suppression when necessary to avoid redundant enhancement.

[0186] For example, in the scenario of fine scratches on a brushed stainless steel plate, attention propagates only along the center of the scratch. The background suppression map removes the dominant frequency of equidistant brushing, and the synthesized fused feature presents a continuous high-response thin band, which facilitates stable operation of subsequent subspace projection and endmember estimation. In the scenario of weft-shaped ripples on a fabric, attention is preserved in regions with consistent periods and suppressed in regions with strong periodic noise. The final fused feature presents continuous and directional energy protrusions at the defect perturbation points.

[0187] Orientation consistency mask and narrowband suppression are two components that can be enabled independently. For crack and scratch scenarios with only orientation priors and no significant periodic interference, only the mask can be enabled; for texture disturbances with significant periodic interference and weak orientation priors, the suppression map can be enabled first; in most industrial surface scenarios, the combined use of both is more effective. The centerline generation, bandwidth setting, and curvature update of the mask can be obtained according to the orientation reference provided by the process or estimated from the local gradient field; the selection of the dominant frequency and bandwidth setting of narrowband suppression can be obtained through online statistics or small sample calibration.

[0188] Optionally, the step of subspatially projecting the fused features into the neighborhood of at least one candidate location determined by the semantic cues to obtain a cue-aligned embedding includes:

[0189] Within the neighborhood of each candidate position, a set of basis vectors is selected from the vector set of the second feature in that neighborhood according to a weight threshold and a linear independence criterion to span a cue-aligned subspace.

[0190] The representation of the fused feature in the neighborhood is covariance normalized to establish local alignment coordinates;

[0191] Based on the dominant direction determined by the morphological constraint information, an oblique projection operator with directional constraints is constructed to decompose the fusion feature and obtain the parallel components and anisotropic orthogonal residuals relative to the basis vector group.

[0192] The decomposition process is repeated at two or more neighborhood scales, and a stable scale is selected based on the stability criterion that the relative change in the explanation rate of the parallel components is not greater than a preset threshold.

[0193] This alternative implementation addresses the challenges of unstable basis due to noise and texture interference in local feature distribution under few-sample conditions, and the weakening of weak targets along the direction in ordinary orthogonal projection. It proposes a process based on directional constraint decomposition and cross-scale stable selection within the semantic subspace. By selecting a set of semantic basis vectors anchored to the second feature in the neighborhood of candidate locations, performing directional constraint oblique projection on the fused features, and using cross-scale stability of the explanatory power as a criterion, robust cue-aligned embeddings are obtained. This provides geometrically verifiable input for subsequent endmember number estimation and in-bag or out-of-bag discrimination.

[0194] In practice, within each neighborhood determined by the candidate location information, the system first sorts the local vector sets of the second feature from highest to lowest semantic weight, eliminating obviously redundant nearest neighbor vectors and retaining several representative vectors to form a candidate set. Then, using linear correlation as a criterion, the independence of each representative vector is examined, skipping vectors that are too similar to the selected set, until a preset upper limit is reached. This process yields a set of linearly independent, semantically significant basis vectors, used to span the cue alignment subspace.

[0195] For example, in the scenario of fine cracks in ceramic glaze, the candidate set is more biased towards semantic responses along the crack direction; in the scenario of pitting corrosion on aluminum plate, the candidate set is biased towards local clump shapes; in the scenario of fabric texture disturbance, the candidate set will cover one or two main texture directions.

[0196] To mitigate differences in gain and contrast across different channels, the system performs amplitude normalization and contrast equalization on the fused features within the neighborhood. It calculates neighborhood statistics and normalizes the covariance accordingly, obtaining locally aligned coordinates with consistent numerical range and scale. Robust statistics are employed for pixel locations that are significantly saturated, highly reflective, or occluded to avoid interference from extreme values ​​in coordinate establishment.

[0197] In roll-to-roll operation, if there is displacement caused by line delay and linear velocity, the system completes line shift compensation before entering this step to ensure that the local alignment coordinates are consistent with the actual geometry.

[0198] When the semantic cues provide one of the dominant direction, length, width, or curvature, the system employs a directionally constrained oblique projection within the local alignment coordinates: components aligned with the dominant direction are given higher priority to ensure the preservation of weak targets extending along that direction; components significantly deviating from the dominant direction are incorporated into the anisotropic residuals. If the cues do not provide directional information, the system degenerates into a neutral decomposition. This decomposition yields two types of results at each pixel location: one is the alignment portion relative to the semantic basis vector set, which can be understood as the combination coefficients of several semantic prototypes; the other is a strength or directional summary of the anisotropic residuals, used to characterize deviations that are inconsistent with the cues but have discriminative value.

[0199] For example, in a fine crack scenario, the aligned portion is a continuous band along the crack direction, and the anisotropic residual is mainly distributed in the texture background on both sides of the crack; in a pitting scenario, the aligned portion is concentrated in the center of the mass, and the anisotropic residual is distributed on the brushed or sprayed texture.

[0200] The system performs a uniform amplitude and range calibration on the two types of results and combines them according to position to form a cue-aligned embedding map. This embedding not only reflects the response strength of pixels to the semantic basis, but also summarizes the anisotropic biases that are inconsistent with the cue in a low-dimensional way, which facilitates the stable evaluation and ranking of subsequent geometric criteria.

[0201] To avoid the randomness brought about by a single neighborhood size, the system repeats the basis selection, coordinate establishment, and orientation constraint decomposition process at at least two different neighborhood scales. Then, it compares the changes in explanatory power, envelope range, and in-bag percentage between adjacent scales: when these indicators remain within the threshold range across consecutive scales, it is considered to have reached stability; if multiple sets of candidate results meet the conditions, the set with lower complexity and more comprehensive coverage is selected as the output for the stable scale. Finally, the system outputs a cue-aligned embedding graph at the stable scale, and uses explanatory power, in-bag percentage, and other indicators as quality markers, passing them to the subsequent endmember estimation stage.

[0202] For example, in the fabric warp and weft perturbation scenario, smaller neighborhoods are often unstable due to local texture undulations. After expanding to a medium neighborhood, the explanation rate and the proportion within the envelope change more slowly between adjacent scales, so this scale is selected as a stable scale. In the pitting scenario, an excessively large neighborhood will introduce redundant textures, and the explanation rate will fluctuate. The system will then fall back to a smaller scale to obtain more reliable results.

[0203] In this way, the cue-aligned embedding strikes a balance between semantic consistency and geometric verifiability, providing a stable and comparable basis for subsequent endmember number estimation and threshold adaptation.

[0204] Optionally, obtaining the cue alignment embedding further includes:

[0205] Based on the affine transformation set defined by the morphological constraint information, an alignment search is performed on the fusion features at the stable scale to determine the affine transformation that minimizes the metric of the anisotropic orthogonal residual.

[0206] The parallel portion of the basis vector group after the affine transformation is mapped back to the original coordinate system by the inverse transformation to obtain the shape-consistent component.

[0207] The parallel components, the morphologically consistent components, and the anisotropic orthogonal residuals are combined according to their positional correspondence and a preset weight to form the prompt alignment embedding.

[0208] Within the candidate neighborhood, targets such as fine cracks and banded scratches often exhibit slight rotation, anisotropic scaling, or shearing. If only the decomposition results in a fixed direction are used as the basis, these "acceptable deformations" are easily misclassified as residuals, leading to unstable endmember estimations. Therefore, this optional implementation introduces a morphologically constrained affine alignment search at a stable scale. The fused features are aligned according to the allowed deformation range to find the deformation that minimizes the anisotropic orthogonal residual. The aligned parallel parts are then inverted back to the original coordinate system to obtain the morphologically consistent component. Finally, the parallel component, the morphologically consistent component, and the anisotropic orthogonal residual are synthesized by position, ensuring that the cue-aligned embedding retains both components consistent with the semantic base and bias information that is inconsistent with the cue but has discriminative value. This process significantly reduces misclassifications caused by slight deformations and improves the stability of subsequent geometric criteria and decoding scheduling.

[0209] In practical implementation, under the stable scale obtained above, the system provides the allowable range of affine transformations based on morphological constraints. The rotation angle oscillates slightly around the dominant direction, with the amplitude determined by the directional confidence provided by the morphological constraints; the anisotropic scaling coefficients in the horizontal and vertical directions vary narrowly around the indicated length and width; the shearing amount is used to compensate for slight skewing or to fit segments with slowly changing curvature. To avoid excessive search overhead, the system adopts a two-stage strategy from coarse to fine: first, a rapid evaluation is performed on a sparser grid to select several candidate deformations; then, a refined search is performed near these candidates, and the stability of the anisotropic orthogonal residuals at each step is judged. If the residual decreases slowly and the resulting boundary expansion is significant, the search is terminated early to avoid overfitting.

[0210] For example, in the scenario of fine cracks in ceramic glaze, the rotation range is usually a tiny angle, that is, very close to zero, the lateral scaling is adjusted very slightly around the crack width, and the shearing amount is a very small value; in the scenario of strip scratches on aluminum plate, the allowable range of rotation and shearing is slightly wider to accommodate slight deflections caused by trailing shadows or oblique movements; in the scenario of fabric texture disturbance, if the prompt gives a constraint that the period is consistent with the warp and weft directions, the rotation and shearing are strictly tightened, and the main adjustment is to adjust the anisotropic scaling to align the strip width.

[0211] For each candidate deformation, the system performs directional constraint decomposition on the fused features under that deformation, calculates the strength of the anisotropic orthogonal residuals, and selects the set of deformations with the smallest residuals as the optimal alignment. Then, it extracts the component consistent with the semantic basis and maps it back from the alignment coordinate system to the original coordinate system, obtaining the morphologically consistent component. This component can be understood as the effective component consistent with the cue within the allowed deformation set, correcting deviations caused by slight rotation, anisotropic scaling, or shearing, while maintaining consistency with the original image coordinates. This facilitates direct integration with subsequent steps such as in-package / out-of-package discrimination and decoding channel gating.

[0212] Furthermore, the system performs positional synthesis of parallel components, morphologically consistent components, and anisotropic orthogonal residuals at the pixel level. The synthesis weights are determined by three types of information: first, the alignment quality score, derived from the residual strength and boundary expansion under optimal deformation; second, the morphological consistency score, derived from the degree of conformity between the morphological constraints and the current component in terms of direction and width; and third, the neighborhood stability score, derived from the stability level of the explanatory power during the stable scale selection process.

[0213] During synthesis, the weight of morphologically consistent components is prioritized at the defect body, while anisotropic orthogonal residuals are appropriately preserved at boundaries and disputed regions to facilitate the differentiation of background and interference in endmember estimation. In regions where multiple candidate locations overlap, weights are applied according to candidate confidence levels, and non-maximum suppression is performed once to avoid redundant enhancement or excessive weight accumulation.

[0214] In the straightening and alignment of fine cracks in ceramics, affine search typically involves rotations at extremely small angles and subtle lateral scaling. The extracted morphologically consistent components, after alignment, form continuous thin bands along the crack direction, overcoming the fracture response caused by slight serpentine bending. In scratching anodized aluminum plates, due to the oblique cutting during the scratching process, the optimal deformation often includes a small shear term; alignment can significantly reduce the virtual width caused by ghosting. In weft texture perturbation of fabrics, if the strip width deviates from the suggested width, appropriate anisotropic scaling can concentrate the response on the true perturbation band, thereby reducing the interference of the periodic background on subsequent endmember estimation.

[0215] The step size and range of the parameters in the affine search can be calibrated offline with a small number of samples or through online statistical adaptive calibration. To avoid deformation being used to fit noise, a trade-off should be struck between the reduction in the target residual and the complexity of the deformation when selecting the optimal deformation: in addition to requiring the relative reduction of the reconstruction residual to exceed a preset threshold, regularization should be applied to the deformation matrix, such as limiting the norm of the deformation matrix relative to the identity matrix to no more than a certain value, limiting the maximum rotation / scaling / shearing amount, and verifying that the stability at adjacent scales or in the neighborhood meets a predetermined threshold. This prevents the fitting of sporadic noise or highly reflective artifacts with excessive deformation.

[0216] In this way, the cue alignment embedding achieves higher robustness and comparability in small sample scenarios with deformation, providing a reliable basis for subsequent endmember number estimation, intra-packet or extra-packet discrimination, and adaptive decoding threshold.

[0217] Optionally, the neighborhood of the candidate location is a curved tubular neighborhood adaptively determined based on semantic cues and anisotropy metrics. The method for determining the neighborhood of the candidate location includes:

[0218] Starting from the candidate position, a dominant direction field is constructed based on the dominant direction determined by the morphological constraint information and combined with the principal axis of the anisotropy metric. The centerline is obtained by integrating along the dominant direction field.

[0219] The range of values ​​for the cross-sectional radius is determined based on the camera calibration parameters and the width parameter in the shape constraint information, and an anisotropic elliptical cross section is constructed in the normal direction of the centerline using the anisotropy metric as the weight.

[0220] The process is expanded segment by segment along the center line until the interpretation rate of the parallel components is lower than a preset threshold or the number of active endmembers no longer increases. Regions with brightness or saturation exceeding the threshold are then removed to obtain the neighborhood.

[0221] In this optional implementation, the curved tubular neighborhood is jointly driven by semantic cues and anisotropic measurements, adaptively generated at candidate positions and expanded segment by segment along the target direction. The system first constructs a dominant orientation field by combining two types of information: one is the dominant orientation, allowed curvature, and width range derived from semantic cues; the other is the principal axis orientation and intensity of anisotropic measurements within local blocks. These two are fused with confidence weights, prioritizing semantics in simple texture areas and relying more on the metric principal axis in complex texture areas. Abrupt changes are eliminated through local vector smoothing and small-angle constraints, while the rate of orientation change is limited according to the semantically allowed curvature. Subsequently, step-by-step tracking is performed along the orientation field from the candidate position, with real-time re-estimation of the orientation within the forward sector to overcome noise effects. If necessary, the orientation is spliced ​​with the forward trajectory after reverse tracking to obtain a continuous centerline traversing the defective subject. For segments near image edges or low-confidence areas, the system temporarily suspends expansion and resumes after neighborhood recovery to maintain trajectory coherence and reliability.

[0222] After obtaining the centerline, the system converts the width range given by the semantics into pixel scale based on the camera calibration results, and constructs elliptical cross-sections along the centerline normal with weights based on the strength of anisotropy. When anisotropy is strong, the lateral radius is taken as the upper limit of the width, and the longitudinal radius is moderately narrowed to suppress lateral leakage; when anisotropy is weak, the lateral radius is taken as the median value and a smooth band is set at the edge to reduce the hard cutting effect of the neighborhood boundary. Each discrete point of the centerline generates an elliptical cross-section orthogonal to the tangent. If the semantic hint contains curvature information, the cross-section normal is updated segment by segment along the centerline tangent to fit the slightly curved target; if there is local specular highlighting or saturation, the system calls the visibility mask to reduce the cross-section or reduce the cross-section weight at the corresponding position to avoid including unobservable areas in the analysis. The curved tubular neighborhood is formed by splicing these segmented, slightly overlapping elliptical cross-sections along the centerline to form a simply connected tube.

[0223] The neighborhood expansion process is constrained by both geometric and semantic evidence. Within each newly added cross-sectional segment, the system calculates the interpretation rate of the cue-aligned embedding and reads the number of active endmembers for that segment. Expansion stops when the interpretation rate continuously drops below a threshold in consecutive segments, or when the number of active endmembers no longer increases within consecutive segments. To avoid premature termination, the system sets protection conditions for a minimum effective length and a minimum number of segments. For segments with brightness or saturation exceeding the threshold, if the highlight ratio is too large, the expansion is terminated at that point; if it is only localized reflection, interpolation from neighboring cross-sections is used to fill the gap, thus achieving a balance between fidelity and coherence.

[0224] After expansion, the system performs a thinning and hole filling process on the tube to eliminate small protrusions and voids. It also performs appropriate backsliding and resampling at self-intersections or sharp bends, and finally exports the neighborhood mask and centerline trajectory. It also includes metadata such as the cross-sectional radius, anisotropy intensity and interpretation rate of each segment, providing direct input for subsequent endmember estimation and threshold adaptation.

[0225] In the scenario of fine cracks in ceramic glaze, the directional field typically oscillates slowly along the conveying direction, and the centerline deflects smoothly accordingly. The lateral radius is set according to the upper limit of the crack width, while the longitudinal radius is moderately narrowed to reduce interference from glaze reflection. Expansion stops where the explanatory power at the crack end significantly decreases. In the scenario of strip-shaped scratches on anodized aluminum plates, the centerline may experience slight shearing, and the cross-sectional normal is updated with the tangential direction to match the trailing shadow. For bright segments caused by the trailing shadow, the cross-section is reduced, and continuity is maintained by interpolation between adjacent segments. In the scenario of weft disturbance in fabrics, the centerline is consistent with the warp and weft references, and the cross-sectional bandwidth is derived from the prompt period. When the number of active end elements remains unchanged within adjacent segments, it is determined that the boundary of sufficient coverage has been reached, avoiding excessive expansion of the periodic background.

[0226] In this way, the curved tubular neighborhood can accurately fit the true shape of slender or curved defects under conditions of few samples, while suppressing high reflection and periodic textures, providing a stable and verifiable analysis region for subspace projection and geometric criterion calculation within the candidate neighborhood.

[0227] Optionally, the step of performing endmember number estimation and in-packet and / or out-of-packet discrimination under the anisotropy measure, and determining the number of active endmembers based on the cross-scale stability criterion, includes:

[0228] The neighborhood is divided into multiple cross sections along the centerline of the curved tubular neighborhood according to the arc length, and the cue alignment embedding in each cross section is standardized according to the anisotropic metric under the stable scale to obtain anisotropic standard coordinates.

[0229] Within each cross section, an extreme value search is performed based on the principal axis direction of the anisotropy metric to determine several candidate endmember points. The candidate endmembers of adjacent cross sections are matched according to arc length proximity and feature similarity to form candidate endmember trajectories. The number of continuous segments of the candidate endmember trajectories is not less than a preset threshold.

[0230] In the curved coordinate system of the curved tubular neighborhood, an affine packet is constructed for the candidate endmembers of each cross section, and the affine packets of adjacent cross sections are transmitted in parallel along the center line and the envelope is calculated to obtain the curved affine packet.

[0231] When the alignment embedding can be combined by affine curves to represent candidate endmember trajectories within the package, it is determined to be within the package if the reconstruction error does not exceed a preset error threshold; otherwise, it is determined to be outside the package.

[0232] The minimum number of candidate endmember trajectories that satisfy the cross-scale relative change rate of packet volume threshold is used as the endmember number estimation result, and the number of candidate endmember trajectories that simultaneously satisfy the sustained fragment number threshold and the relative change rate of packet volume threshold is determined as the number of active endmembers.

[0233] This optional implementation addresses the issue of unstable endmember count determination within the tubular neighborhood of curves, which is susceptible to noise, texture, and bending geometry. It proposes an integrated workflow: "centerline segmentation - anisotropic standard coordinates - directional extremum search - trajectory matching - curve affine packet - cross-scale stability." This workflow can provide verifiable endmember counts and active endmember counts in bent targets and complex backgrounds, and provides a reliable geometric basis for subsequent decoding and fusion write-back.

[0234] In its implementation, the system uses the centerline of the curved tubular neighborhood as a reference, dividing the neighborhood into several cross-sections at equal intervals according to arc length. Each cross-section covers a segment of an elliptical cross section with slight overlap to ensure the continuity of the tubular structure. For each cross-section, cue alignment is embedded, and standardization is performed based on anisotropy metrics: using robust statistics within the section as the central reference, amplitude normalization and direction alignment are performed according to robust scaling factors in the principal and secondary directions, resulting in easily comparable anisotropic standard coordinates. Conservative weighting or removal strategies are used for clearly saturated or reflective points to prevent outliers from dominating the standardization results.

[0235] Within each cross section, the system performs an extremum search along both the principal axis and the normal direction, using the anisotropy metric as a reference. The search employs local nonmaximum suppression to eliminate spurious peaks and sets a minimum spacing to avoid dense repetitions. The resulting candidate endmember points are sorted by significance. Trajectory matching is then performed between adjacent cross sections: constrained by arc length proximity, feature similarity, and directional coherence, bidirectional nearest matching is used, supplemented by short-track cleanup, to form continuous candidate endmember trajectories. The trajectories must meet a minimum number of consecutive segments threshold to filter out occasional extrema.

[0236] For example, in fine cracks on ceramic glaze, candidate end-members often appear on both sides of the center line, and after matching, they form a thin band-like trajectory that extends along the crack direction; in strip-like scratches on anodized aluminum plates, the trajectory is more parallel and longer; in weft disturbances of fabrics, the trajectory is often periodically distributed, and the system filters them by the number of continuous pieces and the directional coherence.

[0237] In the curvilinear coordinate system of the curved tubular neighborhood, the system constructs an intra-segment affine packet for the candidate endmember set of each cross-section, obtaining the minimum envelope of the segment in anisotropic standard coordinates. Then, parallel transmission is performed along the centerline: using the tangent and normal directions of the centerline as the alignment reference between segments, the affine packets of adjacent cross-sections are smoothly aligned according to curvature, and a joint envelope is calculated, accumulating segment by segment to form a curved affine packet that runs through the entire segment. To avoid excessive envelope expansion, the system evaluates the expansion magnitude at each joint; if it exceeds the upper limit, it reverts to the previous state and retryes with a smaller step size. For obvious bends, short segments are spliced ​​together, and a local contraction is performed at the splice point to ensure that the envelope bends synchronously with the centerline and does not "go straight."

[0238] After obtaining the affine packet, the system evaluates all cue alignment embeddings within the cross-section one by one. If an embedding at a certain position can provide a sufficiently small approximation error from the affine combination of candidate endmember trajectories within the affine packet, it is marked as in-packet; otherwise, it is marked as out-of-packet. The error threshold is adaptively adjusted according to the strength of anisotropy: the stronger the directionality, the stricter the threshold; the weaker the directionality, the more relaxed the threshold. The system simultaneously outputs the in-packet mask and confidence map, providing input for subsequent decoding channel gating and spatialized threshold scheduling.

[0239] To improve robustness, the system repeats the above process at at least two scales. Changes in three metrics between adjacent scales are compared: the number of endmember trajectories, the volume or area of ​​the affine packet, and the packet's intra-packet coverage. Stability is determined when the relative changes of all three metrics within consecutive scales do not exceed a threshold. From the set satisfying the stability condition, the set with fewer trajectories and better intra-packet coverage is preferentially selected as the endmember number estimation result. Finally, the number of trajectories that simultaneously meet both the continuous slice threshold and the cross-scale volume change threshold is determined as the active endmember number, and its stability level is output as a scheduling signal for subsequent decoding and fusion write-back.

[0240] For example, if the weft perturbation of a fabric yields similar trajectory numbers and enclosed proportions at both small and medium scales, then the medium scale is selected as the stable scale; for ceramic fine cracks, the envelope volume increases significantly at large scales while the enclosed proportion decreases, thus reverting to small and medium scales as the stable solution.

[0241] Parameters such as arc length segmentation step size, minimum number of continuous segments, non-maximum suppression interval, upper limit of expansion, and cross-scale change threshold can be calibrated offline using a small number of labeled samples, or fine-tuned online using historical statistics after going online.

[0242] In this way, endmember estimation is upgraded from static planar geometry to dynamic evaluation under the framework of curved geometry, which significantly reduces the instability caused by noise and curved structures, and obtains verifiable and quantifiable active endmember numbers.

[0243] Based on the same inventive concept, this application also provides a few-sample defect detection system based on a large visual language model, corresponding to a few-sample defect detection method based on a large visual language model. Since the principle of the system in this application is similar to the few-sample defect detection method based on a large visual language model described above, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be described again.

[0244] Reference Figure 4 The diagram shown is a schematic of a few-sample defect detection system based on a large visual language model provided in an embodiment of this application. The system includes:

[0245] The acquisition module 10 is used to acquire the original image of the workpiece to be inspected; and to extract features from the original image to obtain a first feature that characterizes the local structure and global semantics.

[0246] The fusion module 20, based on the original image and / or prior information, uses a large visual language model to generate semantic prompts representing suspected defects; converts the semantic prompts into second features representing the semantic attributes of suspected defects and dimensionally alignable with the first feature; performs cross-modal fusion of the first feature and the second feature to obtain fused features; the semantic prompts include: candidate location information;

[0247] The processing module 30 performs subspace projection on the fused features within the neighborhood of at least one candidate position determined by the semantic cues to obtain a cue-aligned embedding, and determines an anisotropy measure based on the local statistics of the cue-aligned embedding.

[0248] The discrimination module 40 performs endmember number estimation and intra-packet and / or extra-packet discrimination under the anisotropy metric, and determines the number of active endmembers based on the cross-scale stability criterion; it adjusts the decoding channel and the decision threshold according to the number of active endmembers, and feeds the adjustment result back to the cross-modal fusion to constrain attention allocation;

[0249] Output module 50 is used to decode the adjusted fusion features and output a pixel-level defect segmentation map of the original image.

[0250] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A few-shot defect detection method based on a visual language large model, characterized in that, include: Acquire the original image of the workpiece to be inspected; Feature extraction is performed on the original image to obtain a first feature representing the local structure and global semantics; Based on the original image and / or prior information, a large visual language model is used to generate semantic prompts representing suspected defects; the semantic prompts are then converted into second features representing the semantic attributes of suspected defects and dimensionally aligned with the first feature. The first feature and the second feature are fused across modes to obtain the fused feature; The semantic prompts include: candidate location information; Within the neighborhood of at least one candidate location determined by the semantic cues, the fused features are subspace-projected to obtain a cue-aligned embedding, and an anisotropy measure is determined based on the local statistics of the cue-aligned embedding. Endmember number estimation is performed under the anisotropy metric, and intra-packet and / or extra-packet discrimination is performed. The number of active endmembers is determined based on the cross-scale stability criterion. The decoding channel and decision threshold are adjusted according to the number of active endmembers, and the adjustment results are fed back to the cross-modal fusion to constrain attention allocation. The adjusted fusion features are decoded to output a pixel-level defect segmentation map of the original image.

2. The method of claim 1, wherein the method is based on a visual language large model. The semantic prompts also include: defect category indications and morphological constraint information; The semantic hints generated to represent suspected defects include: The original image and prior information are provided as conditional information to the visual language model to generate semantic prompts for indicating suspected defects. Based on the camera calibration parameters, the positional elements in the semantic prompts are mapped to at least one candidate position in the image coordinate system; Based on the prior information, the category indication and morphological constraints in the semantic prompt are constrained or corrected so that the category indication and morphological constraints meet the numerical or logical limitations of the prior information.

3. The method for detecting defects with few samples based on a large visual language model according to claim 1, characterized in that, The step of converting the semantic prompt into a second feature that characterizes the semantic attributes of a suspected defect and is dimensionally alignable with the first feature includes: The semantic prompts are embedded to obtain semantic vectors; The semantic vector is converted into an alignment vector that is consistent with the first feature in the channel dimension using a mapping function; Based on the candidate location information in the semantic prompts, a semantic weight map aligned with the image coordinates is generated, and the alignment vector is normalized and scale-aligned. The semantic weight map and the alignment vector are combined according to the positional correspondence to form the second feature.

4. The method for detecting defects with few samples based on a large visual language model according to claim 2, characterized in that, The cross-modal fusion of the first feature and the second feature to obtain the fused feature includes: At each candidate position determined by the candidate position information, a local normalized coordinate system is established based on the dominant direction, length and width in the shape constraint information, and translation, rotation and scale normalization are performed on the local region corresponding to the first feature. Within the local normative coordinate system, at least one structural core is selected from a preset adjustable structural core group according to the defect category indication, and the structural response of the first feature is calculated using the dominant direction, width, and curvature as parameters to obtain a structural response map; The second feature is weighted according to its position using the structural response map, and then residually combined with the first feature to form the fused feature.

5. The method for detecting defects with few samples based on a large visual language model according to claim 2, characterized in that, The cross-modal fusion of the first feature and the second feature to obtain the fused feature includes: Based on the sparse attention mask where the dominant direction and curvature construction direction in the morphological constraint information are consistent, cross-modal connections are established only in the strip-shaped neighborhood along the prediction center line. When the defect category is indicated as a texture perturbation class, narrowband suppression is performed on the first feature in the frequency domain along the dominant direction to obtain a background suppression map; Under the constraint of the sparse attention mask, the cross-modal attention of the first feature and the second feature is calculated, and the obtained attention result is weighted and synthesized with the background suppression map according to the position correspondence to form the fused feature.

6. A few-sample defect detection method based on a large visual language model according to claim 4 or 5, characterized in that, The step of subspace projection of the fused features within the neighborhood of at least one candidate location determined by the semantic cues to obtain a cue-aligned embedding includes: Within the neighborhood of each candidate position, a set of basis vectors is selected from the vector set of the second feature in that neighborhood according to a weight threshold and a linear independence criterion to span a cue-aligned subspace. The representation of the fused feature in the neighborhood is covariance normalized to establish local alignment coordinates; Based on the dominant direction determined by the morphological constraint information, an oblique projection operator with direction constraints is constructed to decompose the fusion feature and obtain the parallel components and anisotropic orthogonal residuals relative to the basis vector group. The decomposition process is repeated at two or more neighborhood scales, and a stable scale is selected based on the stability criterion that the relative change in the explanation rate of the parallel components is not greater than a preset threshold.

7. The method for detecting defects with few samples based on a large visual language model according to claim 6, characterized in that, The method of obtaining cue alignment embedding also includes: Based on the affine transformation set defined by the morphological constraint information, an alignment search is performed on the fusion features at the stable scale to determine the affine transformation that minimizes the metric of the anisotropic orthogonal residual. The parallel portion of the basis vector group after the affine transformation is mapped back to the original coordinate system by the inverse transformation to obtain the shape-consistent component. The parallel components, the morphologically consistent components, and the anisotropic orthogonal residuals are combined according to their positional correspondence and a preset weight to form the prompt alignment embedding.

8. The method for detecting defects with few samples based on a large visual language model according to claim 6, characterized in that, The neighborhood of the candidate location is a curved tubular neighborhood adaptively determined based on semantic cues and anisotropy metrics. The method for determining the neighborhood of the candidate location includes: Starting from the candidate position, a dominant direction field is constructed based on the dominant direction determined by the morphological constraint information and combined with the principal axis of the anisotropy metric. The centerline is obtained by integrating along the dominant direction field. The range of values ​​for the cross-sectional radius is determined based on the camera calibration parameters and the width parameter in the shape constraint information, and an anisotropic elliptical cross section is constructed in the normal direction of the centerline using the anisotropy metric as the weight. The process is expanded segment by segment along the center line until the interpretation rate of the parallel components is lower than a preset threshold or the number of active endmembers no longer increases. Regions with brightness or saturation exceeding the threshold are then removed to obtain the neighborhood.

9. A few-sample defect detection method based on a large visual language model according to claim 8, characterized in that, The step of performing endmember number estimation and in-packet and / or out-of-packet discrimination under the anisotropy measure, and determining the number of active endmembers based on the cross-scale stability criterion includes: The neighborhood is divided into multiple cross sections along the centerline of the curved tubular neighborhood according to the arc length, and the cue alignment embedding in each cross section is standardized according to the anisotropic metric under the stable scale to obtain anisotropic standard coordinates. Within each cross section, an extreme value search is performed based on the principal axis direction of the anisotropy metric to determine multiple candidate endmember points. The candidate endmembers of adjacent cross sections are matched according to arc length proximity and feature similarity to form candidate endmember trajectories. The number of continuous segments of the candidate endmember trajectories is not less than a preset threshold. In the curved coordinate system of the curved tubular neighborhood, an affine packet is constructed for the candidate endmembers of each cross section, and the affine packets of adjacent cross sections are transmitted in parallel along the center line and the envelope is calculated to obtain the curved affine packet. When the alignment embedding can be combined by affine curves to represent candidate endmember trajectories within the package, it is determined to be within the package if the reconstruction error does not exceed a preset error threshold; otherwise, it is determined to be outside the package. The minimum number of candidate endmember trajectories that satisfy the cross-scale relative change rate of packet volume threshold is used as the endmember number estimation result, and the number of candidate endmember trajectories that simultaneously satisfy the sustained fragment number threshold and the relative change rate of packet volume threshold is determined as the number of active endmembers.

10. A few-sample defect detection system based on a large visual language model, characterized in that, include: The acquisition module is used to acquire the original image of the workpiece to be inspected; Feature extraction is performed on the original image to obtain a first feature representing the local structure and global semantics; The fusion module, based on the original image and / or prior information, uses a large visual language model to generate semantic prompts representing suspected defects; and converts the semantic prompts into second features representing the semantic attributes of suspected defects and dimensionally alignable with the first feature. The first feature and the second feature are fused across modes to obtain the fused feature; The semantic prompts include: candidate location information; The processing module performs subspace projection on the fused features within the neighborhood of at least one candidate position determined by the semantic cues to obtain a cue-aligned embedding, and determines an anisotropy measure based on the local statistics of the cue-aligned embedding. The discrimination module performs endmember number estimation and intra-packet and / or extra-packet discrimination under the anisotropy metric, and determines the number of active endmembers based on the cross-scale stability criterion; it adjusts the decoding channel and the decision threshold according to the number of active endmembers, and feeds the adjustment result back to the cross-modal fusion to constrain attention allocation; The output module is used to decode the adjusted fusion features and output a pixel-level defect segmentation map of the original image.

Images