A method, apparatus, equipment and medium for detecting minute defects in bottle bodies
By constructing an image prototype library and a visual language model, and combining counterfactual reasoning and defect morphology feature residual enhancement, the problems of high false detection rate and high migration cost in bottle detection are solved, and high-precision detection of minute defects is achieved.
Patent Information
- Application Number
- CN202611332321.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-31
- Publication Date
- 2026-11-17
AI Technical Summary
Existing bottle inspection methods suffer from high rates of missed detections, high rates of false detections, and high migration costs. They are particularly difficult to effectively identify and adapt to new bottle types, new processes, and new defect types when faced with minute defects.
By building an image prototype library, utilizing visual encoders and visual language models, and combining counterfactual reasoning and defect morphology feature residual enhancement, high-precision detection of minute defects in bottles can be achieved, reducing migration costs.
It effectively reduced the false detection rate and the missed detection rate, reduced the adaptation cost to new bottle types and new processes, and improved the accuracy and efficiency of detecting minor defects in the bottle.
Smart Images

Figure CN123118538A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of defect detection technology, and specifically relates to a method, device, equipment and medium for detecting minor defects in bottle bodies. Background Technology
[0002] Bottles (glass bottles, PET plastic bottles, ampoules, pharmaceutical vials, etc.) serve as bulk packaging carriers for industries such as food, pharmaceuticals, daily chemicals, and beverages. Microscopic defects on their surfaces and walls (microcracks, microbubbles, microscratches, pinpoint stains, uneven wall thickness, and overflow in mold markings, etc.) directly affect the product's sealing, cleanliness, and safety. Especially in the pharmaceutical and food sectors, a single undetected microcrack can lead to the scrapping of an entire batch of products and cause serious safety and compliance incidents. Therefore, highly sensitive, low-miss-rate online visual inspection of bottles has become a crucial aspect of production line quality control.
[0003] Existing traditional machine vision methods rely on manually designed feature operators and a large number of empirical parameters, making them prone to missing detections of subtle defects with weak signals and low contrast. They are also highly sensitive to bottle reflections, surface distortions, and textured backgrounds (such as threads and mold numbers), resulting in high false positive rates and difficulty in transferring detection across bottle types. Deep learning-based supervised detection methods (YOLO, Faster R-CNN, Mask R-CNN, etc.) are effective for common defects, but heavily rely on large-scale, pixel-level labeled defect samples. However, subtle defect samples are naturally scarce in industrial settings, exhibit a long tail distribution, and new bottle types, processes, and defect types are constantly emerging, requiring repeated model retraining and incurring high deployment and maintenance costs. Unsupervised / self-supervised anomaly detection methods (PatchCore, PaDiM, CFlow, etc.): While reducing reliance on defect annotation, recall significantly decreases when dealing with subpixel-level micro-defects because the defect signal in the visual feature space is drowned out by normal background noise. Furthermore, these methods lack position-aware modeling of "what a normal bottle should look like at this location," easily misclassifying normal structural variations such as bottle neck threads and bottom mold numbers as defects. Open detection methods based on Visual Language Models (VLM): In recent years, VLMs have been introduced into industrial inspection to achieve zero-sample, interpretable defect recognition. However, the visual encoder of a general-purpose VLM has a very weak feature response to micro-defects—the energy of micro-defects in the visual token sequence is diluted by a large number of normal tokens, making it almost impossible for the downstream language decoder (LLM) to capture effective defect cues. This results in extremely low usability for directly deploying VLMs for detecting micro-defects in bottles.
[0004] Based on the aforementioned existing technologies, current intelligent bottle detection methods suffer from drawbacks such as high false negative rates, high false positive rates, and high migration costs. Summary of the Invention
[0005] In view of the above problems, this application proposes a method, apparatus, equipment, and medium for detecting minute defects in bottles. This can reduce the migration costs of existing detection methods and improve the detection accuracy of minute defects in bottles.
[0006] One embodiment of this application provides a method for detecting minute defects in a bottle, including:
[0007] We acquire bottle images with defect annotations to build an image prototype library. Each bottle image has a pixel-level defect mask.
[0008] Each bottle image is divided and encoded using a visual encoder to obtain the visual embedding vector of the image block.
[0009] Project the defect mask image of the bottle image in the visual embedding vector onto the image block grid, and calculate the defect occupancy ratio of each image block;
[0010] Based on the defect occupancy ratio, capture the visual patterns of recurring defects on the bottle and construct a global defect-related prototype space.
[0011] The defect image selector based on counterfactual reasoning calculates counterfactual scores for input bottle images and filters images with scores exceeding a threshold and their shape support neighborhoods.
[0012] For images and morphological support neighborhoods that exceed the threshold score, defect morphological feature residual enhancement is performed to capture subtle defects through a visual language model.
[0013] Furthermore, the step of acquiring bottle images with defect annotations to construct an image prototype library, where each bottle image corresponds to a pixel-level defect mask, includes:
[0014] Acquire multiple bottle images, and perform defect masking on each bottle image to obtain a bottle image with defect annotations;
[0015] For each bottle image, defect masking is performed, including pixel-level identification of defective areas in the bottle image as white areas and normal areas as black areas.
[0016] Furthermore, the step of dividing and encoding each bottle image using a visual encoder to obtain a visual embedding vector for the image patch includes:
[0017] The image of the bottle with the defect mask is divided into multiple image blocks;
[0018] Visual embedding vectors of image blocks are obtained by visual embedding encoding based on the size of the divided image blocks.
[0019] The position index of the image block is consistent with the position of the bottle image in the visual embedding encoding.
[0020] Furthermore, the step of projecting the defect mask image of the bottle image in the visual embedding vector onto the image patch grid and calculating the defect occupancy ratio of each image patch includes:
[0021] The defect occupancy ratio of each image block is determined by the ratio of the defect pixels in the defect mask of each image block on the image block grid to the total pixels of each image.
[0022] Image block defects are determined based on the defect occupancy ratio of each image block.
[0023] Furthermore, the step of determining image block defects based on the defect occupancy ratio of each image block includes:
[0024] A defect ratio threshold is set. When the defect occupancy ratio of each image block is greater than the defect ratio threshold, the defect occupancy ratio of each image block is determined as the defect-related region.
[0025] When the defect occupancy ratio of each image block is less than or equal to the defect ratio threshold, the defect occupancy ratio of each image block is determined to be a normal area.
[0026] Furthermore, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space includes:
[0027] Average pooling is performed on the image patch embedding vectors containing defective regions in each bottle image;
[0028] The locations of image blocks in the bottle image where the proportion of defects exceeds the defect ratio threshold are marked as a location set;
[0029] The image patch embedding vectors containing defective regions are summed element by element to obtain the sum of the defect embedding vectors;
[0030] The mean vector of the overall defect appearance of the bottle image is obtained by dividing the sum of the defect embedding vectors by the modulus of the position vector.
[0031] Furthermore, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes:
[0032] The mean vector of the overall defect appearance of each bottle image is clustered using the K-Means algorithm to obtain the defect prototype of the defect visual pattern.
[0033] In the K-Means algorithm, each mean vector in the clustering represents a typical defective visual pattern.
[0034] Furthermore, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes:
[0035] Support memory for constructing a defect prototype of a defective visual pattern;
[0036] All original image patch embedding vectors are assigned to corresponding defect prototypes; for each defect prototype, the original defect vectors assigned to the corresponding prototype are collected.
[0037] The original defect vector stores the embeddings of real samples, which are used to provide defect reference directions during subsequent residual enhancement.
[0038] Furthermore, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes:
[0039] Construct a position-aware prototype space for a normal bottle;
[0040] The image positions of each normal bottle image are superimposed in a series based on the index set of normal bottle images;
[0041] Clustering is performed on the position embedding vectors of image blocks in normal bottle images to obtain the normal bottle prototype space.
[0042] Furthermore, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes:
[0043] Construct a priori defects;
[0044] Based on the relative row and column offsets of the defects, a set of locations for all defective image blocks in the image is constructed using an indicator function;
[0045] The defect image is assigned to the defect prototype and normalized to obtain the defect prior of the defect prototype.
[0046] Furthermore, the counterfactual reasoning-based defect image selector calculates counterfactual scores for the input bottle images and filters images and shape support neighborhoods that exceed a threshold score, including:
[0047] Similar prototypes are retrieved from the global defect prototype space using the cosine similarity function in the similarity function.
[0048] Calculate the similarity score of the similar prototypes, and determine the similarity between the image patches in the global defect prototype space and known defect patterns based on the similarity score.
[0049] Furthermore, the counterfactual reasoning-based defect image selector calculates counterfactual scores for the input bottle images and filters images and shape support neighborhoods that exceed a threshold score, including:
[0050] The cosine similarity function in the similarity function is used to determine similar normal prototypes from the normal prototype space corresponding to the image patch location.
[0051] Calculate the consistency score of the normal prototype, and determine the appearance of the normal bottle at the image block location based on the consistency score.
[0052] Furthermore, the counterfactual reasoning-based defect image selector calculates counterfactual scores for the input bottle images and filters images and shape support neighborhoods that exceed a threshold score, including:
[0053] Counterfactual reasoning is performed based on the defect similarity score of the bottle image and the consistency score of the normal prototype of the bottle image.
[0054] Scores in counterfactual reasoning are calibrated using defect morphology calibration.
[0055] Furthermore, the calibration of scores in counterfactual reasoning through defect morphology calibration includes:
[0056] In counterfactual reasoning, select high-confidence seeds with confidence values greater than the seed threshold, and determine the defect prototype of each high-confidence seed image block;
[0057] For each high-confidence seed, the defect morphology prior of its prototype is used to propagate the confidence to the neighborhood to diffuse the defect morphology.
[0058] Based on the calibrated comprehensive score, image blocks with final scores exceeding the threshold and their defect morphology support neighborhoods are selected.
[0059] Furthermore, defect morphology feature residual enhancement is performed on the image and morphology support neighborhood that exceed the threshold score, and subtle defects are captured through a visual language model, including:
[0060] Determine the defect prototype corresponding to each image patch that exceeds the threshold, and retrieve similar reference embeddings from the support memory model of the defect prototypes of the image patches that exceed the threshold.
[0061] For each selected image block location and for each image block within its defect morphology neighborhood, calculate the defect morphology enhancement weights;
[0062] Residual enhancement is performed on each image block that requires enhancement weights for defect morphology.
[0063] Furthermore, the residual enhancement of each image block requiring enhancement of defect morphology weights includes:
[0064] Determine the prototype support memory corresponding to the nearest seed image block of the defective image block; calculate the residual vector of the prototype support memory corresponding to the image block;
[0065] Residual enhancement is performed based on the original image patch embedding vector, global enhancement intensity coefficient, overall enhancement magnitude, and feature offset.
[0066] The enhanced and unenhanced image patch embedding vectors are recombined in their original positional order to form an enhanced encoding sequence, which is then fed into a frozen visual language model. The language decoder in the visual language model decodes the enhanced encoding sequence to capture subtle defects in the bottle image.
[0067] Based on the same inventive concept, a second aspect of the embodiments of this application provides a device for detecting minute defects in a bottle, comprising:
[0068] The image prototype construction unit is used to acquire bottle images with defect annotations to build an image prototype library. Each bottle image has a pixel-level defect mask.
[0069] The embedding vector encoding unit is used to divide and encode each bottle image by a visual encoder to obtain the visual embedding vector of the image block.
[0070] The defect occupancy calculation unit is used to project the defect mask image of the bottle image in the visual embedding vector onto the image block grid and calculate the defect occupancy ratio of each image block.
[0071] The global defect construction unit is used to capture the visual patterns of repeated defects on the bottle according to the proportion of defects occupied, and to construct a global defect-related prototype space.
[0072] The counterfactual calculation unit is used to calculate the counterfactual score of the input bottle image by the defective image selector based on counterfactual reasoning, and to filter images and shape support neighborhoods that exceed the threshold score.
[0073] The shape residual enhancement unit is used to perform defect shape feature residual enhancement on the image and shape support neighborhood that exceed the threshold score, and to capture subtle defects through a visual language model.
[0074] Furthermore, the image prototype construction unit is also used to acquire multiple bottle images and perform defect masking on each bottle image to obtain a bottle image with defect annotations.
[0075] For each bottle image, defect masking is performed, including pixel-level identification of defective areas in the bottle image as white areas and normal areas as black areas.
[0076] Furthermore, the embedded vector coding unit is also used to divide the bottle image with defect mask into multiple image blocks;
[0077] Visual embedding vectors of image blocks are obtained by visual embedding encoding based on the size of the divided image blocks.
[0078] The position index of the image block is consistent with the position of the bottle image in the visual embedding encoding.
[0079] Furthermore, the defect occupancy calculation unit is also used to determine the defect occupancy ratio of each image block based on the ratio of the defect pixels of the defect mask of each image block on the image block grid to the total pixels of each image.
[0080] Image block defects are determined based on the defect occupancy ratio of each image block.
[0081] Furthermore, the global defect construction unit is also used to perform average pooling on the image block embedding vector containing the defect region in each bottle image;
[0082] The locations of image blocks in the bottle image where the proportion of defects exceeds the defect ratio threshold are marked as a location set;
[0083] The image patch embedding vectors containing defective regions are summed element by element to obtain the sum of the defect embedding vectors;
[0084] The mean vector of the overall defect appearance of the bottle image is obtained by dividing the sum of the defect embedding vectors by the modulus of the position vector.
[0085] Furthermore, the global defect construction unit is also used to cluster the mean vector of the overall defect appearance of each bottle image using the K-Means algorithm to obtain the defect prototype of the defect visual pattern.
[0086] In the K-Means algorithm, each mean vector in the clustering represents a typical defective visual pattern.
[0087] Furthermore, the counterfactual calculation unit is also used to retrieve similar prototypes from the global defect prototype space using the cosine similarity function in the similarity function;
[0088] Calculate the similarity score of the similar prototypes, and determine the similarity between the image patches in the global defect prototype space and known defect patterns based on the similarity score.
[0089] Furthermore, the counterfactual calculation unit is also used to determine similar normal prototypes from the normal prototype space corresponding to the image patch position using the cosine similarity function in the similarity function;
[0090] Calculate the consistency score of the normal prototype, and determine the appearance of the normal bottle at the image block location based on the consistency score.
[0091] Furthermore, the morphology residual enhancement unit is also used to determine the defect prototype corresponding to each image patch that exceeds the threshold, and to retrieve similar reference embeddings from the support memory model of the defect prototypes of the image patches that exceed the threshold.
[0092] For each selected image block location and for each image block within its defect morphology neighborhood, calculate the defect morphology enhancement weights;
[0093] Residual enhancement is performed on each image block that requires enhancement weights for defect morphology.
[0094] Furthermore, the morphology residual enhancement unit is also used to determine the prototype support memory corresponding to the nearest seed image block of the defective image block; and to calculate the residual vector of the prototype support memory corresponding to the image block.
[0095] Residual enhancement is performed based on the original image patch embedding vector, global enhancement intensity coefficient, overall enhancement magnitude, and feature offset.
[0096] The enhanced and unenhanced image patch embedding vectors are recombined in their original positional order to form an enhanced encoding sequence, which is then fed into a frozen visual language model. The language decoder in the visual language model decodes the enhanced encoding sequence to capture subtle defects in the bottle image.
[0097] Based on the same inventive concept, a third aspect of the present application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;
[0098] Memory, which stores computer programs;
[0099] When the processor executes the program stored in memory, it implements a method for detecting minute defects in the bottle.
[0100] Based on the same inventive concept, a third aspect of the present application provides a computer-readable storage medium, characterized in that it stores a computer program, which, when executed by a processor, implements a method for detecting minute defects in a bottle.
[0101] The beneficial effects of this application are as follows: Based on the above technical solution, by using counterfactual defect localization and adopting a counterfactual comparison mechanism of dual-path scoring (defect similarity vs. normal consistency), it effectively distinguishes between "real micro-defects" and "normal but atypical texture variations", avoiding misjudging normal structural variations of the bottle body (such as bottle mouth thread texture and bottle bottom mold number) as defects, and reducing the false detection rate and false detection rate.
[0102] Hot-swappable production lines and zero-sample retraining: During inference, there is no need for backpropagation or parameter fine-tuning of the original model. It can be seamlessly connected to any deployed industrial VLM or visual backbone network. When facing new bottle types or new processes, the defect prototype library can be rebuilt offline once and then put online without stopping production for annotation and retraining. This significantly reduces the cost of production line switching and quality maintenance, thereby reducing migration costs.
[0103] By using reference embeddings retrieved from real samples, a mild residual perturbation is applied to the selected patch along the direction of "pointing to the same type of defect", and the defect morphology prior driven by data is smoothly diffused in space, which amplifies the defect signal while preserving the normal anatomical structure of the bottle, thus improving the detection accuracy of minor defects in the bottle.
[0104] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description and the accompanying drawings. Attached Figure Description
[0105] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0106] Figure 1 A flowchart of a method for detecting minute defects in bottle bodies is shown;
[0107] Figure 2 A schematic diagram of a device for detecting minute defects in a bottle is shown.
[0108] Figure 3 A schematic diagram of an electronic device is shown, which performs a method for detecting minor defects in a bottle. Detailed Implementation
[0109] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0110] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," "longitudinal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings.
[0111] One embodiment of this application provides a method for detecting minute defects in a bottle body, see [link to previous section]. Figure 1 ,include:
[0112] S101: Obtain bottle images with defect annotations to build an image prototype library. Each bottle image has a pixel-level defect mask.
[0113] S102: Each bottle image is divided and encoded using a visual encoder to obtain the visual embedding vector of the image block;
[0114] S103: Project the defect mask image of the bottle image in the visual embedding vector onto the image block grid, and calculate the defect occupancy ratio of each image block;
[0115] S104: Capture the visual patterns of recurring defects on the bottle based on the defect occupancy ratio, and construct a global defect-related prototype space;
[0116] S105: The defect image selector based on counterfactual reasoning calculates counterfactual scores for the input bottle images and filters images with scores exceeding the threshold and the shape support neighborhood.
[0117] S106: Perform defect morphology feature residual enhancement on the image and morphology support neighborhood that exceed the threshold score, and capture subtle defects through a visual language model.
[0118] The bottle micro-defect detection method described in the embodiments of this specification is typically applied in a server, which can be a cloud server or a local server. The server can be an electronic device such as a laptop, desktop computer, tablet computer, or all-in-one computer.
[0119] Specifically, this application acts as a "defect signal relay enhancer" between the visual encoder and the language decoder in the system. It neither replaces nor modifies the parameters of the Vision Encoder (ViT) and Large Language Model Decoder (LLM) of the VLM, but inserts itself into the visual token path between them in a "non-intrusive forward pass with zero parameter updates". Specifically, the workflow of the original VLM is "image → Vision Encoder → Patch Embeddings → LLM → text output". Since the general Vision Encoder is pre-trained on natural image corpora, its feature channels have extremely weak response energy to minor defects in the bottle (subpixel-level micro-cracks, low-contrast bubbles, etc.). When the defect token is fed into the LLM, it has been diluted by a large number of normal background tokens, causing the LLM to "not see" the defect clues even with its strong semantic reasoning ability, and ultimately outputting a general erroneous conclusion such as "the bottle looks normal". After this method is implemented, the workflow is rewritten as follows: "Image → Vision Encoder → Patch Embeddings → Counterfactual Defect Localization Module (screening suspicious tokens) → Defect Morphology-Guided Residual Enhancement Module (amplifying suspicious tokens along the direction of real defects) → Enhanced Patch Embeddings → LLM → Text Output". The counterfactual defect localization module uses an offline-built "global defect prototype + location-aware normal prototype" to perform dual-path comparison scoring on each Patch token, retaining only a few high-confidence tokens that "look like defects and cannot be explained by the normal appearance of that location". The defect morphology-guided residual enhancement module then retrieves reference embeddings from the support memory of real defect samples, applies controllable residual perturbations to these tokens along the "real direction pointing to the same type of defect", and smoothly diffuses them spatially to the coherent neighborhood by the defect morphology prior. The performance improvement path is reflected in three levels: (1) Signal-to-noise ratio level - the visual token sequence after location screening and residual enhancement significantly amplifies the energy of the fine defect signals that were originally submerged in the background before they are sent into the LLM. The LLM can then directly generate a fine-grained report of "what kind of defect, where it is located, and its severity" based on its general semantic capabilities; (2) False detection suppression level - counterfactual reasoning establishes an independent benchmark for "normal appearance at this location". Normal structural variations such as bottle neck threads and bottle bottom mold numbers are not considered. High and excluded from the enhancement target, reducing false detection from the source; (3) Deployment cost level - since VLM and LLM are always frozen, the new bottle model only needs to rebuild the defect prototype library offline once, without retraining any neural network parameters, avoiding the high cost of repeated retraining on small samples and long-tail micro defects of traditional supervised solutions, so that the large model capability can be truly implemented on the industrial production line.
[0120] The entire system is divided into two phases: Phase 1: Bottle defect prototype library construction (offline, one-time construction); Phase 2: Online production line reasoning (frozen VLM + counterfactual defect localization module + defect morphology-guided residual enhancement module, zero backpropagation, zero fine-tuning).
[0121] It should be noted that the following abbreviations are used in this application: Patch (image patch), Embedding (embedding vector), Vision Encoder (visual encoder, such as ViT), VLM (visual language model), LLM (large language model decoder), Sigmoid (sigmoid activation function), K-Means (K-means clustering), Top-M (top M similarity rankings); and the symbols used in this application are as follows:
[0122]
[0123] Specifically, in step S101, the step of acquiring bottle images with defect annotations to construct an image prototype library, where each bottle image corresponds to a pixel-level defect mask, includes:
[0124] Acquire multiple bottle images, and perform defect masking on each bottle image to obtain a bottle image with defect annotations;
[0125] For each bottle image, defect masking is performed, including pixel-level identification of defective areas in the bottle image as white areas and normal areas as black areas.
[0126] In this embodiment, a bottle defect prototype library is constructed offline in one go: four types of knowledge are distilled from masked historical defect data: "global defect prototype + real sample support memory + location-aware normal prototype + defect morphology prior", which serve as reusable detection priors.
[0127] Specifically, in step S102, the step of dividing and encoding each bottle image using a visual encoder to obtain the visual embedding vector of the image block includes:
[0128] The image of the bottle with the defect mask is divided into multiple image blocks;
[0129] Visual embedding vectors of image blocks are obtained by visual embedding encoding based on the size of the divided image blocks.
[0130] The position index of the image block is consistent with the position of the bottle image in the visual embedding encoding.
[0131] Specifically, in the offline construction of the bottle defect prototype library in Phase 1, the prototype library is built once using bottle images with defect annotations before training / deployment. After construction, it remains unchanged and is used for read-only purposes during online inference.
[0132] enter: Zhang Pingti Image Each image includes a pixel-level defect mask. (White represents defective areas, and black represents normal areas).
[0133] Image of each bottle Feeding the frozen industrial VLM Vision Encoder (or alternatively, a Vision Transformer ViT or similar encoder) yields patch-level embeddings:
[0134]
[0135] in , The patch size (e.g., 14×14 or 16×16 pixels). The position index of all patches. Maintain consistency throughout the system.
[0136] Specifically, in step S103, the step of projecting the defect mask image of the bottle image in the visual embedding vector onto the image patch grid and calculating the defect occupancy ratio of each image patch includes:
[0137] The defect occupancy ratio of each image block is determined by the ratio of the defect pixels in the defect mask of each image block on the image block grid to the total pixels of each image.
[0138] Image block defects are determined based on the defect occupancy ratio of each image block.
[0139] Specifically, the image block defect determination based on the defect occupancy ratio of each image block includes:
[0140] A defect ratio threshold is set. When the defect occupancy ratio of each image block is greater than the defect ratio threshold, the defect occupancy ratio of each image block is determined as the defect-related region.
[0141] When the defect occupancy ratio of each image block is less than or equal to the defect ratio threshold, the defect occupancy ratio of each image block is determined to be a normal area.
[0142] Specifically, the defect mask Project onto the same patch grid and compute each patch. Defect occupancy rate:
[0143]
[0144] in Represents a mask In Patch Number of defective pixels within, Patch The total number of pixels.
[0145] Judgment rules:
[0146]
[0147] in This is the defect ratio threshold (typically 0.3~0.5), which controls the strictness of the judgment.
[0148] Specifically, in step S104, capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space includes:
[0149] Average pooling is performed on the image patch embedding vectors containing defective regions in each bottle image;
[0150] The locations of image blocks in the bottle image where the proportion of defects exceeds the defect ratio threshold are marked as a location set;
[0151] The image patch embedding vectors containing defective regions are summed element by element to obtain the sum of the defect embedding vectors;
[0152] The mean vector of the overall defect appearance of the bottle image is obtained by dividing the sum of the defect embedding vectors by the modulus of the position vector.
[0153] For each defective image Perform average pooling on the embeddings of all defect-related patches:
[0154]
[0155] in It is an image The set of locations of all defect patches.
[0156] Explanation of principle: This section performs average pooling of image-level defect features—first, the image is processed... All "defect percentages exceeding the defect ratio threshold" The locations of the "Patch" are collected into a set. , This involves counting the number of elements in the set (i.e., the number of defect patches in the image), then summing the embedding vectors of these patches element by element, and finally dividing by the sum. This yields a mean vector representing the overall defect appearance of the image. The core purpose of this approach is to eliminate the bias caused by defect size. If each defect patch is directly sent to K-Means clustering, a large crack with 100 defect patches in an image will contribute 100 samples, while a tiny bubble with only 3 defect patches in an image will only contribute 3 samples. The cluster centers will be "dominated" by large defects, drowning out the patterns of fine defects. However, after... After normalization, each image generates only one cluster sample regardless of the defect size, ensuring that defect types of different scales and rarity enjoy equal representation in the prototype space.
[0157] The step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes:
[0158] The mean vector of the overall defect appearance of each bottle image is clustered using the K-Means algorithm to obtain the defect prototype of the defect visual pattern.
[0159] In the K-Means algorithm, each mean vector in the clustering represents a typical defective visual pattern.
[0160] Specifically, K-Means are used to represent all image-level defects. Perform clustering:
[0161]
[0162] Each This represents a typical defective visual pattern. For hyperparameters (e.g., 32~64).
[0163] Specifically, in step S104, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space further includes:
[0164] Support memory for constructing a defect prototype of a defective visual pattern;
[0165] All original image patch embedding vectors are assigned to corresponding defect prototypes; for each defect prototype, the original defect vectors assigned to the corresponding prototype are collected.
[0166] The original defect vector stores the embeddings of real samples, which are used to provide defect reference directions during subsequent residual enhancement.
[0167] Specifically, all raw patch embeddings are assigned to the nearest prototype:
[0168]
[0169] For each prototype Collect all original defect patch embeddings assigned to this prototype:
[0170]
[0171] The data stores the embeddings of real samples (rather than cluster centers), which are used to provide specific defect reference directions for subsequent residual enhancement.
[0172] It should be noted that this step involves building a "real sample repository" for each defect prototype—obtained from the previous K-Means clustering. only An abstract "geometric center point" (the average appearance of each type of defect) is used, but averaging would erase the detailed texture of individual samples; therefore, the reverse is true, and the embeddings of all real defect patches are scanned again. ,pass (that is, in) Calculate Euclidean distance in a 3D vector space (Find the prototype center closest to the given patch) and assign each patch to the category it most closely resembles, ultimately obtaining... a set Each set contains real patch embeddings belonging to that type of defect (not just averaged centers). The reason for retaining real samples instead of just using centers is... This is because, during the online inference phase, the defect morphology-guided residual enhancement module requires a "specific, referable target direction" for residual enhancement. If the cluster center is used as a reference, all microcracks will be pushed towards the same "average crack" direction, losing details such as direction, length, and width. By retrieving the real defect sample that is closest to the current patch based on similarity and using it as a reference vector, the unique morphological diversity of each crack can be preserved, so that the VLM sees a "clearer but still real" defect signal, rather than an averaged and flattened template.
[0173] Specifically, in step S104, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space further includes:
[0174] Construct a position-aware prototype space for a normal bottle;
[0175] The image positions of each normal bottle image are superimposed in a series based on the index set of normal bottle images;
[0176] Clustering is performed on the position embedding vectors of image blocks in normal bottle images to obtain the normal bottle prototype space.
[0177] Specifically, for each patch location ( ):
[0178]
[0179] in This is a set of indices for all normal (defect-free) bottle images.
[0180] Specific steps: - Collect the locations from all normal bottle images Patch embedding at the location - Clustering is performed within this location (e.g., K-Means or Mini-Batch K-Means) to form multiple normal prototypes. - Each location Independent clustering allows for variations in the number and content of prototypes at different locations.
[0181] Why is location awareness necessary? - Bottle neck: Has a threaded texture; its normal appearance is relatively complex. - Bottle body: Smooth and transparent; its normal appearance is relatively uniform. - Bottle bottom: Thick-walled area; may have mold number markings. - If the same defect feature appears in different locations, its "abnormality" should be determined with reference to the normal baseline for that location.
[0182] Specifically, in step S104, the step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space further includes:
[0183] Construct a priori defects;
[0184] Based on the relative row and column offsets of the defects, a set of locations for all defective image blocks in the image is constructed using an indicator function;
[0185] The defect image is assigned to the defect prototype and normalized to obtain the defect prior of the defect prototype.
[0186] Specifically, for each defect prototype Defect morphology prior It is a relative offset mapping:
[0187]
[0188] in: - Relative offset (row offset, column offset) - : Indicator function, returns 1 if the condition is true, and 0 otherwise. :image The set of locations of all defect patches - Patch Assigned to the defect prototype - Normalization factor, making
[0189] Intuitive meaning: This indicates that when a certain patch belongs to the first... When a defect is present, its relative offset The probability that the patch at that location also belongs to the same type of defect.
[0190] Typical morphological pattern example: - Microcrack prototype: Extending in a certain direction It exhibits a high response (reflecting the linear continuity of the crack). - Bubble prototype: In the surrounding small area It exhibits a uniform high response (reflecting the clustering characteristics of bubbles). - Scratch prototype: High response along a specific trajectory direction. - Stain prototype: There is a moderate range of response in local areas.
[0191] Specifically, in step S105, the counterfactual reasoning-based defect image selector calculates a counterfactual score for the input bottle image and filters images and shape support neighborhoods that exceed a threshold score, including:
[0192] Similar prototypes are retrieved from the global defect prototype space using the cosine similarity function in the similarity function.
[0193] Calculate the similarity score of the similar prototypes, and determine the similarity between the image patches in the global defect prototype space and known defect patterns based on the similarity score.
[0194] Counterfactual defect localization is a defect patch selector based on counterfactual reasoning. For a new input bottle image, each patch is evaluated to determine if it is a minor defect region, and the subset of patches most worthy of enhancement is selected.
[0195] In this embodiment, the online stage uses dual-path counterfactual scoring: each patch is compared with both the "defect prototype" and the "normal prototype at this location" simultaneously, and counterfactual reasoning is used to determine whether "the patch resembles a defect and cannot be explained by a normal appearance", effectively suppressing false detections caused by normal structural variations such as threads and templates.
[0196] Specifically, for each patch embedding of the test image :
[0197] Step 1: From the global defect prototype space Retrieve the Top-M most similar prototypes:
[0198]
[0199] The similarity function typically uses cosine similarity:
[0200]
[0201] Step 2: Calculate the defect similarity score:
[0202]
[0203] The higher the score, the more similar the patch is to known defect patterns.
[0204] Specifically, in step S105, the counterfactual reasoning-based defect image selector calculates a counterfactual score for the input bottle image and filters images and shape support neighborhoods that exceed a threshold score, including:
[0205] The cosine similarity function in the similarity function is used to determine similar normal prototypes from the normal prototype space corresponding to the image patch location.
[0206] Calculate the consistency score of the normal prototype, and determine the appearance of the normal bottle at the image block location based on the consistency score.
[0207] Specifically, for each patch of the test image :
[0208] Step 1: From the location Corresponding normal prototype space Retrieve the Top-M most similar normal prototypes:
[0209]
[0210] Step 2: Calculate the normal consistency score:
[0211]
[0212] The higher the score, the more closely the patch matches the appearance of a normal bottle at that location.
[0213] Counterfactual score calculation:
[0214] Combining dual-path information for counterfactual reasoning.
[0215]
[0216] in: - For the Sigmoid function, map the output to - The temperature parameter (typically 0.1~0.5) controls the sharpness of the scoring curve.
[0217] Physical meaning: - When High (like a defect) and When low (unlike a normal bottle at this position) High confidence defect - when Low (unlike any known defect) or When the height (which can be explained by a normal bottle) is high The normal region – this enables counterfactual reasoning.
[0218] Specifically, in step S105, the counterfactual reasoning-based defect image selector calculates a counterfactual score for the input bottle image and filters images and shape support neighborhoods that exceed a threshold score, including:
[0219] Counterfactual reasoning is performed based on the defect similarity score of the bottle image and the consistency score of the normal prototype of the bottle image.
[0220] Scores in counterfactual reasoning are calibrated using defect morphology calibration.
[0221] The calibration of scores in counterfactual reasoning through defect morphology calibration includes:
[0222] In counterfactual reasoning, select high-confidence seeds with confidence values greater than the seed threshold, and determine the defect prototype of each high-confidence seed image block;
[0223] For each high-confidence seed, the defect morphology prior of its prototype is used to propagate the confidence to the neighborhood to diffuse the defect morphology.
[0224] Based on the calibrated comprehensive score, image blocks with final scores exceeding the threshold and their defect morphology support neighborhoods are selected.
[0225] Specifically, the initial counterfactual score at the patch level may be noisy due to weak signals from minute defects, requiring spatial calibration.
[0226] Step 1: Select a high-confidence seed:
[0227]
[0228] in Set a seed threshold (e.g., 0.7) and select an initial high-confidence patch as the propagation starting point.
[0229] Step 2: Determine the defect prototype to which each seed patch belongs:
[0230]
[0231] Step 3: Defect morphology diffusion:
[0232] For each seed patch Use its own prototype Defect morphology prior Propagate confidence to neighboring areas:
[0233]
[0234] in To obtain from seed Patch To the target patch The relative offset.
[0235] Step 4: Overall Score:
[0236]
[0237] That is, the calibrated score is the maximum of the original counterfactual score and the propagation score, which preserves the high-confidence independent response and fills the continuous area of the defect morphology.
[0238] Final selection:
[0239] Based on the calibrated overall score, select the patches whose final scores exceed the threshold and their defect morphology support neighborhoods:
[0240]
[0241] in The final screening threshold, Indicates prior knowledge based on defect morphology From seed set The extended coherent neighborhood.
[0242] The final output of the counterfactual defect localization module: - Selected Patch collection - The original embedding for each selected patch - Defect prototype index for each selected patch - The final score for each selected patch .
[0243] By using counterfactual defect localization and employing a counterfactual comparison mechanism of dual-path scoring (defect similarity vs. normal consistency), we can effectively distinguish between "true minor defects" and "normal but atypical texture variations," thus avoiding misjudging normal structural variations of the bottle body (such as bottle mouth thread texture and bottle bottom mold number) as defects.
[0244] Specifically, in step S106, defect morphology feature residual enhancement is performed on the image and morphology support neighborhood that exceed the threshold score, and subtle defects are captured through a visual language model, including:
[0245] Determine the defect prototype corresponding to each image patch that exceeds the threshold, and retrieve similar reference embeddings from the support memory model of the defect prototypes of the image patches that exceed the threshold.
[0246] For each selected image block location and for each image block within its defect morphology neighborhood, calculate the defect morphology enhancement weights;
[0247] Residual enhancement is performed on each image block that requires enhancement weights for defect morphology.
[0248] The residual enhancement of each image block requiring enhancement of defect morphology includes:
[0249] Determine the prototype support memory corresponding to the nearest seed image block of the defective image block; calculate the residual vector of the prototype support memory corresponding to the image block;
[0250] Residual enhancement is performed based on the original image patch embedding vector, global enhancement intensity coefficient, overall enhancement magnitude, and feature offset.
[0251] The enhanced and unenhanced image patch embedding vectors are recombined in their original positional order to form an enhanced encoding sequence, which is then fed into a frozen visual language model. The language decoder in the visual language model decodes the enhanced encoding sequence to capture subtle defects in the bottle image.
[0252] In this embodiment, defect morphology-guided residual enhancement is achieved by using reference embeddings retrieved from real samples to apply mild residual perturbations to the selected patch along the direction of "pointing to the same type of defect," and by using data-driven defect morphology priors to smoothly diffuse in space, thus amplifying the defect signal while preserving the normal anatomical structure of the bottle.
[0253] Specifically, the defect morphology-guided residual enhancement module is a feature residual enhancement unit based on the prior knowledge of defect morphology. It performs gentle yet effective feature enhancement on the defect-related patches selected by the counterfactual defect localization module, making it easier for the VLM to capture subtle defect signals.
[0254] Hot-swappable on production lines and zero-sample retraining: All modules can be seamlessly connected to any deployed industrial VLM or vision backbone network without backpropagation or parameter fine-tuning of the original model during inference. When faced with new bottle types or new processes, the defect prototype library can be rebuilt offline once and then put online without stopping production for annotation and retraining, significantly reducing the cost of production line switching and quality maintenance.
[0255] Among them, reference embeddings that support memory retrieval:
[0256] For each selected patch :
[0257] Step 1: Determine the defect prototype corresponding to this patch. .
[0258] Step Two: From the supporting memories of this prototype Search for the most similar reference embedding in the middle:
[0259]
[0260] That is, among real samples of the same type of defect, find the one most similar to the current patch as the enhancement reference direction.
[0261] Calculate the defect morphology enhancement weights:
[0262] For each selected patch Each patch within its defect morphology neighborhood :
[0263]
[0264] in From seed Patch To Neighborhood Patch The relative offset, It is the prior to the defect morphology corresponding to the defect prototype.
[0265] For patches that are not covered by any seeds (No enhancement).
[0266] Ultimately, each patch The enhancement weight is the maximum value of all seed contributions:
[0267]
[0268] Residual enhancement:
[0269] For each patch that needs enhancement (Right now ):
[0270] Step 1: Determine the best reference source for this patch (from the prototype support memory of the most recent seed patch):
[0271]
[0272] Step 2: Calculate the residual vector:
[0273]
[0274] The residual vector points in the direction of "moving from the current patch to a real sample with the same defect".
[0275] Step 3: Perform residual enhancement:
[0276]
[0277] in: - Original Patch embedding (preserving basis information) - Global enhancement intensity coefficient (hyperparameter, typical value 0.1~0.5), controls the overall enhancement magnitude. Defect morphology weights control the enhancement level of this patch. Feature offset vector towards the defect direction
[0278] Key design considerations: - Residual form: Instead of directly replacing it with This preserves most of the normal information of the original bottle structure and avoids introducing artifacts. - Defect morphology weight: Ensure that enhancements transition smoothly over spatially coherent regions, rather than isolating and enhancing individual noise points. - Support memory: Use real samples instead of the prototype center as the reference direction to preserve the detail and diversity of defects.
[0279] Handling unselected patches:
[0280] For normal patches that are not selected ( ), remain unchanged:
[0281]
[0282] Enhanced sequence output:
[0283] Reassemble all enhanced and unenhanced patch embeddings in their original positional order:
[0284]
[0285] This sequence is used as an enhanced visual token sequence and fed into the frozen VLM language decoder (LLM) for downstream tasks.
[0286] Defect morphology guides residual enhancement, using data-driven defect morphology priors (rather than fixed Gaussian kernels) for spatial propagation. Residual enhancement is performed instead of replacement, preserving normal anatomical information. Different types of defects (cracks / bubbles / scratches) have their own exclusive defect morphology priors.
[0287] Based on the same inventive concept, a second aspect of this application provides a device for detecting minute defects in a bottle body, see [link to previous document]. Figure 2 ,include:
[0288] Image prototype construction unit 201 is used to acquire bottle images with defect annotations to construct an image prototype library. Each bottle image corresponds to a pixel-level defect mask.
[0289] Embedding vector encoding unit 202 is used to divide and encode each bottle image by a visual encoder to obtain the visual embedding vector of the image block.
[0290] The defect occupancy calculation unit 203 is used to project the defect mask image of the bottle image in the visual embedding vector onto the image block grid and calculate the defect occupancy ratio of each image block.
[0291] Global defect construction unit 204 is used to capture the visual pattern of repeated defects on the bottle according to the defect occupancy ratio and construct a global defect-related prototype space.
[0292] The counterfactual calculation unit 205 is used to perform counterfactual score calculation on the input bottle image by the defective image selector based on counterfactual reasoning, and to filter images and shape support neighborhoods that exceed the threshold score.
[0293] The shape residual enhancement unit 206 is used to perform defect shape feature residual enhancement on the image and shape support neighborhood that exceed the threshold score, and capture subtle defects through a visual language model.
[0294] Furthermore, the image prototype construction unit 201 is also used to acquire multiple bottle images and perform defect masking on each bottle image to obtain a bottle image with defect annotations.
[0295] For each bottle image, defect masking is performed, including pixel-level identification of defective areas in the bottle image as white areas and normal areas as black areas.
[0296] Furthermore, the embedding vector coding unit 202 is also used to divide the bottle image with defect mask into multiple image blocks;
[0297] Visual embedding vectors of image blocks are obtained by visual embedding encoding based on the size of the divided image blocks.
[0298] The position index of the image block is consistent with the position of the bottle image in the visual embedding encoding.
[0299] Furthermore, the defect occupancy calculation unit 203 is also used to determine the defect occupancy ratio of each image block based on the ratio of the defect pixels of the defect mask of each image block on the image block grid to the total pixels of each image.
[0300] Image block defects are determined based on the defect occupancy ratio of each image block.
[0301] Furthermore, the global defect construction unit 204 is also used to perform average pooling on the image block embedding vector containing the defect region in each bottle image;
[0302] The locations of image blocks in the bottle image where the proportion of defects exceeds the defect ratio threshold are marked as a location set;
[0303] The image patch embedding vectors containing defective regions are summed element by element to obtain the sum of the defect embedding vectors;
[0304] The mean vector of the overall defect appearance of the bottle image is obtained by dividing the sum of the defect embedding vectors by the modulus of the position vector.
[0305] Furthermore, the global defect construction unit 204 is also used to cluster the mean vector of the overall defect appearance of each bottle image using the K-Means algorithm to obtain the defect prototype of the defect visual pattern.
[0306] In the K-Means algorithm, each mean vector in the clustering represents a typical defective visual pattern.
[0307] Furthermore, the counterfactual calculation unit 205 is also used to retrieve similar prototypes from the global defect prototype space using the cosine similarity function in the similarity function;
[0308] Calculate the similarity score of the similar prototypes, and determine the similarity between the image patches in the global defect prototype space and known defect patterns based on the similarity score.
[0309] Furthermore, the counterfactual calculation unit 205 is also used to determine similar normal prototypes from the normal prototype space corresponding to the image patch position through the cosine similarity function in the similarity function;
[0310] Calculate the consistency score of the normal prototype, and determine the appearance of the normal bottle at the image block location based on the consistency score.
[0311] Furthermore, the topography residual enhancement unit 206 is also used to determine the defect prototype corresponding to each image block that exceeds the threshold, and to retrieve similar reference embeddings from the support memory model of the defect prototypes of the image blocks that exceed the threshold.
[0312] For each selected image block location and for each image block within its defect morphology neighborhood, calculate the defect morphology enhancement weights;
[0313] Residual enhancement is performed on each image block that requires enhancement weights for defect morphology.
[0314] Furthermore, the morphology residual enhancement unit 206 is also used to determine the prototype support memory corresponding to the nearest seed image block of the defective image block; and to calculate the residual vector of the prototype support memory corresponding to the image block.
[0315] Residual enhancement is performed based on the original image patch embedding vector, global enhancement intensity coefficient, overall enhancement magnitude, and feature offset.
[0316] The enhanced and unenhanced image patch embedding vectors are recombined in their original positional order to form an enhanced encoding sequence, which is then fed into a frozen visual language model. The language decoder in the visual language model decodes the enhanced encoding sequence to capture subtle defects in the bottle image.
[0317] Based on the same inventive concept, a third aspect of this disclosure also provides an electronic device 161, see [link to relevant documentation]. Figure 3 It includes a processor 164, a communication interface 165, a memory 162, and a communication bus, wherein the processor 164, the communication interface 165, and the memory 162 communicate with each other through the communication bus;
[0318] Memory 162 stores computer program 163;
[0319] When the processor 164 executes the program stored in the memory 162, it implements the method for detecting minute defects in the bottle body.
[0320] The aforementioned communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.
[0321] The communication interface 165 is used for communication between the aforementioned electronic device 161 and other devices.
[0322] The memory 162 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 162 may also be at least one storage device located remotely from the aforementioned processor 164.
[0323] The processor 164 mentioned above can be a general-purpose processor 164, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0324] Based on the same inventive concept, the fourth aspect of this disclosure also provides a computer-readable storage medium storing a computer program 163, which, when executed by a processor 164, implements the method for detecting minute defects in a bottle.
[0325] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist independently and not assembled into the device / apparatus. The computer-readable storage medium carries one or more programs that, when executed, implement the bottle micro-defect detection method according to embodiments of this disclosure.
[0326] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for detecting minute defects in a bottle, characterized in that, include: We acquire bottle images with defect annotations to build an image prototype library. Each bottle image has a pixel-level defect mask. Each bottle image is divided and encoded using a visual encoder to obtain the visual embedding vector of the image block. Project the defect mask image of the bottle image in the visual embedding vector onto the image block grid, and calculate the defect occupancy ratio of each image block; Based on the defect occupancy ratio, capture the visual patterns of recurring defects on the bottle and construct a global defect-related prototype space. The defect image selector based on counterfactual reasoning calculates counterfactual scores for input bottle images and filters images with scores exceeding a threshold and their shape support neighborhoods. For images and morphological support neighborhoods that exceed the threshold score, defect morphological feature residual enhancement is performed to capture subtle defects through a visual language model.
2. The method according to claim 1, characterized in that, The process of acquiring bottle images with defect annotations to construct an image prototype library, where each bottle image corresponds to a pixel-level defect mask, includes: Acquire multiple bottle images, and perform defect masking on each bottle image to obtain a bottle image with defect annotations; For each bottle image, defect masking is performed, including pixel-level identification of defective areas in the bottle image as white areas and normal areas as black areas.
3. The method according to claim 1 or 2, characterized in that, The step of dividing and encoding each bottle image using a visual encoder to obtain the visual embedding vector of the image patch includes: The image of the bottle with the defect mask is divided into multiple image blocks; Visual embedding vectors of image blocks are obtained by visual embedding encoding based on the size of the divided image blocks. The position index of the image block is consistent with the position of the bottle image in the visual embedding encoding.
4. The method according to claim 3, characterized in that, The step of projecting the defect mask image of the bottle image in the visual embedding vector onto the image patch grid and calculating the defect occupancy ratio of each image patch includes: The defect occupancy ratio of each image block is determined by the ratio of the defect pixels in the defect mask of each image block on the image block grid to the total pixels of each image. Image block defects are determined based on the defect occupancy ratio of each image block.
5. The method according to claim 4, characterized in that, The process of determining image block defects based on the defect occupancy ratio of each image block includes: A defect ratio threshold is set. When the defect occupancy ratio of each image block is greater than the defect ratio threshold, the defect occupancy ratio of each image block is determined as the defect-related region. When the defect occupancy ratio of each image block is less than or equal to the defect ratio threshold, the defect occupancy ratio of each image block is determined to be a normal area.
6. The method according to claim 1, characterized in that, The step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space includes: Average pooling is performed on the image patch embedding vectors containing defective regions in each bottle image; The locations of image blocks in the bottle image where the proportion of defects exceeds the defect ratio threshold are marked as a location set; The sum of the defect embedding vectors is obtained by adding the image patch embedding vectors containing the defective region element by element; The mean vector of the overall defect appearance of the bottle image is obtained by dividing the sum of the defect embedding vectors by the modulus of the position vector.
7. The method according to claim 6, characterized in that, The step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes: The mean vector of the overall defect appearance of each bottle image is clustered using the K-Means algorithm to obtain the defect prototype of the defect visual pattern. In the K-Means algorithm, each mean vector in the clustering represents a typical defective visual pattern.
8. The method according to claim 7, characterized in that, The step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes: Support memory for constructing a defect prototype of a defective visual pattern; All original image patch embedding vectors are assigned to corresponding defect prototypes; for each defect prototype, the original defect vectors assigned to the corresponding prototype are collected. The original defect vector stores the embeddings of real samples, which are used to provide defect reference directions during subsequent residual enhancement.
9. The method according to claim 8, characterized in that, The step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes: Construct a position-aware prototype space for a normal bottle; The image positions of each normal bottle image are superimposed in a series based on the index set of normal bottle images; Clustering is performed on the position embedding vectors of image blocks in normal bottle images to obtain the normal bottle prototype space.
10. The method according to claim 9, characterized in that, The step of capturing recurring visual patterns of defects on the bottle based on the defect occupancy ratio and constructing a global defect-related prototype space also includes: Construct a priori defects; Based on the relative row and column offsets of the defects, a set of locations for all defective image blocks in the image is constructed using an indicator function; The defect image is assigned to the defect prototype and normalized to obtain the defect prior of the defect prototype.
11. The method according to claim 1, characterized in that, The counterfactual reasoning-based defect image selector calculates counterfactual scores for the input bottle images and filters images and shape support neighborhoods that exceed a threshold score, including: Similar prototypes are retrieved from the global defect prototype space using the cosine similarity function in the similarity function. Calculate the similarity score of the similar prototypes, and determine the similarity between the image patches in the global defect prototype space and known defect patterns based on the similarity score.
12. The method according to claim 11, characterized in that, The counterfactual reasoning-based defect image selector calculates counterfactual scores for the input bottle images and filters images and shape support neighborhoods that exceed a threshold score, including: The cosine similarity function in the similarity function is used to determine similar normal prototypes from the normal prototype space corresponding to the image patch location. Calculate the consistency score of the normal prototype, and determine the appearance of the normal bottle at the image block location based on the consistency score.
13. The method according to claim 12, characterized in that, The counterfactual reasoning-based defect image selector calculates counterfactual scores for the input bottle images and filters images and shape support neighborhoods that exceed a threshold score, including: Counterfactual reasoning is performed based on the defect similarity score of the bottle image and the consistency score of the normal prototype of the bottle image. Scores in counterfactual reasoning are calibrated using defect morphology calibration.
14. The method according to claim 13, characterized in that, The calibration of scores in counterfactual reasoning through defect morphology calibration includes: In counterfactual reasoning, select high-confidence seeds with confidence values greater than the seed threshold, and determine the defect prototype of each high-confidence seed image block; For each high-confidence seed, the defect morphology prior of its prototype is used to propagate the confidence to the neighborhood to diffuse the defect morphology. Based on the calibrated comprehensive score, image blocks with final scores exceeding the threshold and their defect morphology support neighborhoods are selected.
15. The method according to claim 1, characterized in that, For images with scores exceeding the threshold and their shape support neighborhoods, defect shape feature residual enhancement is performed to capture subtle defects using a visual language model, including: Determine the defect prototype corresponding to each image patch that exceeds the threshold, and retrieve similar reference embeddings from the support memory model of the defect prototypes of the image patches that exceed the threshold. For each selected image block location and for each image block within its defect morphology neighborhood, calculate the defect morphology enhancement weights; Residual enhancement is performed on each image block that requires enhancement weights for defect morphology.
16. The method according to claim 15, characterized in that, The residual enhancement of each image block requiring enhancement of defect morphology includes: Determine the prototype support memory corresponding to the nearest seed image block of the defective image block; calculate the residual vector of the prototype support memory corresponding to the image block; Residual enhancement is performed based on the original image patch embedding vector, global enhancement intensity coefficient, overall enhancement magnitude, and feature offset. The enhanced and unenhanced image patch embedding vectors are recombined in their original positional order to form an enhanced encoding sequence, which is then fed into a frozen visual language model. The language decoder in the visual language model decodes the enhanced encoding sequence to capture subtle defects in the bottle image.
17. A device for detecting minute defects in a bottle, characterized in that, include: The image prototype construction unit is used to acquire bottle images with defect annotations to build an image prototype library. Each bottle image has a pixel-level defect mask. The embedding vector encoding unit is used to divide and encode each bottle image by a visual encoder to obtain the visual embedding vector of the image block. The defect occupancy calculation unit is used to project the defect mask image of the bottle image in the visual embedding vector onto the image block grid and calculate the defect occupancy ratio of each image block. The global defect construction unit is used to capture the visual patterns of repeated defects on the bottle according to the proportion of defects occupied, and to construct a global defect-related prototype space. The counterfactual calculation unit is used to calculate the counterfactual score of the input bottle image by the defective image selector based on counterfactual reasoning, and to filter images and shape support neighborhoods that exceed the threshold score. The shape residual enhancement unit is used to perform defect shape feature residual enhancement on the image and shape support neighborhood that exceed the threshold score, and to capture subtle defects through a visual language model.
18. The apparatus according to claim 17, characterized in that, The image prototype construction unit is also used to acquire multiple bottle images and perform defect masking on each bottle image to obtain a bottle image with defect annotations. For each bottle image, defect masking is performed, including pixel-level identification of defective areas in the bottle image as white areas and normal areas as black areas.
19. The apparatus according to claim 17, characterized in that, The embedded vector coding unit is also used to divide the bottle image with defect mask into multiple image blocks; Visual embedding vectors of image blocks are obtained by visual embedding encoding based on the size of the divided image blocks. The position index of the image block is consistent with the position of the bottle image in the visual embedding encoding.
20. The apparatus according to claim 17, characterized in that, The defect occupancy calculation unit is also used to determine the defect occupancy ratio of each image block based on the ratio of the defect pixels of the defect mask of each image block on the image block grid to the total pixels of each image. Image block defects are determined based on the defect occupancy ratio of each image block.
21. The apparatus according to claim 17, characterized in that, The global defect construction unit is also used to perform average pooling on the image block embedding vector containing the defect region in each bottle image; The locations of image blocks in the bottle image where the proportion of defects exceeds the defect ratio threshold are marked as a location set; The sum of the defect embedding vectors is obtained by adding the image patch embedding vectors containing the defective region element by element; The mean vector of the overall defect appearance of the bottle image is obtained by dividing the sum of the defect embedding vectors by the modulus of the position vector.
22. The apparatus according to claim 21, characterized in that, The global defect construction unit is also used to cluster the mean vector of the overall defect appearance of each bottle image using the K-Means algorithm to obtain the defect prototype of the defect visual pattern. In the K-Means algorithm, each mean vector in the clustering represents a typical defective visual pattern.
23. The apparatus according to claim 17, characterized in that, The counterfactual calculation unit is also used to retrieve similar prototypes from the global defect prototype space using the cosine similarity function in the similarity function; Calculate the similarity score of the similar prototypes, and determine the similarity between the image patches in the global defect prototype space and known defect patterns based on the similarity score.
24. The apparatus according to claim 23, characterized in that, The counterfactual calculation unit is also used to determine similar normal prototypes from the normal prototype space corresponding to the image patch position through the cosine similarity function in the similarity function; Calculate the consistency score of the normal prototype, and determine the appearance of the normal bottle at the image block location based on the consistency score.
25. The apparatus according to claim 17, characterized in that, The topography residual enhancement unit is also used to determine the defect prototype corresponding to each image patch that exceeds the threshold, and to retrieve similar reference embeddings from the support memory model of the defect prototypes of the image patches that exceed the threshold. For each selected image block location and for each image block within its defect morphology neighborhood, calculate the defect morphology enhancement weights; Residual enhancement is performed on each image block that requires enhancement weights for defect morphology.
26. The apparatus according to claim 25, characterized in that, The morphology residual enhancement unit is also used to determine the prototype support memory corresponding to the nearest seed image block of the defective image block; and to calculate the residual vector of the prototype support memory corresponding to the image block. Residual enhancement is performed based on the original image patch embedding vector, global enhancement intensity coefficient, overall enhancement magnitude, and feature offset. The enhanced and unenhanced image patch embedding vectors are recombined in their original positional order to form an enhanced encoding sequence, which is then fed into a frozen visual language model. The language decoder in the visual language model decodes the enhanced encoding sequence to capture subtle defects in the bottle image.
27. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. Memory, which stores computer programs; When the processor executes the program stored in the memory, it implements the method for detecting minute defects in a bottle as described in any one of claims 1 to 16.
28. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the method for detecting minute defects in a bottle as described in any one of claims 1 to 16.
