Tire appearance defect detection method and system based on retrieval enhancement generation technology

By adopting a tire appearance defect detection method based on retrieval enhancement generation technology, combined with edge computing and multimodal large model, the problems of inconsistent judgment standards and insufficient utilization of historical data in tire appearance inspection are solved, and efficient and accurate defect detection and quality control are achieved.

CN121860955APending Publication Date: 2026-04-14TRIANGLE TIRE
View PDF 0 Cites 1 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-22
Publication Date
2026-04-14

AI Technical Summary

Technical Problem

Existing tire appearance inspection methods suffer from inconsistent judgment standards, susceptibility to human interference, poor adaptability, low training efficiency of traditional models, difficulty in real-time updating of defect judgment standards, and insufficient utilization of historical data.

Method used

A tire appearance defect detection method based on retrieval enhancement generation technology is adopted. Image preprocessing is performed through edge computing, image vectorization and retrieval are performed using the retrieval enhancement generation module, defect judgment is performed by combining multimodal large model, and dynamic knowledge injection is achieved through historical defect vector library and multimodal large model to support human-machine collaborative judgment.

Benefits of technology

It enables accurate detection of tire defect types and severity, improves detection efficiency and accuracy, supports real-time updates of defect judgment standards, and has efficient quality control capabilities and traceability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121860955A_ABST
    Figure CN121860955A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of tire manufacturing and detection, in particular to a tire appearance defect detection method and system based on a retrieval enhancement generation technology, and the method comprises the steps: collecting image data, and completing the preprocessing of the image data through an edge calculation end; the retrieval enhancement generation module is used for sequentially carrying out picture vectorization processing on the preprocessed data, carrying out vectorization retrieval in a historical defect vector library and then sending the data into a multi-modal large model through the Ethernet to complete data fusion and an image defect judgment result; and the retrieval enhancement generation module outputs a judgment result to the production management end and the edge calculation end, and the production management end archives the judgment result. Compared with the prior art, the detection automation and intelligence are realized, the detection accuracy and efficiency and the production line quality control capability are also remarkably improved, and the production efficiency is improved. The method is suitable for defect identification and quality evaluation tasks of tire appearance detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention patent relates to the field of tire manufacturing and testing technology, specifically a tire appearance defect detection method and system based on search-enhanced generation technology that can accurately detect the type and severity of tire defects. Background technology: Currently, tire appearance inspection mainly relies on manual visual inspection or traditional image recognition algorithms. These methods are typically used to detect defects on the tire surface, and the results are judged manually or analyzed based on fixed algorithms. These technologies have been widely applied in the tire manufacturing industry and, to some extent, meet the needs of tire appearance inspection. However, traditional tire appearance inspection methods suffer from inconsistent judgment standards and are easily influenced by human experience. Especially in mass production, these problems severely restrict quality control and production line pace. Furthermore, traditional inspection methods have poor adaptability and cannot quickly meet changes in defect judgment standards. In particular, traditional artificial intelligence algorithms rely on model training, making it difficult to update judgment standards in real time for new defect types. Additionally, traditional inspection generally lacks the ability to intelligently utilize historical inspection data, hindering the development of intelligent inspection technology. Summary of the Invention

[0002] This invention addresses the problems of low detection efficiency, poor accuracy, insufficient utilization of historical data, and low training efficiency of traditional models in existing detection technologies. It proposes a tire appearance defect detection method and system based on retrieval enhancement generation technology that can accurately detect the type and severity of tire defects.

[0003] This invention achieves its purpose through the following measures: A tire appearance defect detection method based on retrieval enhancement generation technology, characterized by the following steps: Step 1: Acquire image data and preprocess the image data using edge computing. Step 2: The retrieval enhancement generation module sequentially performs image vectorization processing on the preprocessed data, performs vectorization retrieval in the historical defect vector database, and then sends it to the multimodal large model via Ethernet to complete data fusion and image defect judgment results. Step 3: The retrieval enhancement generation module outputs the judgment results to the production management end and the edge computing end. The production management end archives the judgment results. Specifically, when the judgment result is a new defect, the image data corresponding to the new defect is vectorized and the vectorized data is stored in the historical defect vector library.

[0004] In step 1 of this invention, the preprocessing of image data specifically includes the following steps: Step 1-1: Grayscale Conversion. Grayscale conversion converts the color images captured by the industrial camera into single-channel grayscale images. Since the tire surface is mainly composed of black rubber, product defects such as cracks, cuts, and bulges show almost no significant color difference under visible light. However, the redundant information in the RGB channels of the color image can introduce interference factors such as light color temperature fluctuations and camera white balance deviations, reducing the stability of subsequent texture analysis. Therefore, a weighted average model that conforms to the characteristics of human visual perception is adopted. Gray=0.299R+0.587G+0.114B (1), This weighting method fully considers the physiological optical characteristics of the green channel contributing the most to brightness, followed by red, and then blue. While preserving the light and dark structure information of the tire surface, it effectively suppresses color shifts caused by workshop lighting sources (such as LED cool white light or sodium lamp warm light). Compared with the simple three-channel mean method (Gray = (R+G+B) / 3), this method can more accurately reflect the true surface reflectivity distribution on the tire image, providing a more stable brightness benchmark for subsequent CLAHE enhancement, Gabor texture extraction and other steps, and avoiding false edges or texture distortion caused by color noise. Step 1-2: Adaptive Histogram Equalization (CLAHE): Divide the grayscale image obtained in the previous step into multiple small regions, perform histogram equalization on each region to enhance local contrast, prevent excessive noise amplification by setting a contrast limit threshold clipLimit, and use interpolation to smooth adjacent regions to improve the clarity of tire texture details. Steps 1-3: Bilateral filtering denoising: While smoothing image noise, edge features are preserved. The bilateral filtering denoising algorithm comprehensively considers the spatial distance between pixels and gray-level similarity to effectively remove imaging noise and dust interference. Steps 1-4: Multi-scale Retinex Illumination Compensation (MSRCR): Adaptively corrects image brightness, eliminates shadows and reflections caused by uneven illumination, and restores the true brightness and detail of the image through multi-scale Gaussian filtering and logarithmic transformation, so that the tire surface features remain consistent under different lighting conditions. Steps 1-5: Gamma correction, which performs non-linear adjustment to the image brightness. The γ value is set to 1.2-1.5 and fine-tuned according to the actual effect to enhance the details in the shadows and prevent overexposure in the highlights. Steps 1-6: Gabor Filter Texture Enhancement: Multi-directional Gabor filters are used to extract tire surface texture information. By setting different filter kernels (such as 0°, 45°, 90°, 135°) to convolve the image, the pattern structure and wear features are highlighted, and the sensitivity of subsequent detection to subtle texture changes is improved.

[0005] The image preprocessing workflow described in this invention is a collaborative optimization scheme designed to address the challenges of industrial inspection, such as high reflectivity of tire surfaces, complex tread patterns, and the ease with which minor defects are masked by noise. Specifically, the combined use of CLAHE and MSRCR effectively overcomes localized overexposure or underexposure caused by uneven lighting in the workshop; the sequential combination of Gabor filtering and bilateral filtering enhances the directional texture of the tread pattern while preventing noise from being misjudged as defect edges; the Gamma correction parameter γ is limited to the range of 1.2–1.5, preserving details in the dark areas of the tire sidewall while preventing the loss of information in the bright areas of the tire crown. This combined strategy significantly improves the robustness of subsequent vectorization and defect identification. After preprocessing, the image data acquired in this invention produces output images with the following characteristics: uniform illumination and moderate contrast; significantly reduced noise and clearly discernible details; prominent edge features such as cracks and cuts; and effective removal of background interference.

[0006] In step 2 of this invention, the retrieval enhancement generation module performs image vectorization processing on the preprocessed data. Specifically, it uses the CLIP-ViT-B / 16 model to encode the preprocessed images. This model can be adapted and trained on a domain fine-tuning dataset consisting of a large number of tire defect samples, enabling its visual encoder to have stronger semantic perception and feature discrimination capabilities for tire-specific geometric structures, periodic patterns, highly reflective surfaces, and minute defect morphologies (such as fine cracks and shallow cuts). Theoretically, compared to directly using a general visual model that has not been adapted to the domain, the fine-tuning strategy adopted in this invention can significantly improve the discriminativeness of the generated vectors in tire defect similarity measurement, effectively alleviating the feature confusion problem caused by the complex texture of the tire surface and uneven lighting. After processing by this model, feature vectors are output, and then L2 normalization is performed on the feature vectors. PCA / FAISS PQ is used to compress the image vectors to reduce transmission and storage costs.

[0007] In step 2 of this invention, vectorized retrieval is performed in the historical defect vector database. Specifically, a historical defect vector database is constructed using Milvus. This database adopts a hierarchical index structure (HNSW+IVF_PQ) and, considering the characteristics of large intra-class differences and high inter-class similarity of tire defects, a sub-database partitioning mechanism based on defect type is introduced. This allows for priority neighbor search within the same type of defect sub-database during retrieval, significantly improving retrieval accuracy and efficiency. Each record in the vector database includes: image ID, vector features, defect type (crack, bulge), defect description (text description), and acquisition time information. Vectorized retrieval refers to performing a Top-K similarity search on the current image vector in Milvus and returning the results, which include: Top-K sample IDs, vector distance (similarity), and defect identification information.

[0008] In step 2 of this invention, the data returned by vectorized retrieval is sent to the multimodal large model via Ethernet. The model then completes the final defect judgment. Specifically, the system constructs a structured prompt by combining the current preprocessed tire image with Top-K similar samples (including corresponding images, defect types, text descriptions, and confidence information) retrieved from the historical defect vector library. This prompt serves as the input context for the multimodal large model. The multimodal large model includes, but is not limited to, BLIP-2, LLaVA, or a fusion architecture based on CLIP and Transformer. Upon receiving the enhanced prompt, the model performs visual understanding of the current image and performs joint reasoning by combining the retrieved historical defect semantic information. This results in a comprehensive assessment of the defect type (such as cracks, bulges, and cuts), location information, and natural language judgment criteria, which is then submitted for manual review.

[0009] Compared to traditional single-modal detection models (such as CNN classifiers) or multimodal models that do not incorporate external knowledge, this invention achieves dynamic knowledge injection through the RAG mechanism: the model does not need to cover all defect types during the training phase, but instead uses the most relevant historical cases as the basis for judgment in real time during inference. This not only avoids the problem of traditional models becoming completely ineffective for new defects due to limitations in training data, but also significantly improves the transparency and traceability of the judgment logic.

[0010] In particular, in tire appearance inspection scenarios, due to the characteristics of the tire tread, such as strong periodic patterns, highly reflective areas, and the fact that minor defects can be easily masked by normal textures, it is difficult to accurately distinguish between "manufacturing features" and "real defects" relying solely on the image itself. This invention provides analogical references for the model by retrieving highly relevant historical samples, enabling the multimodal large model to focus on key difference features under guidance, effectively suppressing misjudgments. When the current image has low similarity to samples in the historical database, the system can mark it as a potential new defect and still generate a preliminary judgment description based on limited similar samples, supporting subsequent manual confirmation and knowledge feedback, thus achieving continuous evolution of defect judgment standards.

[0011] The judgment logic of the multimodal large model in step 2 of this invention is as follows: (1) If the similarity with the Top-1 sample is ≥ 0.9, the defect type is automatically determined and the multimodal large model is called to generate judgment information; (2) If the similarity with the Top-1 sample is 0.7 ≤ and the similarity with the Top-1 sample is < 0.9, the multimodal large model is called to review and generate judgment information; (3) If the similarity with the Top-1 sample is < 0.7, it is marked as a new defect and the multimodal large model is called to generate judgment information.

[0012] In this invention, the determination result is returned to the edge terminal via Ethernet, and the determination result is also returned to the edge computing end via Ethernet. The edge computing end triggers the corresponding local response mechanism based on the received defect type and severity information, including but not limited to: highlighting the defect location on the local human-machine interface, triggering an audible and visual alarm, or sending a stop signal to the production line PLC to isolate suspected non-conforming products.

[0013] This invention also proposes a tire appearance defect detection system based on retrieval enhancement generation technology as described above. The system is characterized by comprising a data acquisition module, an edge computing terminal for preprocessing acquired data, a retrieval enhancement generation module deployed on a server, and a production management terminal. The data acquisition module is implemented using an industrial camera, and its output is connected to the edge computing terminal, which is equipped with an image preprocessing module. Both the edge computing terminal and the production management terminal establish data communication with the server.

[0014] This invention employs a high-precision industrial camera to capture high-resolution images of the tire surface, ensuring image quality meets subsequent analysis requirements. The images are then transmitted to an edge computing system, which performs preprocessing, including image enhancement, noise reduction, and brightness adjustment, to improve image clarity and detail retention. These processes ensure that subsequent analysis can accurately capture even minute defects on the tire surface. A server-based retrieval enhancement and generation module includes an image vectorization module, a vectorized retrieval module, an Ethernet module, and a multimodal large model. After image preprocessing, the image vectorization module vectorizes the tire images using a vector model. This model converts images into vectors, which are used for subsequent similarity retrieval and defect determination. The vectorized retrieval module constructs a historical defect sample database based on historical tire appearance defect data. This database stores vectorized result sets of various tire defect images and corresponding defect information. During each detection, the vector of the current image is matched with historical defect vectors in the database, quickly identifying the possible defect types and information in the current image through similarity retrieval. Meanwhile, the vector database can perform real-time vectorization processing and storage of new types of defect images and defect information, enabling rapid and dynamic updates of appearance judgment standards; retrieved historical defect samples and defect information are input to the multimodal large model inference module via Ethernet, and the multimodal large model generates the final detection result based on the input information.

[0015] All detection data in this invention will be transmitted in real time to the enterprise production management terminal for storage and management. The production management system archives and statistically analyzes the detection results. Furthermore, the production management terminal is also connected to the operator's terminal and the manager's terminal. The information of the production management system is displayed to the operator through a graphical interface to help the operator quickly determine the type of defect and the handling method. As needed, the production management system can push the detection results to the manager in a timely manner to facilitate quality control, defect tracing and production plan optimization.

[0016] Compared with the prior art, the present invention has the following advantages: (1) Intelligent recognition: The vector knowledge base built based on historical data can be continuously expanded, and the expansion method only requires vectorization of the push and press and labeling of the defect content. Compared with traditional machine learning, which requires training a large model, this method is more convenient. (2) Real-time response and edge processing: Image preprocessing and retrieval tasks are realized through edge computing nodes, which greatly reduces network latency, improves detection speed, and meets the real-time detection needs of tire production lines. (3) Intelligent judgment and context enhancement: The retrieval enhancement generation module makes full use of historical defect data and combines the generation capabilities of large models to realize human-like judgment and defect explanation, and improves the understandability and credibility of decision-making. (4) Traceability and human-machine collaboration: The system supports the recording of the detection process and the traceability of results, which is convenient for quality control and problem review. At the same time, it supports manual intervention and knowledge feedback to build a human-machine collaboration closed loop. It not only realizes the automation and intelligence of detection, but also significantly improves the accuracy, efficiency and quality control capabilities of the detection and production line. It is suitable for defect identification and quality assessment tasks in tire appearance inspection. Attached image description: Appendix Figure 1 This is a system flowchart of the present invention. Detailed implementation method: The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0017] Figure 1 The tire appearance defect detection system based on retrieval enhancement generation technology proposed in this invention uses a high-precision industrial camera to capture images of the tire surface. The acquired images are transmitted to the edge computing end for preprocessing, including image enhancement, noise reduction and other operations, to ensure the image quality for subsequent analysis. The system's retrieval enhancement generation module includes: an image vectorization module that uses an image vectorization model to vectorize preprocessed images, and the generated vectors are the basis for defect retrieval and judgment; a vectorization retrieval module that compares the current image vector with historical defect vectors and quickly finds the most relevant historical defect samples and defect information through a vector retrieval module, providing support for subsequent defect judgment; an Ethernet module for transmitting user data to a multimodal large model; and a multimodal model for judging tire surface defects by combining historical defect information and image data with operator prompts. The final detection results output by the above multimodal model are transmitted to the enterprise's production management system, which will store and manage the data. The production management system will push the results to the operators as needed to help them quickly identify tire appearance defects. The production management system will also push the results to the managers as needed to facilitate further decision-making.

[0018] The technical solution of this invention achieves intelligent detection and quality management optimization of tire appearance defects by organically combining multiple aspects such as edge computing, image vectorization, vectorized retrieval, RAG technology, multimodal large model, and production management system. The entire detection process is real-time, efficient, and highly accurate, significantly improving the level of tire production quality control.

[0019] Example: This example provides a tire appearance defect detection method for the system described above, specifically including the following steps: Step 1: Image Data Acquisition and Preprocessing Step 1-1: Grayscale Conversion. The color tire images captured by the industrial camera are converted into single-channel grayscale images, and a weighted average model is used to improve the stability of subsequent analysis.

[0020] Step 1-2: CLAHE Enhancement. Adaptive Histogram Equalization (CLAHE) is applied to enhance local contrast through block processing and contrast limiting, while suppressing noise amplification.

[0021] Steps 1-3: Bilateral filtering for noise reduction. This process smooths the image while preserving edge features, effectively removing imaging noise and dust effects.

[0022] Steps 1-4: MSRCR Illumination Compensation. The multi-scale Retinex algorithm is used to adjust the image brightness, reduce tire shadows and reflections caused by uneven lighting, and ensure consistent performance under different lighting conditions.

[0023] Steps 1-5: Gamma correction. A non-linear adjustment is made to the image brightness (γ value set to 1.2–1.5) to improve shadow detail and prevent overexposure in highlight areas.

[0024] Steps 1-6: Gabor Filter Texture Enhancement. Gabor filters with different orientations are used to highlight the tread pattern and wear characteristics of the tire surface, enhancing sensitivity to subtle texture changes.

[0025] Step 2: The retrieval enhancement generation module sequentially performs image vectorization processing on the preprocessed data, conducts vectorization retrieval in the historical defect vector database, and then sends it to the multimodal large model via Ethernet to complete data fusion and image defect judgment results. Specifically, this includes the following steps: Step 2-1: The retrieval enhancement generation module performs image vectorization processing on the preprocessed data. Specifically, it uses the CLIP-ViT-B / 16 model to encode the preprocessed image, outputs feature vectors, performs L2 normalization on the feature vectors, and uses PCA / FAISS PQ to compress the image vectors to reduce transmission and storage costs. Step 2-2: Perform vectorized retrieval in the historical defect vector database. Specifically, use Milvus to build a historical defect vector database. Each record contains: image ID, vector features, defect type (crack, bulge), defect description (text description), and acquisition time information. Vectorized retrieval refers to performing a Top-K similarity search on the current image vector in Milvus and returning the results. The returned results include: Top-K sample ID, vector distance (similarity), and defect identification information. Steps 2-3: Multimodal large model fusion judgment. Based on the information of Top-K similar samples, the image defects are finally determined by combining multimodal large models such as BLIP-2, LLaVA, or CLIP+Transformer, and detailed defect type, location information, and natural language explanation are provided for manual review. Step 3: The retrieval enhancement generation module outputs the judgment results to the production management end and the edge computing end. The edge end is used for appearance defect early warning, and the production management end archives the judgment results. Specifically, when the judgment result is a new defect, the image data corresponding to the new defect is vectorized and the vectorized data is stored in the historical defect vector library.

[0026] In this example, the image vectorization module is deployed on the server to vectorize the preprocessed images and ensure the accuracy of the vectors in similarity calculation through feature normalization. These vectors serve as the basic data for defect retrieval and judgment, and are stored in the local cache and synchronized to the vector database. The vectorized retrieval module supports fast vector retrieval based on cosine similarity or Euclidean distance. The newly acquired tire image vector is compared with the historical defect sample database. The system can complete the similar sample retrieval in milliseconds and obtain the defect sample and defect information that is closest to the current tire defect. In this example, the multimodal large model is deployed on a high-performance server (supporting GPU clusters or distributed inference frameworks). The model input includes information such as the current tire image vector and historical defect retrieval results, and the output includes information such as defect type and defect location.

[0027] The aforementioned architecture, which combines vector retrieval with large-scale model generation, belongs to the Retrieval-Augmented Generation (RAG) technology paradigm. RAG technology has been widely used in various fields such as question-answering systems, intelligent customer service, and medical diagnostic assistance. Its core idea is to dynamically enhance the reasoning ability of the generated model through an external knowledge base, thereby avoiding model illusion and improving the accuracy of results. This invention introduces the RAG mechanism into the tire appearance defect detection scenario and constructs a dynamic discrimination system based on historical defect cases as knowledge sources, enabling multimodal large models to perform reliable reasoning in the context of real industrial data. This mechanism can improve the tire appearance defect detection capability in the following aspects: (1) Improve the response capability to new defect types - without retraining the model, it can support the preliminary identification and description of unseen defects simply by expanding the vector library; (2) Enhance the interpretability of the judgment results - the generated judgment information can be associated with specific historical cases, providing semantic basis for "why it is judged as this type of defect"; (3) Improve the discrimination robustness in complex texture backgrounds - by retrieving similar historical samples as contextual references, it helps the model distinguish between normal pattern changes and real defect features; (4) Realize the dynamic evolution of defect judgment standards - as new samples are continuously added to the database, the system automatically inherits the latest quality inspection experience and adapts to changes in production line standards.

Claims

1. A method for detecting tire appearance defects based on retrieval-enhanced generation technology, characterized in that, Includes the following steps: Step 1: Acquire image data and preprocess the image data using edge computing. Step 2: The retrieval enhancement generation module sequentially performs image vectorization processing on the preprocessed data, performs vectorization retrieval in the historical defect vector database, and then sends it to the multimodal large model via Ethernet to complete data fusion and image defect judgment results. Step 3: The retrieval enhancement generation module outputs the judgment results to the production management end and the edge computing end. The production management end archives the judgment results. Specifically, when the judgment result is a new defect, the image data corresponding to the new defect is vectorized and the vectorized data is stored in the historical defect vector library.

2. The tire appearance defect detection method based on retrieval enhancement generation technology according to claim 1, characterized in that, In step 1, the preprocessing of image data specifically includes the following steps: Step 1-1: Grayscale processing, converting the color image into a single-channel grayscale image, unifying the brightness range, reducing the interference of redundant color information, and using a weighted average model as shown in equation (1): Gray=0.299R+0.587G+0.114B (1), while preserving the light and dark structure information of the tire surface, effectively suppressing the color shift caused by the workshop lighting source, and avoiding false edges or texture distortion caused by color noise; Step 1-2: Adaptive Histogram Equalization (CLAHE): Divide the grayscale image obtained in the previous step into multiple small regions, perform histogram equalization on each region to enhance local contrast, prevent excessive noise amplification by setting a contrast limit threshold clipLimit, and use interpolation to smooth adjacent regions to improve the clarity of tire texture details. Steps 1-3: Bilateral filtering denoising: While smoothing image noise, edge features are preserved. This algorithm comprehensively considers the spatial distance between pixels and gray-level similarity to effectively remove imaging noise and dust interference. Steps 1-4: Multi-scale Retinex Illumination Compensation (MSRCR): Adaptively corrects image brightness to eliminate the effects of shadows and reflections caused by uneven illumination; restores the true brightness and detail levels of the image through multi-scale Gaussian filtering and logarithmic transformation, so that the tire surface features remain consistent under different lighting conditions. Steps 1-5: Gamma correction, which performs non-linear adjustment to the image brightness. The γ value is set to 1.2-1.5 and fine-tuned according to the actual effect to enhance the details in the shadows and prevent overexposure in the highlights. Steps 1-6: Gabor Filter Texture Enhancement: Multi-directional Gabor filters are used to extract tire surface texture information. By setting filter kernels in different directions, the image is convolved to highlight the pattern structure and wear features, thereby improving the sensitivity of subsequent detection to subtle texture changes.

3. The tire appearance defect detection method based on retrieval enhancement generation technology according to claim 2, characterized in that, The retrieval enhancement generation module described in step 2 performs image vectorization processing on the preprocessed data. Specifically, it uses the CLIP-ViT-B / 16 model to encode the preprocessed image, outputs feature vectors, performs L2 normalization on the feature vectors, and uses PCA / FAISS PQ to compress the image vectors to reduce transmission and storage costs.

4. The tire appearance defect detection method based on retrieval enhancement generation technology according to claim 3, characterized in that, In step 2, a vectorized retrieval is performed in the historical defect vector library. Specifically, a historical defect vector library is constructed using Milvus. Each record contains: image ID, vector features, defect type, defect description, and acquisition time information. Vectorized retrieval refers to performing a Top-K similarity search on the current image vector in Milvus and returning the results. The returned results include: Top-K sample ID, vector distance, and defect identification information.

5. The tire appearance defect detection method based on retrieval enhancement generation technology according to claim 4, characterized in that, In step 2, the data returned by vectorized retrieval is sent to the multimodal large model via Ethernet. The multimodal large model completes data fusion and image defect judgment results. The multimodal large model includes BLIP-2 / LLaVA / CLIP+Transformer to fuse image and text information. Cross-attention is used to enable the model to refer to the text information or labels of historical samples when judging the current image defect, aligning image features with text / historical sample features, and finally outputting defect type and defect location information for manual review.

6. The tire appearance defect detection method based on retrieval enhancement generation technology according to claim 5, characterized in that, The judgment logic for the multimodal large model in step 2 is as follows: (1) If the similarity with the Top-1 sample is ≥ 0.9, the defect type is automatically determined, and the determination information is generated by calling the multimodal large model; (2) If 0.7 ≤ similarity with Top-1 sample < 0.9, then the multimodal large model is called for verification and judgment information is generated; (3) If the similarity with the Top-1 sample is < 0.7, it is marked as a new defect, and the multimodal large model is called to generate judgment information.

7. The tire appearance defect detection method based on retrieval enhancement generation technology according to claim 6, characterized in that, In step 3, the judgment result is returned to the edge terminal via Ethernet. The edge computing terminal triggers the corresponding local response mechanism based on the received defect type and severity information, including but not limited to: highlighting the defect location on the local human-machine interface, triggering an audible and visual alarm, or sending a stop signal to the production line PLC to isolate suspected non-conforming products.

8. A system for tire appearance defect detection based on retrieval enhancement generation technology as described in any one of claims 1-7, characterized in that, The system includes a data acquisition module, an edge computing terminal for preprocessing acquired data, a retrieval enhancement generation module deployed on a server, and a production management terminal. The data acquisition module is implemented using an industrial camera, and its output is connected to the edge computing terminal, which is equipped with an image preprocessing module. Both the edge computing terminal and the production management terminal establish data communication with the server.

Citation Information

Cited By

  • Tire surface defect detection method and system based on multispectral image fusion

    CN122244036A