Optical detection method for apparent defects of flexible production line

Abstract

The invention relates to the technical field of industrial intelligent manufacturing, in particular to a flexible production line appearance defect optical detection method. Comprising the steps of performing image acquisition on semi-finished products and finished products on a production line; carrying out preprocessing and spatial transformation on the collected image; performing feature extraction and clustering analysis on the positive sample image of the to-be-detected product by using unsupervised learning to form a feature library; comparing the feature vector of the to-be-detected image with a feature library, and outputting the defect position and type of the sample; an improved RTDETR model with supervised learning is adopted, a sample judged as a defect is used for training, and the improved RTDETR model adopts a ResNet and Visual Transform network structure; and performing real-time detection and result feedback on the deployment model. According to the technical scheme, the appearance defect detection efficiency and the product quality of products manufactured on the flexible production line can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial intelligent manufacturing, and in particular to a method for optically detecting appearance defects of a flexible production line. Background Art

[0002] In today's rapidly changing market environment, consumers' demand for personalized and diversified products is increasing, which puts higher demands on the production model of the manufacturing industry. As a highly flexible production system, flexible production lines can quickly adjust production configurations to meet the production needs of different product batches, so they have been widely used in many industries. The core advantage of flexible production lines lies in their convenience in adjusting machines and changing lines, which enables companies to respond to market changes efficiently, shorten product launch cycles, and enhance market competitiveness.

[0003] However, with the increase in product types and the acceleration of production pace, the challenges of semi-finished and finished product appearance defect detection in the process flow of flexible production lines have become increasingly prominent. Appearance defects not only affect the quality of the product, but may also directly affect the yield rate of the process flow and the production cost of the overall finished product.

[0004] Traditionally, appearance defect detection mainly relies on manual visual inspection or automated inspection equipment based on fixed patterns. Although manual visual inspection is flexible, it is limited by human subjective judgment, fatigue and inspection speed, and it is difficult to ensure the consistency and efficiency of inspection. Especially in high-intensity and long-term production environments, the risk of misjudgment and missed inspection increases significantly. Although automated inspection equipment based on fixed patterns has improved inspection efficiency to a certain extent, its preset inspection algorithms and parameters are generally difficult to adapt to the frequent switching of product types in flexible production lines, resulting in poor inspection adaptability and increased misjudgment rate, especially when facing new or complex defects. Summary of the invention

[0005] The purpose of the present invention is to provide a method for optically detecting appearance defects of a flexible production line, which can improve the detection efficiency and product quality of appearance defects of products manufactured on the flexible production line.

[0006] To achieve the above-mentioned objectives, an embodiment of the present disclosure provides an optical detection method for appearance defects of a flexible production line, comprising: real-time image acquisition of semi-finished products and finished products on the production line; preprocessing and spatial transformation of the acquired images; using unsupervised learning to extract features and perform cluster analysis on positive sample images of the products to be inspected to form a feature library; comparing the feature vector of the image to be inspected with the feature library, and outputting the defect location and type of the sample; using an RTDETR model improved by supervised learning and using samples judged to be defective for training, the improved RTDETR model using ResNet and Visual Transformer network structures; and deploying the model for real-time detection and result feedback.

[0007] Beneficial effects of the basic solution: By collecting images of semi-finished and finished products on the production line, quality problems in the production process can be discovered in a timely manner, thereby preventing defective products from flowing into subsequent processes and significantly improving production efficiency and product quality. Preprocessing and spatial transformation of the collected images can eliminate noise in the images, improve image clarity, and correct geometric distortion in the images, providing more accurate and stable image data for subsequent feature extraction, while spatial changes to the images can increase the diversity of sample spatial scales for data augmentation.

[0008] Through the deep learning model, feature extraction and cluster analysis are performed on the positive sample images of the products to be inspected to form a feature library, which can realize intelligent detection and classification of defects. It not only improves the accuracy of detection, but also can adapt to the inspection needs of products of different types and models (including but not limited to defects such as dirt, scratches, overflowing glue, chipping, short circuit, leaking solder joints, and cold solder joints). At the same time, this method only requires a small number of positive samples to achieve a higher detection effect.

[0009] By comparing the feature vector of the image to be inspected with the feature library, the defect location and type of the sample can be quickly output to ensure that the defects are accurately labeled and recorded. The advantage of unsupervised learning is that it can achieve efficient detection performance without a large number of labeled samples. It is suitable for rapidly changing flexible production environments. It not only improves the efficiency of detection, but also provides detailed data support for subsequent quality analysis.

[0010] The improved RTDETR model adopts the ResNet and Visual Transformer network structure, which can extract global features and focus on important local features. The model training based on deep learning can effectively reduce the probability of misjudgment and missed judgment, and improve the accuracy of detection. Compared with traditional methods, the detection accuracy is improved by more than 30%.

[0011] The combination of unsupervised learning and supervised learning significantly improves the detection speed, enables the system to quickly respond to production changes, and reduces product scrapping due to defects. The unsupervised learning algorithm ensures 100% detection of known and unknown defects, and at the same time, the detected image samples to be confirmed are annotated and sent to the supervised learning algorithm for model training, achieving secondary high-precision confirmation of defective samples, thereby improving the efficiency of machine adjustment and line change.

[0012] Through the combination of the above technical features, the present invention greatly improves the detection efficiency of flexible production lines when dealing with the production of different industrial products. Flexible detection schemes can quickly respond to market demand and reduce the scrap rate of products due to appearance defects, thereby improving the overall product yield, reducing production costs, and enhancing the economic benefits and market competitiveness of enterprises.

[0013] As an implementable preferred solution, a multi-color array light source and a high-resolution camera are used for image acquisition, and the multi-color array light source uses an LED light source.

[0014] As an implementable preferred solution, the collected images are preprocessed and spatially transformed, including the following: Gaussian filtering is used to denoise the image, and histogram equalization is used to enhance the image contrast; Random scaling, random cropping and random splicing are used to spatially transform images to generate diverse training samples.

[0015] As an implementable preferred solution, a deep learning model is used to perform feature extraction and cluster analysis on the positive sample images of the product to be tested to form a feature library, including the following: A convolutional neural network is used to extract features from the preprocessed image. The pre-trained VGG16 network is selected as the feature extractor. The network parameters are fine-tuned to adapt it to the defect detection task. The fully connected layer output of the VGG16 network is used as the feature vector. The K-means clustering algorithm is used to classify the extracted feature vectors, and similar features are grouped into one category to form a feature library.

[0016] As an implementable preferred solution, the feature vector of the image to be detected is compared with the feature library, and the defect location and type of the sample are output, including the following: By calculating the cosine similarity between the feature vector of the image to be detected and the central feature vector of each category in the feature library, it is determined whether the image to be detected is a defective product; For defective products, the sliding window algorithm is used to locate the defective area, and the defect type of the defective area is determined by calculating the cosine similarity between the defective area and the central feature vector of each category in the feature library.

[0017] As an implementable preferred solution, the RTDETR model improved by supervised learning is used and trained using samples judged to be defects, including the following: The samples that are judged to be defects are marked and collected, and the marking content includes defect type, defect location and defect size; The improved RTDETR model adopts the ResNet and Visual Transformer network structures. The ResNet module is used to extract local features of defect images, and the Visual Transformer module is used to extract global features by annotating semantic information blocks of defect images. The feature coupling unit is used to fuse the features extracted by the ResNet module and the Visual Transformer module.

[0018] As an implementable preferred solution, the Visual Transformer module is used to extract global features from the semantic information blocks of the defect image annotations, including the following: The defect image is divided into blocks of a certain size, and each block is converted into a feature vector through an embedding layer. The Transformer structure is used to calculate the feature vector using a self-attention mechanism to extract global features.

[0019] As an implementable preferred solution, a feature coupling unit is used to fuse the features extracted by the ResNet module and the VisualTransformer module, including the following: The FPN network structure is used to perform multi-scale fusion of the features of the ResNet module and the Visual Transformer module. By using the FPN network to interactively fuse local and global features at different feature scales, comprehensive extraction of image features is achieved.

[0020] As an implementable and optimal solution, during the training process, 1*1 convolution is used to align the channel dimension, the up / down sampling strategy of the FPN network is used to align the feature resolution, and LayerNorm and BatchNorm are used to align the feature values; The defect image is input and encoded in blocks combined with the annotated semantic information; the Multi-head Self-attention mechanism is introduced to reduce or increase the convolution of the image through the FPN convolutional network; the Hungarian algorithm is used for result post-processing, and finally the defect classification result information and defect position regression information of the defect image are output.

[0021] As an implementable and optimal solution, the model is deployed for real-time detection and result feedback, including the following: Automatically adjust the brightness of the multi-color array light source and the camera exposure time by monitoring the accuracy and stability of the test results; Provide real-time feedback through display interface and / or alarm device, the feedback information includes the number, type and location of defective products. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The figure is a logical schematic diagram of an optical detection method for appearance defects of a flexible production line.

[0023] Figure 2 This is a schematic diagram of the architecture of the RTDETR model.

[0024] Figure 3 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the technical solution and advantages of the present application clearer, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings. It will be understood that the specific embodiments described herein are only partial embodiments of the present invention, which are only used to explain the present application, rather than to limit the present application. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated, and they can be combined with each other to achieve better technical effects. The same reference numerals appearing in the drawings of the following embodiments represent the same features or components, which can be applied to different embodiments.

[0026] In addition, unless otherwise defined, the technical terms or scientific terms used in the description of the present invention should have the common meanings understood by those skilled in the art in the art to which the present invention belongs.

[0027] The present invention is further described in detail below in conjunction with the accompanying drawings: Reference numerals: electronic device 500 , processor 501 , communication interface 502 , memory 503 , bus 504 .

[0028] Embodiment 1 Reference Figure 1 , a method for optically detecting appearance defects of a flexible production line comprises the following steps.

[0029] Step S100, image data acquisition, specifically includes the following contents: Step S101, select a high-stability multi-color array light source, which can provide stable and uniform illumination to ensure the stability of the collected image quality. In this embodiment, the multi-color array light source uses an LED light source, and by accurately controlling the luminous intensity and color of the LED, uniform illumination of products of different colors and materials is achieved.

[0030] Step S102, a high-resolution camera is selected, which has a high resolution of a single pixel physical size of 2.8 microns, and can clearly capture product details. In this embodiment, the high-resolution camera uses an industrial-grade CMOS camera, which has the characteristics of high dynamic range and low noise, and can ensure that the collected image is of high quality and low noise.

[0031] Step S103, install a multi-color high-stability array light source and a high-resolution camera on the production line to collect real-time images of semi-finished products and finished products on the production line. By accurately adjusting the position, angle and focal length of the light source and camera, it is ensured that the collected images can fully reflect the appearance characteristics of the product. According to actual measurements, the defects that can be detected can reach 10 microns.

[0032] Step S200, data processing, specifically including the following contents: Step S201 , preprocessing the collected image, including denoising, contrast enhancement, etc. In one embodiment, Gaussian filtering is used to denoise the image, and histogram equalization is used to enhance the contrast of the image to improve the accuracy of subsequent feature extraction.

[0033] Step S202, spatially transform the image, including scaling, cropping, splicing, etc., to increase the diversity of sample spatial scales for data augmentation. In one embodiment, random scaling, random cropping, and random splicing are used to spatially transform the image to generate diverse training samples and improve the generalization ability of the deep learning model.

[0034] Step S300, unsupervised learning is performed, using a deep learning model to extract features and perform cluster analysis on positive sample images of the product to be tested to form a feature library, which specifically includes the following contents: Step S301, using a convolutional neural network (CNN) to extract features from the preprocessed image. In one embodiment, a pre-trained VGG16 network can be selected as a feature extractor, and the network parameters can be fine-tuned to adapt it to the defect detection task of the present technical solution. The VGG16 network has a deeper hierarchical structure and powerful feature extraction capabilities, and can extract deep features in the image, with higher accuracy and efficiency than traditional methods.

[0035] The feature vector extracted by the convolutional neural network is used as the basis for subsequent clustering analysis and defect detection. In one embodiment, the fully connected layer output of the VGG16 network is used as a feature vector, which has a high dimension and rich information and can fully reflect the appearance characteristics of the image.

[0036] Step S302: Use K-means clustering algorithm to classify the extracted feature vectors, classify similar features into one category, and form a feature library. K-means clustering algorithm is simple and efficient, and can achieve effective classification of feature vectors.

[0037] This unsupervised learning is mainly used to detect and analyze known and unknown defects in the appearance of semi-finished and finished products on the production line, and provide negative samples and defect annotations for subsequent supervised learning. The advantage of unsupervised learning is that it can achieve efficient detection performance without a large number of labeled samples, which is suitable for fast-changing flexible production environments.

[0038] Step S400, performing defect detection, specifically includes the following contents: Step S401, compare the feature vector of the image to be detected with the feature library, identify the abnormal feature area (using edge detection and contour extraction technology), and preliminarily determine it as a defective product. In one embodiment, cosine similarity is used as a metric between feature vectors, and the cosine similarity between the feature vector of the image to be detected and the central feature vector of each category in the feature library is calculated to determine whether the image to be detected is a defective product. If the cosine similarity between the feature vector of the image to be detected and the central feature vector of all categories in the feature library is lower than a preset threshold, the image is determined to be a defective product.

[0039] Step S402: If the image to be inspected is determined to be a defective product, the defect position and type are further output. In this embodiment, a sliding window algorithm is used to locate the defective area, and the category to which the defective area belongs, that is, the defect type, is determined by calculating the cosine similarity between the defective area and the central feature vector of each category in the feature library. At the same time, the output of the defect position is achieved by recording the position information of the defective area in the image.

[0040] Step S500, constructing a supervised learning data set, specifically including the following contents: Step S501: annotate and collect samples that are determined to be defects, and the annotation content includes information such as defect type, defect location, and defect size. At the same time, the annotated defect samples and positive samples together form a supervised learning data set.

[0041] Step S502, training is performed using an improved RTDETR (Real-Time DEtection TRansformer) model, referring to Figure 2Schematic diagram of the architecture of the RTDETR model. In one embodiment, the model uses the ResNet (Residual Network) and Visual Transformer network structures to achieve joint extraction of local and global features of the image, including: Step S502-1, using the ResNet module to extract local features of the defect image. In this embodiment, ResNet-50 is selected as the local feature extractor, and the local features of the image are extracted through its convolutional layer, pooling layer and fully connected layer.

[0042] Step S502-2, using the Visual Transformer module to extract global features from the semantic information blocks annotated on the defect image. In one embodiment, the defect image is divided into blocks of a certain size, each block is regarded as a "word", and each "word" is converted into a feature vector through an embedding layer. Then, the Transformer structure is used to calculate the feature vector using a self-attention mechanism to achieve the extraction of global features.

[0043] Step S502-3, a feature coupling unit is used to fuse the features extracted by the ResNet module and the Visual Transformer module. In one embodiment, a FPN (Feature Pyramid Network) network structure is used to perform multi-scale fusion of the features of the ResNet module and the Visual Transformer module. By using the FPN network to interactively fuse local and global features at different feature scales, comprehensive extraction of image features is achieved.

[0044] Step S502-4, during the training process, align the channel dimension through 1*1 convolution, align the feature resolution using the up / down sampling strategy of the FPN network, and align the feature values ​​using LayerNorm and BatchNorm to fuse the features of the two styles. By continuously eliminating the semantic differences between them, the global perception of local features and the local details of the global representation are enhanced.

[0045] Step S502-5, in the training of the improved RTDETR model, the defect image category or size, shape, position and other images are input, and block encoding is performed in combination with the annotated semantic information. The Multi-head Self-attention mechanism is introduced, and the convolution reduction and enlargement of the image can be performed using the FPN convolutional network. The Hungarian algorithm is used in the result post-processing part, and finally the defect classification result information and defect position regression information of the defect image are output.

[0046] This technical solution uses the feature extraction and self-attention mechanism of the RTDETR model to efficiently learn the labeled defect samples. Through continuous iterative training, the model can accurately judge the authenticity of defects in real-time detection, significantly reducing the probability of misjudgment and missed judgment.

[0047] Step S600, deploying the model to achieve real-time monitoring, specifically includes the following contents: Step S601: In the actual production process, the defect detection system of the present invention is deployed on the production line to monitor the products on the production line in real time. The defects of the products can be quickly identified through real-time image acquisition, preprocessing, feature extraction, cluster analysis and defect detection.

[0048] Step S700, providing result feedback, specifically including the following contents: Step S701, according to the detection results, the system automatically adjusts the detection parameters to improve the accuracy and efficiency of subsequent detection. In one embodiment, by monitoring the accuracy and stability of the detection results, the parameters such as the brightness of the multi-color array light source and the camera exposure time are automatically adjusted to improve the detection efficiency and accuracy.

[0049] Step S702: Provide real-time feedback through a display interface and / or an alarm device, etc. The feedback information includes the number, type, location, etc. of defective products. Real-time feedback is used to timely understand and review the operating status of the production line, and take corresponding measures to improve and optimize.

[0050] The disclosed embodiment also provides a system for optically detecting appearance defects of a flexible production line, which uses the above-mentioned method for optically detecting appearance defects of a flexible production line.

[0051] The embodiment of the present disclosure further provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, all steps of the above-mentioned method for optical detection of appearance defects of a flexible production line can be implemented.

[0052] A person of ordinary skill in the art can understand that the implementation of all or part of the process in a method for optical detection of appearance defects of a flexible production line can be completed by instructing related hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the process of each embodiment of a method for optical detection of appearance defects of a flexible production line. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0053] The embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for optical detection of appearance defects of a flexible production line when executing the program. In the embodiment of the present application, the processor is the control center of the computer system, which can be a processor of a physical machine or a processor of a virtual machine.

[0054] Reference Figure 3 The electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one bus 504. The bus 504 is used to realize the connection and communication between these components, the communication interface 502 is used to communicate with other node devices for signaling or data, and the memory 503 stores machine-readable instructions executable by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 through the bus 504, and when the machine-readable instructions are called by the processor 501, the steps of the above-mentioned method for optical detection of appearance defects of a flexible production line are executed.

[0055] The above contents are only embodiments of the present invention. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for the ordinary technicians in the relevant field to implement this application. It should be pointed out that for the technicians in this field, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.

Claims

1. A method for optical detection of appearance defects of a flexible production line, characterized by: include: Capture images of semi-finished products and finished products on the production line; perform pre-processing and spatial transformation on the captured images; Using unsupervised learning, feature extraction and cluster analysis are performed on the positive sample images of the product to be inspected to form a feature library; the feature vector of the image to be inspected is compared with the feature library, and the defect location and type of the sample are output; the RTDETR model improved by supervised learning is trained using samples judged to be defects. The improved RTDETR model uses the ResNet and Visual Transformer network structures; the model is deployed for real-time detection and result feedback.

2. The method for optical detection of appearance defects of a flexible production line according to claim 1, characterized in that: A multi-color array light source and a high-resolution camera are used for image acquisition, and the multi-color array light source adopts an LED light source.

3. The method for optically detecting appearance defects of a flexible production line according to claim 1, characterized in that: The collected images are preprocessed and spatially transformed, including the following: Gaussian filtering is used to denoise the image, and histogram equalization is used to enhance the image contrast; Random scaling, random cropping and random splicing are used to spatially transform images to generate diverse training samples.

4. The method for optically detecting appearance defects of a flexible production line according to claim 1, characterized in that: Using unsupervised learning, feature extraction and cluster analysis are performed on the positive sample images of the products to be tested to form a feature library, including the following: A convolutional neural network is used to extract features from the preprocessed image. The pre-trained VGG16 network is selected as the feature extractor. The network parameters are fine-tuned to adapt it to the defect detection task. The fully connected layer output of the VGG16 network is used as the feature vector. The K-means clustering algorithm is used to classify the extracted feature vectors, and similar features are grouped into one category to form a feature library.

5. The method for optically detecting appearance defects of a flexible production line according to claim 1, characterized in that: Compare the feature vector of the image to be inspected with the feature library, and output the defect location and type of the sample, including the following: By calculating the cosine similarity between the feature vector of the image to be detected and the central feature vector of each category in the feature library, it is determined whether the image to be detected is a defective product; For defective products, the sliding window algorithm is used to locate the defective area, and the defect type of the defective area is determined by calculating the cosine similarity between the defective area and the central feature vector of each category in the feature library.

6. The method for optically detecting appearance defects of a flexible production line according to claim 1, characterized in that: The RTDETR model improved by supervised learning is trained using samples judged as defects, including the following: The samples that are judged to be defects are marked and collected, and the marking content includes defect type, defect location and defect size; The improved RTDETR model adopts the ResNet and Visual Transformer network structures; the ResNet module is used to extract local features of defect images, and the Visual Transformer module is used to annotate semantic information blocks of defect images for global feature extraction; A feature coupling unit is used to fuse the features extracted by the ResNet module and the Visual Transformer module.

7. The method for optically detecting appearance defects of a flexible production line according to claim 6, characterized in that: The Visual Transformer module is used to extract global features from the semantic information blocks of defect images. Includes the following: The defect image is divided into blocks of a certain size, and each block is converted into a feature vector through an embedding layer. The Transformer structure is used to calculate the feature vector using a self-attention mechanism to extract global features.

8. The method for optically detecting appearance defects of a flexible production line according to claim 7, characterized in that: The feature coupling unit is used to fuse the features extracted by the ResNet module and the Visual Transformer module, including the following: The FPN network structure is used to perform multi-scale fusion of the features of the ResNet module and the Visual Transformer module. By using the FPN network to interactively fuse local and global features at different feature scales, comprehensive extraction of image features is achieved.

9. The method for optically detecting appearance defects of a flexible production line according to claim 8, characterized in that: During training, 1*1 convolution is used to align channel dimensions, up / down sampling strategies of FPN network are used to align feature resolutions, and LayerNorm and BatchNorm are used to align feature values. The defect image is input and encoded in blocks combined with the annotated semantic information; the Multi-head Self-attention mechanism is introduced to reduce or increase the convolution of the image through the FPN convolutional network; the Hungarian algorithm is used for result post-processing, and finally the defect classification result information and defect position regression information of the defect image are output.

10. The method for optically detecting appearance defects of a flexible production line according to claim 1, characterized in that: Deploy the model for real-time detection and result feedback, including the following: Automatically adjust the brightness of the multi-color array light source and the camera exposure time by monitoring the accuracy and stability of the test results; Provide real-time feedback through display interface and / or alarm device, the feedback information includes the number, type and location of defective products.

Images