Plate surface defect detection system and method based on machine vision
By combining multi-dimensional imaging and illumination compensation algorithms with convolutional neural networks and edge computing, the imbalance between false alarm and false alarm rates in machine vision board inspection systems under varying illumination conditions was solved, achieving efficient and accurate defect detection.
Patent Information
- Application Number
- CN202411160494.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing machine vision-based sheet metal surface defect detection systems are prone to issues with unbalanced false alarm and missed detection rates when lighting conditions change, making it difficult to maintain detection accuracy and efficiency on high-load production lines.
Image acquisition and preprocessing are performed using multi-dimensional imaging methods and illumination compensation algorithms. Combined with convolutional neural networks and multi-task learning methods, the model is optimized through edge computing to provide decision support to adapt to different illumination conditions.
It improves the accuracy and robustness of surface defect detection on sheet metal, reduces the risk of imbalance between false alarm and missed alarm rates, and enhances the system's adaptability and flexibility in different environments.
Smart Images

Figure CN119048472A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine vision technology, and more specifically, to a plate surface defect detection system and method based on machine vision. Background Art
[0002] Plate surface defect detection based on machine vision is a method that uses image processing and computer vision technology to automatically identify and analyze surface defects of plates such as metal plates, wood, glass, plastic, etc. The image of the plate surface is captured by a camera or other imaging device, and an image processing algorithm is used to detect, identify and classify possible surface defects such as scratches, cracks, dents, stains or other irregularities. The detection method based on machine vision may be affected by the detection environment, such as the projection effect of light on the surface texture of the plate, and cause abnormal failures. By improving the fitting ability of the visual detection model, the missed alarm rate of defect detection can be reduced, but at the same time, the false alarm rate of defect detection will be increased. At the same time, in order to meet the flow speed of the production line for real-time detection, maintaining a balance between the missed alarm rate and the false alarm rate when the visual detection model is running at high load has become a new challenge.
[0003] In order to solve the above defects, a technical solution is now proposed. Summary of the invention
[0004] The purpose of the present invention is to provide a system and method for detecting plate surface defects based on machine vision to solve the problems in the above-mentioned background technology.
[0005] To achieve the above object, the present invention provides the following technical solution: a plate surface defect detection method based on machine vision, the specific steps include:
[0006] Adaptively adjust the lighting conditions of the testing environment, collect plate images through multi-element imaging, dynamically adjust the real-time collected images using the lighting compensation algorithm, and pre-process the plate image data;
[0007] A convolutional neural network model is established to detect and classify defects in plate image data, and the adaptability of the convolutional neural network model to lighting conditions is improved through multi-task learning methods.
[0008] The edge computing method is used to reduce the transmission delay and calculation time of image data, and the model compression technology is used to optimize the model;
[0009] Combine the performance of the convolutional neural network model and the processing efficiency of edge computing, and provide decision support based on the production status of the plate.
[0010] Preferably, the process of dynamically adjusting the real-time collected image using the illumination compensation algorithm is as follows:
[0011] The collected plate image is subjected to Gaussian blur processing to generate a lighting estimation image. The original image is divided by the lighting estimation image to remove the influence of uneven lighting. The contrast and brightness of the compensated image are adjusted to enhance the details, and the processed image is output.
[0012] Preferably, the specific steps of adjusting the image according to the illumination estimation image generated by the Retinex algorithm are:
[0013] The plate image is processed using Gaussian blur to obtain a smooth image L(x,y) representing the illumination. L(x,y) is defined to represent the image as an illumination estimation image.
[0014] Divide the original image by the illumination estimation image to calculate the reflectance image R(x,y). The calculation expression of R(x,y) is Where I(x,y) is the brightness value of the plate image, and L(x,y) is the illumination estimation value obtained by Gaussian blur;
[0015] The dynamic range of the reflectance image is compressed by logarithmic transformation, and the calculation expression is R 1 (x,y)=log(1+R(x,y));
[0016] Apply the Retinex algorithm to each color channel in the image separately, and perform color restoration in combination with the color restoration factor. The calculation expression is: Where C(x,y) is the original color channel value, C 1 (x, y) is the restored color channel value, G is the global gain coefficient;
[0017] Output the image processed by the Retinex algorithm.
[0018] Preferably, preprocessing the plate image data includes normalizing the image data using a grayscale normalization method, enhancing the image using an adaptive histogram equalization method, and denoising the image using a high-pass filtering method.
[0019] Preferably, the method of normalizing the image data using the grayscale normalization method is:
[0020] The grayscale value of the calibrated image pixel (x, y) is Gr(dx, y), and the normalized calculation expression is Grno(x, y) = Z × log(1 + Gr(x, y)), where Grno(x, y) is the normalized pixel grayscale value and Z is the scaling constant.
[0021] Preferably, the method of enhancing the image using adaptive histogram equalization is:
[0022] Divide the image into several small local areas, and perform histogram equalization on each local area separately;
[0023] Calculate the histogram of each local area and perform histogram equalization on the local area;
[0024] The adjacent local regions are smoothed using an interpolation method, and all processed local regions are re-merged to form an enhanced image;
[0025] The specific process of denoising an image using the high-pass filtering method is as follows:
[0026] Use the Laplacian filter to convolve the image to highlight the edges and details of the image and suppress smooth areas;
[0027] Combine the original image with the filtered result to preserve the enhanced edges and reduce the noise in other areas;
[0028] To suppress and denoise high-frequency components, the image is converted to the frequency domain through a frequency domain filter for processing. The processing method is as follows:
[0029] Perform Fourier transform on the image to convert it into frequency domain representation;
[0030] Use a high-pass filter to apply to the frequency domain image and suppress low-frequency components;
[0031] Perform inverse Fourier transform on the filtered image and return to the spatial domain to obtain the processed image.
[0032] Preferably, the modeling process of the convolutional neural network model is:
[0033] Based on the plate image data, including samples collected under different lighting conditions, the defect type and the lighting conditions of the image are annotated, and the training set is expanded using data augmentation technology;
[0034] Use a convolutional neural network model to extract hierarchical features of the image. The layers include convolutional layer, pooling layer, ReLU activation function, and fully connected layer.
[0035] A shared convolutional layer is designed at the front end of the convolutional neural network model to extract common features of the image. Common features are shared between different tasks.
[0036] After the shared feature extraction layer, the defect detection branch and the illumination classification branch are designed. The defect detection branch targets different defect types and uses convolutional layers and fully connected layers for classification. The illumination classification branch is used to classify the illumination conditions of the image.
[0037] Define a joint loss function to combine the defect detection loss and the illumination classification loss and optimize them simultaneously;
[0038] Use existing image classification datasets to pre-train the model, then fine-tune the model on plate image data, and use learning rate decay or adaptive learning rate optimizer to promote convergence of the later model training;
[0039] K-fold cross validation is used to evaluate the performance of the model, and the confusion matrix is used to evaluate the classification performance of the model under different lighting conditions. When the model performance and classification ability meet the production standards, the model is output.
[0040] Preferably, the setting method of the joint loss function is:
[0041] The cross entropy loss function is used for the classification task of plate defects. The classification task has Ca defect categories. For category i, the category probability predicted by the model is p i , the true label is y i , then the loss function of the classification task is expressed as
[0042] For the illumination adaptation task, the cross entropy loss function is used to classify the illumination conditions. Assuming that the illumination classification task has Lig categories, the illumination category probability predicted by the model for category j is q j , the true label is z j , then the loss function of the illumination adaptation task is expressed as
[0043] The expression of the joint loss function is L tot =α*L cro +β*L ill , where L tot is the joint loss function, α and β are L cro and L ill The weight coefficient of , and α and β are both positive integers;
[0044] Add a regularization term to the joint loss function for L2 regularization. The calculation expression of L2 regularization is L tot =α*L cro +β*L ill +λ*R(W), where λ is the regularization coefficient and R(W) is the regularization term representing the model weight W.
[0045] Preferably, the logic for providing decision support is:
[0046] The time for a complete inference of the calibrated neural network model in edge computing is In t , the time for image data to be transmitted from the edge device to the cloud is Tr l , the moving speed of the detection plate is v pAccording to the actual monitoring log data after the plate surface defect detection model based on convolutional neural network and multi-task learning is applied, the actual false alarm rate and missed alarm rate of the model are calculated, and the false alarm rate is marked as Fa r , the false negative rate is Fn r , then the balance control index of plate surface defect detection is In the formula, e is a natural constant;
[0047] The calculated balance control index B cr With the preset first balance control limit B t1 , the second balance control limit B t2 Compare, the first balance control limit B t1 Less than the second balance control limit B t2 , when the balance control index B cr Greater than or equal to the second balance control limit B t2 When , the fitting of the plate surface defect detection is marked too deep, indicating that the false alarm rate and missed alarm rate of defect detection are unbalanced;
[0048] When the balance control index B cr Less than or equal to the first balance control limit B t1 When , the fitting of the plate surface defect detection is marked as too shallow, indicating that the false alarm rate and missed alarm rate of defect detection are unbalanced;
[0049] When the first balance control limit B t1 Less than balance control index B cr Less than the second balance control limit B t2 When , the fitting of the surface defect detection of the marked plate is appropriate, indicating that the false alarm rate and missed alarm rate of defect detection are balanced.
[0050] Plate surface defect detection system based on machine vision, including multivariate imaging module, convolution modeling module, edge computing module, and decision support module;
[0051] The multivariate imaging module is used to adaptively adjust the lighting conditions of the detection environment, collect plate images through multivariate imaging, dynamically adjust the real-time collected images using the lighting compensation algorithm, and pre-process the plate image data;
[0052] The convolution modeling module is used to establish a convolutional neural network model to detect and classify defects in plate image data, and to improve the adaptability of the convolutional neural network model to lighting conditions through multi-task learning methods;
[0053] The edge computing module is used to reduce the transmission delay and computing time of image data through edge computing methods, and optimize the model using model compression technology;
[0054] The decision support module is used to combine the performance of the convolutional neural network model and the processing efficiency of edge computing, and provide decision support based on the production status of the plate.
[0055] In the above technical solution, the technical effects and advantages provided by the present invention are:
[0056] This application combines convolutional neural networks with multi-task learning methods, sets a joint loss function, and optimizes the detection and classification capabilities of plate surface defect detection at the same time. By combining the processing capabilities of edge computing with the actual false alarm rate and missed alarm rate in production data, the application effect of plate surface defect detection is comprehensively evaluated, the fitting ability of the detection is effectively verified, and the control difficulties of over-fitting and under-fitting are avoided. The modeling method of the model is effectively verified through actual detection performance, which reduces the reuse ability of the model and increases the optimization space of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0058] Figure 1 The figure is a flow chart of the method of the present invention.
[0059] Figure 2 It is a system module diagram of the present invention. DETAILED DESCRIPTION
[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0061] Example 1: Please refer to Figure 1 As shown, the present invention is a plate surface defect detection method based on machine vision, and the specific steps include:
[0062] Adaptively adjust the lighting conditions of the testing environment, collect plate images through multi-element imaging, dynamically adjust the real-time collected images using the lighting compensation algorithm, and pre-process the plate image data;
[0063] A convolutional neural network model is established to detect and classify defects in plate image data, and the adaptability of the convolutional neural network model to lighting conditions is improved through multi-task learning methods.
[0064] The edge computing method is used to reduce the transmission delay and calculation time of image data, and the model compression technology is used to optimize the model;
[0065] Combine the performance of the convolutional neural network model and the processing efficiency of edge computing, and provide decision support based on the production status of the plate.
[0066] Lighting conditions have a significant impact on machine vision-based plate surface defect detection. This is because lighting directly affects the quality and feature performance of the image, thereby affecting the effectiveness of the detection algorithm. Changes in light intensity will cause changes in image brightness and contrast. If the light is too strong, the image may be overexposed and details may be lost; insufficient light may cause the image to be too dark and the defects may not be obvious. Reduced contrast makes it difficult for the detection algorithm to distinguish between defects and normal surface features. For example, in an overexposed image, small scratches may be completely invisible, and in an overly dark image, tiny dents may be ignored.
[0067] Uneven lighting can easily create shadows or reflections on the surface. These optical effects can be mistaken for surface defects. Shadows can mask actual defects, while reflections can create bright areas in the image, interfering with detection. For example, on a metal plate with a smooth surface, strong reflections can mask small cracks or scratches, leading to false negatives.
[0068] The angle of incidence of the light determines how surface features appear in the image. Some defects, such as tiny scratches or dents, are only visible at certain angles and may not be obvious at other angles. For example, a small scratch is more visible at an angle parallel to or slightly oblique to the light, but may not be visible at all under vertical lighting.
[0069] Adaptive processing of lighting can improve the accuracy of detection. Adaptive lighting processing can ensure consistent image quality under different lighting conditions, so that defects can be clearly identified under various conditions. Adaptive processing can reduce missed reports and false alarms caused by lighting changes, and improve the reliability of overall detection;
[0070] Adaptive lighting processing can reduce interference from shadows, reflections, etc., which can prevent these factors from being mistaken for defects or covering up actual defects. Through lighting compensation and image enhancement technology, real surface defects can be detected more accurately;
[0071] In actual production environments, lighting conditions will inevitably change, for example, due to interference from external light sources, equipment aging, or other environmental factors. Lighting adaptability processing enables the system to maintain stable detection performance in different environments, enhancing the robustness and adaptability of the system;
[0072] At the same time, different production lines, different materials and product types may require different lighting conditions. By adapting the lighting, the machine vision system can be flexibly applied to various production environments without the need for frequent debugging or replacement of equipment.
[0073] The multi-dimensional imaging method refers to the acquisition of plate images by multi-angle imaging and multi-light source illumination in the plate surface defect detection process. Multiple cameras are arranged at different angles. The cameras simultaneously or sequentially shoot the plate surface to capture the same defect at different angles. For example, the cameras are arranged at different inclination angles relative to the plate surface, including directly above, 45 degrees, and side angles. The imaging areas of the cameras overlap to a certain extent, so that multiple images can be fused and analyzed later.
[0074] Arrange multiple light sources to illuminate the surface of the plate from different directions and at different angles, and reveal more surface details by taking images under different lighting conditions. For example, install multiple light sources, including ring lights, line lights, and point lights, and place them at different positions of the plate, such as above, on the side, or at the bottom. Use an adjustable light source control system to light up different light sources in sequence or simultaneously, and control the intensity and angle of the light source to avoid excessive reflections or shadows. Each time the light source changes, the camera takes images to record the surface features under different lighting conditions;
[0075] Using structured light projection, different patterns of light, such as stripes or grids, are projected onto the surface of the sheet and images are captured to enhance the recognition of surface features. A structured light projector is installed next to the camera to project predefined light patterns onto the surface of the sheet. Surface images under different light patterns are captured and the captured pattern deformation is used to reveal surface unevenness and defects.
[0076] The process of dynamically adjusting the real-time collected images using the illumination compensation algorithm is as follows: Gaussian blur processing is performed on the collected plate image to generate an illumination estimation image, the original image is divided by the illumination estimation image to remove the influence of uneven illumination, the contrast and brightness of the compensated image are adjusted to enhance the details, and the processed image is output;
[0077] The generation of the illumination estimation image is formed by estimating the illumination distribution of the image through Gaussian blur. The illumination compensation algorithm used is the Retinex algorithm. The specific steps of using the Retinex algorithm to adjust the image are:
[0078] The plate image is processed using Gaussian blur to obtain a smooth image L(x,y) representing the illumination. L(x,y) is defined to represent the image as an illumination estimation image.
[0079] Divide the original image by the illumination estimation image to calculate the reflectance image R(x,y). The calculation expression of R(x,y) is Where I(x,y) is the brightness value of the plate image, and L(x,y) is the illumination estimation value obtained by Gaussian blur;
[0080] The dynamic range of the reflectance image is compressed by logarithmic transformation, and the calculation expression is R 1 (x,y=log(1+R(x,y));
[0081] Apply the Retinex algorithm to each color channel in the image separately, and perform color restoration in combination with the color restoration factor. The calculation expression is: Where C(x,y) is the original color channel value, C 1 (x, y) is the restored color channel value, G is the global gain coefficient;
[0082] Output the image processed by the Retinex algorithm.
[0083] It should be noted that color channels refer to the various components used to represent color in an image. In the RGB color space, the color of an image is represented by different combinations of the three basic colors: red, green, and blue. Each color channel stores the brightness value of a certain color in the image separately, and the three color channels together determine the final color representation of the image. In the Retinex algorithm, the color restoration process usually requires processing each color channel separately, and then combining the processed channels into the final color image;
[0084] The color restoration factor G is a global gain coefficient and is a parameter used to adjust and correct color distortion in the color restoration step of the Retinex algorithm. Since the Retinex algorithm may cause color changes when eliminating illumination unevenness, especially imbalanced proportions of various color channels, the color restoration factor is needed to adjust the relationship between these channels to ensure that the color of the final image is as close to the original image as possible, while retaining the brightness and contrast enhancement effects brought by the algorithm.
[0085] The preprocessing of the plate image data includes normalizing the image data using the grayscale normalization method, enhancing the image using the adaptive histogram equalization method, and denoising the image using the high-pass filtering method;
[0086] Preprocessing images can improve image quality, making subsequent image analysis, detection, or computer vision tasks including feature extraction, classification, and object detection more accurate and effective.
[0087] The image may be interfered by various noises during the acquisition process, such as thermal noise of electronic equipment, interference from ambient light, etc. Preprocessing removes or reduces noise through filtering, making the image clearer and reducing the impact of noise on subsequent analysis;
[0088] By adjusting the contrast of the image, the contrast between light and dark in the image is made more obvious, thereby highlighting the details or certain feature areas. Common methods include histogram equalization, adaptive histogram equalization, etc.
[0089] Adjust the brightness of the image to ensure that the image shows a more balanced brightness distribution under different lighting conditions and avoid the loss of overly bright or dark areas;
[0090] Standardize or normalize the image to ensure that the image data has a uniform scale in subsequent processing steps. This is especially important for training deep learning models, because the range and scale of the input image will affect the convergence speed and accuracy of the model;
[0091] Perform geometric correction (such as rotation, cropping, scaling, distortion correction, etc.) on the image to correct the geometric deformation caused by shooting angle, lens distortion, etc. Geometric correction can restore the objects in the image to the correct position and proportion, thereby improving the accuracy of subsequent analysis;
[0092] Image preprocessing is to clean, enhance and standardize image data, so that subsequent calculations and analysis are more accurate and efficient, and can maintain high stability and reliability under different conditions.
[0093] The method of normalizing image data using grayscale normalization method is:
[0094] The grayscale value of the calibrated image pixel (x, y) is Gr(x, y), and the normalized calculation expression is Grno(x, y) = Z × log(1 + Gr(x, y)), where Grno(x, y) is the normalized pixel grayscale value and Z is the scaling constant.
[0095] Grayscale normalization can improve the contrast and visibility of the image, making the image consistent under different lighting or imaging conditions, which facilitates subsequent image processing and analysis.
[0096] The method of using adaptive histogram equalization to enhance the image is:
[0097] Divide the image into several small local areas, and perform histogram equalization on each local area separately;
[0098] Calculate the histogram of each local area and perform histogram equalization on the local area;
[0099] The adjacent local regions are smoothed using an interpolation method, and all processed local regions are re-merged to form an enhanced image.
[0100] The specific process of denoising an image using the high-pass filtering method is as follows:
[0101] Use the Laplacian filter to convolve the image to highlight the edges and details of the image and suppress smooth areas;
[0102] Combine the original image with the filtered result to preserve the enhanced edges and reduce noise in other areas.
[0103] In a specific embodiment, for suppressing and denoising high-frequency components, the image is converted to the frequency domain through a frequency domain filter for processing, and the processing method is:
[0104] Perform Fourier transform on the image to convert it into frequency domain representation;
[0105] Use a high-pass filter to apply to the frequency domain image and suppress low-frequency components;
[0106] Perform inverse Fourier transform on the filtered image and return to the spatial domain to obtain the processed image.
[0107] High-pass filtering is used to retain high-frequency information in an image and weaken low-frequency components. High-pass filtering is mainly used to enhance the edges and details of an image, but it can remove certain types of noise through specific methods, such as smooth or low-frequency noise, high-frequency information such as edges, textures, etc., and low-frequency components such as backgrounds, uniform areas, etc.
[0108] A convolutional neural network model is established to detect and classify defects in plate image data, and the adaptability of the convolutional neural network model to lighting conditions is improved through multi-task learning methods. The modeling process of the convolutional neural network model is as follows:
[0109] Based on the plate image data, including samples collected under different lighting conditions, the defect type and the lighting conditions of the image are annotated, and the training set is expanded using data augmentation technology;
[0110] Use a convolutional neural network model to extract hierarchical features of the image. The layers include convolutional layer, pooling layer, ReLU activation function, and fully connected layer.
[0111] A shared convolutional layer is designed at the front end of the convolutional neural network model to extract common features of the image. Common features are shared between different tasks.
[0112] After the shared feature extraction layer, the defect detection branch and the illumination classification branch are designed. The defect detection branch targets different defect types and uses convolutional layers and fully connected layers for classification. The illumination classification branch is used to classify the illumination conditions of the image.
[0113] Define a joint loss function to combine the defect detection loss and the illumination classification loss and optimize them simultaneously;
[0114] Use existing image classification datasets to pre-train the model, then fine-tune the model on plate image data, and use learning rate decay or adaptive learning rate optimizer to promote convergence of the later model training;
[0115] K-fold cross validation is used to evaluate the performance of the model, and the confusion matrix is used to evaluate the classification performance of the model under different lighting conditions. When the model performance and classification ability meet the production standards, the model is output.
[0116] In multi-task learning, a joint loss function is designed to combine the classification task and the illumination adaptation task. The losses of the two tasks are combined. The setting method of the joint loss function is:
[0117] The cross entropy loss function is used for the classification task of plate defects. The classification task has Ca defect categories. For category i, the category probability predicted by the model is p i , the true label is y i , then the loss function of the classification task is expressed as
[0118] For the illumination adaptation task, the cross entropy loss function is used to classify the illumination conditions. Assuming that the illumination classification task has Lig categories, the illumination category probability predicted by the model for category j is q j , the true label is z j , then the loss function of the illumination adaptation task is expressed as
[0119] The expression of the joint loss function is L tot =α*L cro +β*L ill , where L tot is the joint loss function, α and β are L cro and L ill The weight coefficient of , and α and β are both positive integers.
[0120] To prevent the model from overfitting, a regularization term is added to the joint loss function for L2 regularization. The regularization method is:
[0121] The calculation expression of L2 regularization is L tot =α*L cro+β*L ill +λ*R(W), where λ is the regularization coefficient and R(W) is the regularization term representing the model weight W.
[0122] It should be noted that data enhancement techniques such as rotation, flipping, brightness adjustment, etc., different tasks such as defect detection and lighting classification, existing image classification datasets such as ImageNet, and commonly used adaptive learning rate optimizers include Adam, RMSprop, etc.
[0123] The edge computing method is used to reduce the transmission delay and calculation time of image data, and the model compression technology is used to optimize the model;
[0124] Simple image processing tasks, including denoising, preprocessing, and format conversion, are completed on edge devices. Images are compressed before being transmitted to reduce data volume and transmission time.
[0125] Deploy lightweight neural network models on edge devices for real-time inference, use TinyML models designed for embedded devices, and reduce the computational complexity of the models through model pruning and quantization techniques;
[0126] Reasonably allocate computing tasks to edge devices and the cloud, and use a coordinator to dynamically classify edge device processing tasks and cloud processing tasks based on current network conditions and computing resource usage.
[0127] Edge devices refer to computing devices located at the edge of the network, close to the data source or user end. Edge devices directly process, store and transmit data without sending all data to a central data center or cloud. Edge computing is a distributed computing paradigm that places computing, storage and network resources at the edge of the network, that is, close to the data generation source or end user, rather than relying entirely on a remote central data center or cloud. The main purpose of edge computing is to perform computing at or near where the data is generated to reduce latency, lower bandwidth usage, and improve data privacy and security.
[0128] Edge computing is able to process data where it is generated, so it can detect defects and make decisions in real time, avoiding the delay of sending data to remote servers for processing. Edge devices process large amounts of data locally and only send processed results or important data to the cloud or central server, which reduces the amount of data that needs to be transmitted and saves bandwidth. Even if the network bandwidth connecting to the central server is limited or unstable, edge devices can continue to operate to ensure the continuity and reliability of the system. Edge devices can adjust defect detection models in real time according to actual production conditions, making the detection system more flexible and robust. Edge computing allows models to be learned online or fine-tuned locally to adapt to specific environments or emerging defect types without relying on central servers.
[0129] Combine the performance of the convolutional neural network model and the processing efficiency of edge computing, and provide decision support based on the production status of the plate.
[0130] The time for a complete inference of the calibrated neural network model in edge computing is In t , the time for image data to be transmitted from the edge device to the cloud is Tr l , the moving speed of the detection plate is v p According to the actual monitoring log data after the plate surface defect detection model based on convolutional neural network and multi-task learning is applied, the actual false alarm rate and missed alarm rate of the model are calculated, and the false alarm rate is marked as Fa r , the false negative rate is Fn r , then the balance control index of plate surface defect detection is In the formula, e is a natural constant;
[0131] The calculated balance control index B cr With the preset first balance control limit B t1 , the second balance control limit B t2 Compare, the first balance control limit B t1 Less than the second balance control limit B t2 , when the balance control index B cr Greater than or equal to the second balance control limit B t2 When , the fitting of the plate surface defect detection is marked too deep, indicating that the false alarm rate and missed alarm rate of defect detection are unbalanced;
[0132] When the balance control index B cr Less than or equal to the first balance control limit B t1 When , the fitting of the plate surface defect detection is marked as too shallow, indicating that the false alarm rate and missed alarm rate of defect detection are unbalanced;
[0133] When the first balance control limit B t1 Less than balance control index B cr Less than the second balance control limit B t2 When , the fitting of the surface defect detection of the marked plate is appropriate, indicating that the false alarm rate and missed alarm rate of defect detection are balanced.
[0134] Example 2: Please refer to Figure 2 As shown, the present invention is a plate surface defect detection system based on machine vision, including a multivariate imaging module, a convolution modeling module, an edge computing module, and a decision support module;
[0135] The multivariate imaging module is used to adaptively adjust the lighting conditions of the detection environment, collect plate images through multivariate imaging, dynamically adjust the real-time collected images using the lighting compensation algorithm, and pre-process the plate image data;
[0136] The convolution modeling module is used to establish a convolutional neural network model to detect and classify defects in plate image data, and to improve the adaptability of the convolutional neural network model to lighting conditions through multi-task learning methods;
[0137] The edge computing module is used to reduce the transmission delay and computing time of image data through edge computing methods, and optimize the model using model compression technology;
[0138] The decision support module is used to combine the performance of the convolutional neural network model and the processing efficiency of edge computing, and provide decision support based on the production status of the plate.
[0139] This application combines convolutional neural networks with multi-task learning methods, sets a joint loss function, and optimizes the detection and classification capabilities of plate surface defect detection at the same time. By combining the processing capabilities of edge computing with the actual false alarm rate and missed alarm rate in production data, the application effect of plate surface defect detection is comprehensively evaluated, the fitting ability of the detection is effectively verified, and the control difficulties of over-fitting and under-fitting are avoided. The modeling method of the model is effectively verified through actual detection performance, which reduces the reuse ability of the model and increases the optimization space of the model.
[0140] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.
[0141] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of computer program goods. The computer program goods include one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0142] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0143] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0145] If the functions are implemented in the form of software functional units and sold or used as independent goods, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of software goods, and the computer software goods are stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.
[0146] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A plate surface defect detection method based on machine vision, characterized in that: The specific steps include: Adaptively adjust the lighting conditions of the testing environment, collect plate images through multi-element imaging, dynamically adjust the real-time collected images using the lighting compensation algorithm, and pre-process the plate image data; A convolutional neural network model is established to detect and classify defects in plate image data, and the adaptability of the convolutional neural network model to lighting conditions is improved through multi-task learning methods. The edge computing method is used to reduce the transmission delay and calculation time of image data, and the model compression technology is used to optimize the model; Combine the performance of the convolutional neural network model and the processing efficiency of edge computing, and provide decision support based on the production status of the plate.
2. The plate surface defect detection method based on machine vision according to claim 1 is characterized in that: The process of dynamically adjusting the real-time acquired image using the illumination compensation algorithm is as follows: The collected plate image is subjected to Gaussian blur processing to generate a lighting estimation image. The original image is divided by the lighting estimation image to remove the influence of uneven lighting. The contrast and brightness of the compensated image are adjusted to enhance the details, and the processed image is output.
3. The plate surface defect detection method based on machine vision according to claim 2 is characterized in that: The specific steps to adjust the image based on the illumination estimation image generated by the Retinex algorithm are: The plate image is processed using Gaussian blur to obtain a smooth image L(x,y) representing the illumination. L(x,y) is defined to represent the image as an illumination estimation image. Divide the original image by the illumination estimation image to calculate the reflectance image R(x,y). The calculation expression of R(x,y) is: Where I(x,y) is the brightness value of the plate image, and L(x,y) is the illumination estimation value obtained by Gaussian blur; The dynamic range of the reflectance image is compressed by logarithmic transformation, and the calculation expression is R 1 (x,y)=log(1+R(x,y)); Apply the Retinex algorithm to each color channel in the image separately, and perform color restoration in combination with the color restoration factor. The calculation expression is: Where C(x,y) is the original color channel value, C 1 (x, y) is the restored color channel value, G is the global gain coefficient; Output the image processed by the Retinex algorithm.
4. The plate surface defect detection method based on machine vision according to claim 1 is characterized in that: The preprocessing of plate image data includes normalizing the image data using the grayscale normalization method, enhancing the image using the adaptive histogram equalization method, and denoising the image using the high-pass filtering method.
5. The plate surface defect detection method based on machine vision according to claim 4 is characterized in that: The method of normalizing image data using grayscale normalization method is: The grayscale value of the calibrated image pixel (x, y) is Gr(x, y), and the normalized calculation expression is Grno(x, y) = Z × log(1 + Gr(x, y)), where Grno(x, y) is the normalized pixel grayscale value and Z is the scaling constant.
6. The plate surface defect detection method based on machine vision according to claim 4 is characterized in that: The method of using adaptive histogram equalization to enhance the image is: Divide the image into several small local areas, and perform histogram equalization on each local area separately; Calculate the histogram of each local area and perform histogram equalization on the local area; The adjacent local regions are smoothed using an interpolation method, and all processed local regions are re-merged to form an enhanced image; The specific process of denoising an image using the high-pass filtering method is as follows: Use the Laplacian filter to convolve the image to highlight the edges and details of the image and suppress smooth areas; Combine the original image with the filtered result to preserve the enhanced edges and reduce the noise in other areas; To suppress and denoise high-frequency components, the image is converted to the frequency domain through a frequency domain filter for processing. The processing method is as follows: Perform Fourier transform on the image to convert it into frequency domain representation; Use a high-pass filter to apply to the frequency domain image and suppress low-frequency components; Perform inverse Fourier transform on the filtered image and return to the spatial domain to obtain the processed image.
7. The plate surface defect detection method based on machine vision according to claim 1 is characterized in that: The modeling process of the convolutional neural network model is: Based on the plate image data, including samples collected under different lighting conditions, the defect type and the lighting conditions of the image are annotated, and the training set is expanded using data augmentation technology; Use a convolutional neural network model to extract hierarchical features of the image. The layers include convolutional layer, pooling layer, ReLU activation function, and fully connected layer. A shared convolutional layer is designed at the front end of the convolutional neural network model to extract common features of the image. Common features are shared between different tasks. After the shared feature extraction layer, the defect detection branch and the illumination classification branch are designed. The defect detection branch targets different defect types and uses convolutional layers and fully connected layers for classification. The illumination classification branch is used to classify the illumination conditions of the image. Define a joint loss function to combine the defect detection loss and the illumination classification loss and optimize them simultaneously; Use existing image classification datasets to pre-train the model, then fine-tune the model on plate image data, and use learning rate decay or adaptive learning rate optimizer to promote convergence of the later model training; K-fold cross validation is used to evaluate the performance of the model, and the confusion matrix is used to evaluate the classification performance of the model under different lighting conditions. When the model performance and classification ability meet the production standards, the model is output.
8. The plate surface defect detection method based on machine vision according to claim 7 is characterized in that: The setting method of the joint loss function is: The cross entropy loss function is used for the classification task of plate defects. The classification task has Ca defect categories. For category i, the category probability predicted by the model is p i , the true label is y i , then the loss function of the classification task is expressed as For the illumination adaptation task, the cross entropy loss function is used to classify the illumination conditions. Assuming that the illumination classification task has Lig categories, the illumination category probability predicted by the model for category j is q j , the true label is z j , then the loss function of the illumination adaptation task is expressed as The expression of the joint loss function is L tot =α*L cro +β*L ill , where L tot is the joint loss function, α and β are L cro and L ill The weight coefficient of , and α and β are both positive integers; Add a regularization term to the joint loss function for L2 regularization. The calculation expression of L2 regularization is L tot =α*L cro +β*L ill +λ*R(W), where λ is the regularization coefficient and R(W) is the regularization term representing the model weight W.
9. The plate surface defect detection method based on machine vision according to claim 8, characterized in that: The logic for providing decision support is: The time for a complete inference of the calibrated neural network model in edge computing is In t , the time for image data to be transmitted from the edge device to the cloud is Tr l , the moving speed of the detection plate is v p According to the actual monitoring log data after the plate surface defect detection model based on convolutional neural network and multi-task learning is applied, the actual false alarm rate and missed alarm rate of the model are calculated, and the false alarm rate is marked as Fa r , the false negative rate is Fn r , then the balance control index of plate surface defect detection is In the formula, e is a natural constant; The calculated balance control index B cr With the preset first balance control limit B t1 , the second balance control limit B t2 Compare, the first balance control limit B t1 Less than the second balance control limit B t2 , when the balance control index B cr Greater than or equal to the second balance control limit B t2 When , the fitting of the plate surface defect detection is marked too deep, indicating that the false alarm rate and missed alarm rate of defect detection are unbalanced; When the balance control index B cr Less than or equal to the first balance control limit B t1 When , the fitting of the plate surface defect detection is marked too shallow, indicating that the false alarm rate and missed alarm rate of defect detection are unbalanced; When the first balance control limit B t1 Less than balance control index B cr Less than the second balance control limit B t2 When , the fitting of the surface defect detection of the marked plate is appropriate, indicating that the false alarm rate and missed alarm rate of defect detection are balanced.
10. The plate surface defect detection system based on machine vision is characterized by: Includes multivariate imaging module, convolution modeling module, edge computing module, and decision support module; The multivariate imaging module is used to adaptively adjust the lighting conditions of the detection environment, collect plate images through multivariate imaging, dynamically adjust the real-time collected images using the lighting compensation algorithm, and pre-process the plate image data; The convolution modeling module is used to establish a convolutional neural network model to detect and classify defects in plate image data, and to improve the adaptability of the convolutional neural network model to lighting conditions through multi-task learning methods; The edge computing module is used to reduce the transmission delay and computing time of image data through edge computing methods, and optimize the model using model compression technology; The decision support module is used to combine the performance of the convolutional neural network model and the processing efficiency of edge computing, and provide decision support based on the production status of the plate.
Citation Information
Patent Citations
Battery surface defect detection method and device based on deep learning
CN116912218A
Cable defect detection method and system based on large model
CN118115483A
Automatic bearing defect detection system and method
CN118305101A
Cited By
Wood block color detection method based on machine vision
CN119762602A
Pulse neural network image processing method and system based on frequency domain filtering module
CN120071022A
Multi-mode online detection method and system for transformer oil
CN120176768A
Color ring element appearance detection device and detection system based on machine vision
CN120177368A
Machine vision detection method and system for railway sleeper supporting fastener missing
CN120259291A
