Industrial metal nut defect detection method based on VisionMaster platform

Abstract

The invention discloses an industrial metal nut defect detection system and detection method based on machine learning, and relates to the technical field of industrial metal nut detection, and the system comprises an image detection and processing system, a model design and test system, an analysis and comparison system, and a product classification system. Wherein the image detection and processing system is used for collecting metal nut image data, processing the image data and extracting detection data; the model design and test system uses a machine learning algorithm to create a model, uses the marked data set to train the model, and tests the model; the analysis and comparison system is used for displaying and comparing the defect data and finally forming a comparison data table; and the product classification system is used for blowing the products through an air pump and classifying the products to correct areas. In one or more embodiments of the invention, a combined smooth filtering model is used for image preprocessing, proper filters and parameters are selected based on industrial metal nuts, and meanwhile, in order to reinforce a strong feature interval of a defect edge, an edge enhancement algorithm is adopted for edge detection and enhancement of the image, so that the accuracy in a model training process is improved.

Description

Technical Field

[0001] The invention relates to the technical field of defect detection, and in particular to an industrial metal four-claw nut detection method based on a VisionMaster platform. Background Art

[0002] Metal nuts are a common type of metal fastener products in industrial production processes. They are widely used in industries such as energy, electrical appliances, electronics, chemicals, and metallurgy, playing an important role in manufacturing and our daily lives. Metal nuts are mainly divided into four-claw nuts, hexagonal nuts, welding nuts, etc. As commonly used connection elements in industry, their quality and reliability are crucial to the safety and performance of products.

[0003] At present, for the defect detection of nuts in the market, most companies still use traditional detection methods. For example, the visual inspection method uses the naked eye to observe whether the surface of the nut is flat, as well as the manufacturing process and oxidation degree to judge the quality of the product; the hand feel inspection method uses the hand senses to check the surface characteristics, internal structure and shape precision of the nut. This method requires different types of nuts to be distinguished; the measuring instrument inspection method uses measuring instruments to check the accuracy and pass rate of the nuts. For example, a micrometer is used to measure the outer diameter and height of the nut; there are also optical microscope testing methods, material testing methods, and so on. These detection methods inevitably have the possibility of contact with the nut in the actual factory production inspection process, and have the disadvantages of low efficiency, cumbersome manual operation, poor recognition accuracy and flexibility. These problems will lead to pain points such as low industrial production efficiency, unreliable quality control and safety hazards, and cannot meet the use needs of various manufacturing companies.

[0004] Therefore, in response to the problems and pain points in the above traditional detection methods, we proposed an industrial metal nut detection method based on the VisionMaster software platform to solve these problems and pain points.

[0005] The metal nut defect detection system we invented has the following characteristics. It fits the industrial production line and can quickly and accurately detect nut defects in a short time. Through real-time data analysis and feedback systems, it can provide results and suggestions instantly. In addition, the product has a high degree of automation and can automatically identify and classify various nut defects to improve detection efficiency and consistency. Applied to the production, use and maintenance processes of nut-related industries, it helps to reduce labor costs, reduce nut losses, save resources, and optimize resource utilization. It can also reduce the emission of industrial waste gas and greenhouse gases such as carbon dioxide, and has a certain degree of driving effect on the construction of a beautiful China, the overall coordination of industrial structure adjustment, pollution control, ecological protection, and the coordinated promotion of carbon reduction, pollution reduction, green expansion, and growth, and the promotion of ecological priority, conservation and intensive, green and low-carbon development. Summary of the invention

[0006] Aiming at solving the problems in the traditional nut defect detection in the current market proposed in the above background, the present invention proposes an industrial metal nut defect detection system and surface detection method based on the VisionMaster platform. The detection speed can reach nearly 300psc / min and the detection accuracy is close to 99.5%.

[0007] The present invention adopts the deep learning module in VisionMaster to produce a metal nut surface defect detection system integrating optical, electrical, mechanical and machine vision technologies for metal nut surface defect detection. The nut features are tracked and detected by VisionMaster's BLOB calibration module, and then cluster analysis is performed by the image segmentation module. Finally, feature point matching and target detection and identification are performed by the high-precision matching module. The use of these modules can greatly improve the product detection speed and can be applied to large-scale production and manufacturing scenarios.

[0008] The present invention provides the following technical solutions:

[0009] A detection method for industrial metal nuts based on the VisionMaster platform, including an image detection and processing system, a model design and testing system, an analysis and comparison system, and a product classification system;

[0010] Among them, the image detection and processing system is used to collect image data of the surface of the industrial metal nut to be detected;

[0011] The model design and testing system uses deep learning algorithms to create models, trains the models using labeled data sets, and tests the models;

[0012] The analysis and comparison system is used to display and compare defect data, form test results and provide feedback;

[0013] The product sorting system is used to sort the products into the correct area by blowing air through an air pump.

[0014] In one or more embodiments of the present invention, the image detection and processing system includes an image acquisition module, an image preprocessing module, and a feature extraction module;

[0015] Among them, the image acquisition module is used to shoot the industrial metal nuts, obtain image data, and mark the industrial metal nuts;

[0016] The image preprocessing module is used to preprocess the collected image data, including image denoising, image enhancement, and image segmentation to enhance contrast, adjust color balance and brightness. All collected images are in RGB8 pixel format, and each pixel is composed of three colors, R\G\B. Each color is quantized to 256 levels when stored and represented, and each color can be stored and represented by an 8-bit binary number.

[0017] The feature extraction module is used to extract features from preprocessed images using computer vision technology.

[0018] In one or more embodiments of the present invention, the model design and testing system includes a model training module and a model testing module;

[0019] Among them, the model training module is used to train data through deep learning algorithms to establish a defect detection model;

[0020] The model testing module is used to input the collected images into the trained model for defect detection to determine whether there are defects on the nut surface.

[0021] In one or more embodiments of the present invention, the analysis and comparison system includes a defect detection and classification module, a defect comparison module, a communication module, and a display result module;

[0022] Among them, the defect detection and classification module is used to extract features of the shape, size, and location of the detected defects; The defect comparison module is used to input the detected defects into the trained model again to optimize the model;

[0023] The communication module is used to report the defective industrial metal nuts during the detection process and upload the signal to the PLC controller;

[0024] The display result module is used to output the detection and recognition results to the display end, and finally form the detection results and feedback.

[0025] In one or more embodiments of the present invention, the product classification system includes a plc control module and an air pump blowing execution module;

[0026] Among them, the plc control module is used for when the image segmentation and BLOB calibration do not find obvious defects and the high-precision matching is positive, the conditional detection code returns a signal 1, thereby transmitting the signal to the plc controller; when the image segmentation and BLOB calibration find obvious defects or the high-precision matching is wrong, the conditional detection code returns a logical value of 0;

[0027] The air pump blowing module is used to perform corresponding operations on the logical values ​​returned by the condition detection code, so as to blow the product to the corresponding area;

[0028] An industrial metal nut detection method based on the VisionMaster platform includes the following steps:

[0029] S1. Photographing industrial metal nuts using a high-resolution camera;

[0030] S2, using a combined smoothing filter model to preprocess the extracted nut images, and using an edge enhancement algorithm to detect the edges of the images;

[0031] S201, firstly grayscale the image of the industrial metal nut to convert the color image into a grayscale image;

[0032] S202, using a Gaussian filter to smooth the grayscale image and remove major high-frequency noise;

[0033] S203, using a median filter to smooth the smoothed image to remove major low-frequency noise;

[0034] S204, performing weighted averaging on the outputs of the Gaussian filter and the median filter to obtain a final filtering result;

[0035] S3. Integrate the trained model into the industrial metal nut detection system and use the R-CNN deep learning model for image segmentation based on region selection;

[0036] S301, extracting candidate boxes using the selective search algorithm;

[0037] S302, performing serial feature extraction on the candidate boxes through a convolutional network;

[0038] S303, using SVM to classify and predict the candidate boxes;

[0039] S304, using a regression method to correct the region frame and obtain a segmented image;

[0040] S305, comparing the segmented image with the trained model to detect whether there are defects;

[0041] S4. Use the BLOB calibration module to extract features and visualize the image after image segmentation to detect whether it is a real defect;

[0042] S401, further eliminating and reducing the interference of noise on the target;

[0043] S402, extracting color, texture, and shape features from each region, and performing image enhancement, normalization, and cropping;

[0044] S403, describing the topological relationship between the targets;

[0045] S404, calculating the 2-D shape features of the target;

[0046] S405, comparing the feature with the set defect threshold to detect whether it is a real defect;

[0047] S6. Use the PATMAX high-precision matching algorithm to detect the industrial metal nuts one by one and determine that they are positive;

[0048] S601, searching for an area by using prior knowledge and ROI;

[0049] S602, in the search area, using a sub-pixel matching algorithm to perform pattern matching, compare the shape descriptor with the local image in the search area, and calculate a similarity index;

[0050] S603, determining that it is a front face according to a similarity index;

[0051] S7, use PLC control to blow industrial metal nuts into qualified and defective areas;

[0052] S701, condition detection performs AND processing on the logical values ​​transmitted by BLOB calibration and high-precision matching. When both are 1, The condition detection module transmits the logic value 1 to the PLC controller, which controls the blowing to the qualified product area;

[0053] S702, when the condition detection module transmits the logic value 0 to the PLC controller, the control blows out to the defective area; S8. During the detection process, the detected defective industrial metal nuts are transmitted to the PLC controller through the communication module, and finally the integrated industrial metal defect quantity results are output to the display end to form a comparison data table.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] This machine vision nut defect detection method uses advanced non-contact technology to detect defects without physical contact with the nut, eliminating interference factors that may be introduced in traditional detection methods, improving detection efficiency and protecting the integrity of the nut.

[0057] The Siemens S7-1200 PLC control circuit is used to provide reliable system control for automation;

[0058] Visual inspection technology has high accuracy and reliability in image processing and analysis. Using advanced image processing algorithms and machine learning methods, it can accurately identify and analyze defects on the surface of nuts and provide reliable inspection results.

[0059] No human intervention is required, and real-time data analysis and feedback can be achieved with the customized host computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a module schematic diagram of the industrial metal nut defect detection system based on the VisionMaster platform of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1 ,The present invention provides an industrial metal nut detection method based on the VisionMaster platform, including an image detection and processing system, a model design and testing system, an analysis and comparison system, and a product classification system;

[0063] Among them, the image detection and processing system is used to collect industrial metal nut image data, process the image data, and extract detection data;

[0064] The model design and testing system uses machine learning algorithms to create models, trains the models using labeled data sets, and tests the models;

[0065] The analysis and comparison system is used to display and compare defect data, form test results and provide feedback;

[0066] The product sorting system is used to sort the products into the correct area by blowing air through an air pump.

[0067] Specifically, the image detection and processing system includes an image acquisition module, an image preprocessing module, and a feature extraction module;

[0068] Among them, the image acquisition module is used to shoot the industrial metal nuts, obtain image data, and mark the industrial metal nuts;

[0069] The image preprocessing module is used to preprocess the collected image data, including image denoising, image enhancement, and image segmentation;

[0070] The feature extraction module is used to extract features from the preprocessed image using computer vision technology;

[0071] Among them, in the existing manufacturing workshop, industrial metal nuts are photographed by using high-resolution cameras, including defective and normal industrial metal nuts;

[0072] Specifically, the model design and testing system includes a model training module and a model testing model;

[0073] Among them, the model training module is used to train data through deep learning algorithms to establish a defect detection model;

[0074] The model testing module is used to input the collected images into the trained model to perform defect detection and determine whether there are defects on the surface of industrial metal nuts;

[0075] Specifically, the analysis and comparison system includes a defect detection and classification module, a defect comparison module, a communication module and a display result module;

[0076] Among them, the defect detection and classification module is used to extract features of the shape, size, and location of the detected defects;

[0077] The defect comparison module is used to input the detected defects into the trained model again to optimize the model;

[0078] The communication module is used to report the defective industrial metal nuts during the detection process and upload the signal to the PLC controller;

[0079] The display result module is used to output the detection and recognition results to the display end, and finally form the detection results and feedback.

[0080] A detection method for an industrial metal nut defect detection system based on deep learning, comprising the following steps:

[0081] S1. Photographing industrial metal nuts using a high-resolution camera;

[0082] S2, using a combined smoothing filter model to preprocess the extracted nut images, and using an edge enhancement algorithm to detect the edges of the images;

[0083] S201, firstly grayscale the image of the industrial metal nut to convert the color image into a grayscale image;

[0084] S202, using a Gaussian filter to smooth the grayscale image and remove major high-frequency noise;

[0085] S203, using a median filter to smooth the smoothed image to remove major low-frequency noise;

[0086] S204, performing weighted averaging on the outputs of the Gaussian filter and the median filter to obtain a final filtering result;

[0087] S3. Integrate the trained model into the industrial metal nut detection system and use the R-CNN deep learning model for image segmentation based on region selection;

[0088] S301, extracting candidate frames from the processed image;

[0089] S302, performing serial feature extraction on the candidate boxes;

[0090] S303, classify and predict the candidate boxes;

[0091] S304, using a regression method to correct the region frame and obtain a segmented image;

[0092] S305, comparing the segmented image with the trained model to detect whether there are defects;

[0093] S4. Use the BLOB calibration module to extract features and visualize the image after image segmentation to detect whether it is a real defect;

[0094] S401, further eliminating and reducing the interference of noise on the target;

[0095] S402, extracting color, texture, and shape features from each region, and performing image enhancement, normalization, and cropping;

[0096] S403, describing the topological relationship between the targets;

[0097] S404, calculating the 2-D shape features of the target;

[0098] S405, comparing the feature with the set defect threshold to detect whether it is a real defect;

[0099] S6. Use the PATMAX high-precision matching algorithm to detect the industrial metal nuts one by one and determine that they are positive;

[0100] S601, searching for an area by using prior knowledge and ROI;

[0101] S602, in the search area, using a sub-pixel matching algorithm to perform pattern matching, compare the shape descriptor with the local image in the search area, and calculate a similarity index;

[0102] S603, determining that it is a front face according to a similarity index;

[0103] S7, use PLC control to blow industrial metal nuts into qualified and defective areas;

[0104] S701, condition detection performs AND processing on the logical values ​​transmitted by BLOB calibration and high-precision matching. When both are 1, The condition detection module transmits the logic value 1 to the PLC controller, which controls the blowing to the qualified product area;

[0105] S702, when the condition detection module transmits the logic value 0 to the PLC controller, the control blows out to the defective area; S8. During the detection process, the detected defective industrial metal nuts are transmitted to the PLC controller through the communication module, and finally the integrated industrial metal defect quantity results are output to the display end to form a comparison data table.

[0107] Among them, in step S3, extracting candidate frames: using the selective search algorithm to extract candidate frames from the processed image, and selecting a partial area of ​​the picture with characteristics;

[0108] Serial feature extraction: Use the convolutional neural network model to extract the features of the candidate box serial;

[0109] #Convolutional neural network model python code:

[0110] importtorch

[0111] importtorch.nnasnn

[0112] classRCNN(nn.Module):

[0113] def_init_(self):

[0114] super(RCNN,self)._init_()

[0115] #Define convolutional layer

[0116] self.conv1=nn.Conv2d(in_channels,out_channels,kernel_size)

[0117] self.relu1 = nn.ReLU()

[0118] self.pool1=nn.MaxPool2d(kernel_size)

[0119] self.conv2=nn.Conv2d(in_channels,out_channels,kernel_size)

[0120] self.relu2 = nn.ReLU()

[0121] self.pool2=nn.MaxPool2d(kernel_size)

[0122] #Define the fully connected layer

[0123] self.fc1=nn.Linear(in_features,hidden_units)

[0124] self.relu3 = nn.ReLU()

[0125] self.fc2=nn.Linear(hidden_units,num_classes)

[0126] defforward(self,x):

[0127] x = self.conv1(x)

[0128] x = self.relu1(x)

[0129] x = self.pool1(x)

[0130] x = self.conv2(x)

[0131] x = self.relu2(x)

[0132] x = self.pool2(x)

[0133] x=x.view(x.size(0),-1)#Flatten the features into a one-dimensional vector

[0134] x = self.fc1(x)

[0135] x = self.relu3(x)

[0136] x = self.fc2(x)

[0137] returnx

[0138] Classification prediction: Use the SVM supervised learning algorithm, use the training samples as support vectors in high-dimensional space, and build decision boundaries based on the support vectors to achieve classification prediction of candidate boxes;

[0139] Color extraction: Use the color histogram algorithm to extract the color distribution of each area in the industrial metal nut image to obtain the main color and color distribution of each area;

[0140] Texture extraction: Use the local binary pattern algorithm to extract the texture features of each area in the industrial metal nut image to obtain the texture features of each area;

[0141] Shape extraction: Use edge enhancement algorithm to extract the edge of each area in the industrial metal nut image to obtain the shape features of each area;

[0142] Feature extraction: Use the gray-level co-occurrence matrix algorithm to extract the gray-level distribution features of each area in the industrial metal nut image to obtain the features of each area;

[0143] Image enhancement: Use the histogram equalization algorithm to enhance the industrial metal nut image to improve the contrast and clarity of the image;

[0144] Normalization: Use the normalization algorithm to normalize the feature values ​​of each area in the industrial metal nut image so that the feature values ​​of different areas are comparable;

[0145] Cropping: Use image segmentation algorithm to segment each area in the industrial metal nut image to obtain the image of each area.

[0146] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0147] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0148] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An industrial metal nut detection method based on the VisionMaster platform, characterized in that: Including image detection and processing system, model design and testing system, analysis and comparison system, product classification system; Among them, the image detection and processing system is used to collect industrial metal nut image data, process the image data, and extract detection data; The model design and testing system uses machine learning algorithms to create models, trains the models using labeled data sets, and tests the models; The analysis and comparison system is used to display and compare defect data, and finally form a comparison data table; The product sorting system is used to sort the products into the correct area by blowing air through an air pump.

2. The industrial metal nut detection method based on the VisionMaster platform according to claim 1 is characterized in that: The image detection and processing system includes an image acquisition module, an image preprocessing module, and a feature extraction module; Among them, the image acquisition module is used to shoot the industrial metal nuts, obtain image data, and mark the industrial metal nuts; The image preprocessing module is used to preprocess the collected image data, including image denoising, image enhancement, and image segmentation to enhance contrast, adjust color balance and brightness. All collected images are in RGB8 pixel format, and each pixel is composed of three colors, R\G\B. Each color is quantized to 256 levels when stored and represented, and each color can be stored and represented by an 8-bit binary number; The feature extraction module is used to extract features from preprocessed images using computer vision technology.

3. The industrial metal nut detection method based on the VisionMaster platform according to claim 1 is characterized in that: The model design and testing system includes a model training module and a model testing module; Among them, the model training module is used to train data through deep learning algorithms to establish a defect detection model; The model testing module is used to input the collected images into the trained model for defect detection to determine whether there are defects on the nut surface.

4. The industrial metal nut detection method based on the VisionMaster platform according to claim 1 is characterized in that: The analysis and comparison system includes a defect detection and classification module, a defect comparison module, a communication module and a display result module; Among them, the defect detection and classification module is used to extract features of the shape, size, and location of the detected defects; The communication module is used to report the defective industrial metal nuts during the detection process and upload the signal to the PLC controller; The display result module is used to output the detection and recognition results to the display end, and finally form the detection results and feedback.

5. An industrial metal nut detection method based on the VisionMaster platform, characterized in that: The product classification system includes a plc control module and an air pump blowing execution module; Among them, the plc control module is used for when the image segmentation and BLOB calibration do not find obvious defects and the high-precision matching is positive, the conditional detection code returns a signal 1, thereby transmitting the signal to the plc controller; when the image segmentation and BLOB calibration find obvious defects or the high-precision matching is wrong, the conditional detection code returns a logical value of 0; The air pump blowing module is used to perform corresponding operations on the logical values ​​returned by the condition detection code, thereby blowing the product to the corresponding area.

6. An industrial metal nut detection method based on the VisionMaster platform, characterized in that: The following steps are involved: S1. Photographing industrial metal nuts using a high-resolution camera; S2, using a combined smoothing filter model to preprocess the extracted nut images, and using an edge enhancement algorithm to detect the edges of the images; S201, firstly grayscale the image of the industrial metal nut to convert the color image into a grayscale image; S202, using a Gaussian filter to smooth the grayscale image and remove major high-frequency noise; S203, using a median filter to smooth the smoothed image to remove major low-frequency noise; S204, performing weighted averaging on the outputs of the Gaussian filter and the median filter to obtain a final filtering result; S3. Integrate the trained model into the industrial metal nut detection system and use the R-CNN deep learning model for image segmentation based on region selection; S301, extracting candidate boxes using the selective search algorithm; S302, performing serial feature extraction on the candidate boxes through a convolutional network; S303, using SVM to classify and predict the candidate boxes; S304, using a regression method to correct the region frame and obtain a segmented image; S305, comparing the segmented image with the trained model to detect whether there are defects; S4. Use the BLOB calibration module to extract features and visualize the image after image segmentation to detect whether it is a real defect; S401, further eliminating and reducing the interference of noise on the target; S402, extracting color, texture, and shape features from each region, and performing image enhancement, normalization, and cropping; S403, describing the topological relationship between the targets; S404, calculating the 2-D shape features of the target; S405, comparing the feature with the set defect threshold to detect whether it is a real defect; S6. Use the PATMAX high-precision matching algorithm to detect the industrial metal nuts one by one and determine that they are positive; S601, searching for an area by using prior knowledge and ROI; S602, in the search area, using a sub-pixel matching algorithm to perform pattern matching, compare the shape descriptor with the local image in the search area, and calculate a similarity index; S603, determining that it is a front face according to a similarity index; S7, use PLC control to blow industrial metal nuts into qualified and defective areas; S701, condition detection performs AND processing on the logic values ​​transmitted by BLOB calibration and high-precision matching. When both are 1, the condition detection module transmits the logic value 1 to the PLC controller to control the blowing to the qualified product area; S702, when the condition detection module transmits the logic value 0 to the PLC controller, the control blows out to the defective area; S8. During the detection process, the detected defective industrial metal nuts are transmitted to the PLC controller through the communication module, and finally the integrated industrial metal defect quantity results are output to the display end to form a comparison data table.

Images