A glass bottle defect recognition method and system

Abstract

The application relates to the technical field of image processing, and particularly discloses a glass bottle defect recognition method and system, which is characterized in that the mouth, bottom and body of a glass bottle are photographed and imaged to obtain corresponding three views for preprocessing, so as to weaken and inhibit the noise of the collected three views and obtain a preprocessed image; the preprocessed image of a single channel is further fused, the pictures in three directions are subjected to feature extraction, and the pictures are fused into a three-channel feature map for defect detection according to different weights, so that the defect detection precision can be improved, and the detection task can be completed at a relatively fast speed; then, the calculation capacity of a convolutional neural network is utilized to collect and process image features of different degrees, and the features are subjected to convolution, pooling and feature fusion of different degrees, and then convolution up-sampling and the like, so that a target feature image is obtained, the image features can be fully extracted, and the defect recognition precision can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a glass bottle defect recognition method and system. BACKGROUND

[0002] In the glass bottle manufacturing process, quality inspection is a crucial but challenging task. Given the current market situation, the domestic glass bottle market share is high, so there is a huge demand for glass bottle detection. However, in terms of glass bottle quality detection, manual methods still dominate. Using industrial cameras and appropriate light sources and other hardware, the images of the glass bottle are captured from multiple angles to ensure that all parts of the bottle are captured under sufficient lighting conditions. Then, image processing algorithms are used to analyze and filter these images to achieve machine vision detection. This method has been widely used around the world, and compared to traditional manual or semi-manual detection methods, machine vision detection has many advantages such as higher efficiency, greater stability, and faster detection speed.

[0003] It is worth mentioning that machine vision detection completely replaces manual operation and completely avoids direct contact between operators and glass bottles and related equipment. This advantage has made machine vision detection widely recognized in the detection industry, especially in glass bottle detection, which has become a highly respected technology. However, the current glass bottle defect recognition technology uses a complex recognition network, such as the glass bottle mouth defect detection method, device, equipment and storage medium disclosed in application number 202211234402.9, which uses a multi-scale multi-attention convolutional neural network MSMA-CNN module for defect detection. The network is complex and the recognition accuracy is still not high enough. SUMMARY

[0004] The present application provides a glass bottle defect recognition method and system, which solves the technical problem of how to achieve high-precision defect recognition of glass bottles with a relatively simple neural network.

[0005] To solve the above technical problems, the present application provides a glass bottle defect recognition method, comprising the steps of:

[0006] S1, photographing the mouth, bottom and body of the glass bottle to obtain the corresponding mouth view, bottom view and body view;

[0007] S2, preprocessing the mouth view, bottom view and body view to obtain the preprocessed mouth view, preprocessed bottom view and preprocessed body view;

[0008] S3, fusing the preprocessed mouth view, preprocessed bottom view and preprocessed body view to obtain a fused image;

[0009] S4, input the fusion image into the trained defect recognition network to perform defect position positioning and output a defect recognition result image.

[0010] Further, in the step S4, the defect recognition network performs defect position positioning and outputs a defect recognition result image, specifically including steps of:

[0011] S41, performing feature extraction on the fusion image to generate a target feature image;

[0012] S42, performing edge detection on the feature image to obtain an edge detection image;

[0013] S43, performing defect position positioning on the edge detection image to obtain a defect recognition result image.

[0014] Further, the step S41 specifically includes steps of:

[0015] S411, inputting the fusion image into a convolutional neural network to generate a first feature map;

[0016] S412, inputting the first feature map into a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a maximum pooling layer respectively to obtain a second feature map, a third feature map, a fourth feature map, a fifth feature map and a sixth feature map and perform fusion and splicing to obtain a seventh feature map; performing convolution processing on the first feature map with a convolution kernel of 1x1 to obtain an eighth feature map;

[0017] S413, performing convolution processing on the seventh feature map with a convolution kernel of 1x1, then performing upsampling and fusing and splicing with the eighth feature map to obtain a ninth feature map;

[0018] S414, performing convolution processing on the ninth feature map with a convolution kernel of 3x3, then performing upsampling to obtain a target feature image.

[0019] Further, in the step S412:

[0020] The first convolutional layer performs dilated convolution processing on the input first feature map with a convolution kernel of 1x1 and an expansion coefficient r=1 to obtain the second feature map;

[0021] The second convolutional layer performs dilated convolution processing on the input first feature map with a convolution kernel of 3x3 and an expansion coefficient r=6 to obtain the third feature map;

[0022] The third convolutional layer performs dilated convolution processing on the input first feature map with a convolution kernel of 3x3 and an expansion coefficient r=12 to obtain the fourth feature map;

[0023] The fourth convolutional layer performs a convolution kernel of 3*3 and a dilated convolution processing with a dilated coefficient of r=18 on the input first feature map to obtain the fifth feature map.

[0024] Further, the step S42 comprises steps of:

[0025] S421, calculating the amplitude and direction of the gradient of each point of the target feature image by using the finite difference of the first-order partial derivative;

[0026] S422, performing a detection scan on the target feature image to determine whether the amplitude of the point on the edge is the maximum amplitude in the same direction, if yes, the point is reserved, if not, the point is deleted;

[0027] S423, determining the point with a pixel value greater than a threshold T2 among the reserved points as a boundary point, and discarding the point with a pixel value less than a threshold T1, T1

[0028] S424, performing Canny edge detection on the target feature image after the image enhancement processing of steps S421-S423 to obtain an edge detection image, the edge detection image including an outer contour and an inner contour.

[0029] Further, in the step S421, the amplitude and the direction of the target feature image are calculated according to the following formulas:

[0030]

[0031]

[0032] are functions after the operation of the target feature image and two first-order difference convolution templates;

[0033] The step S43 specifically comprises steps of:

[0034] S431, reserving the pixels in the detection region between the outer contour and the inner contour of the edge detection image, and discarding the pixels outside the outer contour and inside the inner contour;

[0035] S432, performing threshold segmentation on the detection region to make the pixel value of the defect be 255 and the pixel value of the rest be 0 to obtain the connected region of the defect part;

[0036] S433、According to the size of each connected region, it is determined whether it is a prescribed defect, if yes, the connected region is reserved and it is determined that the glass bottle is a defective glass bottle, if not, the pixel value of the pixel point of the connected region is set to 0.

[0037] Further, in the training process, the loss function of the defect recognition network is designed as:

[0038] Loss = 0.4 * Loss obj + 0.3 * Loss rect + 0.3 * Loss cls

[0039] wherein, Loss obj represents the confidence loss, Loss rect represents the rectangular frame loss, and Loss cls represents the classification loss, which is calculated by the following formula respectively:

[0040]

[0041]

[0042]

[0043] wherein, p o represents the target confidence score in the prediction frame, p iou represents the IOU value of the prediction frame and the target frame corresponding thereto, w obj represents the weight of the positive sample when calculating , represents the binary cross-entropy loss between p o and p iou ; IOU represents the overlapping ratio of the prediction frame and the actual frame, b represents the center point of the prediction frame, b gt represents the center point of the actual frame, c represents the diagonal length of the smallest rectangle containing the prediction frame and the actual frame, w represents the width of the prediction frame, w gt represents the width of the actual frame, C w represents the width of the smallest rectangle of the prediction frame and the actual frame, h represents the height of the prediction frame, h gt represents the height of the actual frame, C h represents the width of the smallest rectangle of the prediction frame and the actual frame, and ρ() represents the Euclidean distance; c p represents the prediction category, c gt represents the true category, w cls represents the weight of the positive sample when calculating , represents the binary cross-entropy loss between c p and c gt .

[0044] Further, the step S2 performs preprocessing, specifically comprising steps of:

[0045] S21, performing Gaussian filtering on the bottle mouth view, the bottle bottom view and the bottle body view to obtain respective first intermediate images;

[0046] S22, performing median filtering on the first intermediate images to obtain respective second intermediate images;

[0047] S23, performing gray value transformation on the second intermediate images to obtain respective third intermediate images;

[0048] S24, performing sharpening on the third intermediate images to obtain respective fourth intermediate images;

[0049] S25, performing frequency domain filtering on the fourth intermediate images to obtain the preprocessed bottle mouth view, the preprocessed bottle bottom view and the preprocessed bottle body view;

[0050] In the step S23, the formula of the gray value transformation is:

[0051]

[0052] wherein g2(x, y) is the second intermediate image, the range of the original pixel point gray value is [a, b], the gray value range to be reached after image enhancement is [c, d], and the output third intermediate image is g3(x, y);

[0053] In the step S24, the gradient value is taken as the sharpening output, wherein the modulus point of the gradient vector of the point (x, y) in the third intermediate image g3(x, y) is represented as the fourth intermediate image g4(x, y):

[0054] g4(x, y) = |Δ x g3(x, y) | + |Δ y g3(x, y) |

[0055] Δ x g3(x, y), Δ y g3(x, y) respectively represent the first-order difference in the x direction and the first-order difference in the y direction of the point (x, y) in the third intermediate image g3(x, y), and are represented as:

[0056]

[0057] Further, the step S3 specifically comprises steps of:

[0058] S31, a single-channel pre-processed bottle mouth view, a pre-processed bottle bottom view and a pre-processed bottle body view are subjected to a shared convolution operation, the convolution kernel size is 3*3, and then a ReLU function is used for activation, the padding mode is same, feature extraction is carried out, and three view feature maps are obtained;

[0059] S32, the three view feature maps are spliced by using bottle body 0.5, bottle mouth 0.3 and bottle bottom 0.2 feature weights to form a fusion feature;

[0060] S33, the fusion feature is subjected to a convolution operation again, the convolution kernel size is 1*1, the channel number of the output image is adjusted to 3, and then a linear activation function is used, and finally a 3-channel fusion image with the same size as the input image of step S31 is output.

[0061] The application also provides a glass bottle defect recognition system, and the key lies in that the system comprises an image acquisition module, an image preprocessing module and a defect recognition module; the image acquisition module, the image preprocessing module, the image fusion module and the defect recognition module are respectively used for executing steps S1, S2, S3 and S4 in the above method.

[0062] The image acquisition module comprises a mechanical arm, a monocular camera and two mirrors, the monocular camera is arranged in front of the glass bottle on a detection work station, and the two mirrors are symmetrically arranged at the left rear and right rear of the glass bottle; the mechanical arm is used for:

[0063] placing the glass bottle vertically on the detection work station, at this time, the monocular camera shoots the bottle body of the glass bottle to obtain a bottle body image;

[0064] clamping the middle part of the bottle body of the vertically placed glass bottle to rotate, so that the bottle mouth of the glass bottle faces the monocular camera, at this time, the camera shoots the bottle mouth of the glass bottle to obtain a bottle mouth image;

[0065] clamping the middle part of the bottle body of the vertically placed glass bottle to rotate, so that the bottle bottom of the glass bottle faces the monocular camera, at this time, the camera shoots the bottle bottom of the glass bottle to obtain a bottle bottom image;

[0066] clamping the middle part of the bottle body of the glass bottle to take the glass bottle away from the detection work station.

[0067] This invention provides a method and system for identifying defects in glass bottles. The method involves capturing images of the bottle mouth, bottom, and body to obtain three-view diagrams. These diagrams are then preprocessed to reduce and suppress noise, resulting in preprocessed images. Further, the preprocessed images from each channel are fused, extracting features from the three directions and fusing them into a three-channel feature map for defect detection. This improves detection accuracy and allows for faster detection. Then, the computational power of a convolutional neural network is utilized to collect and process image features at different levels. Different levels of convolution and pooling are then performed for feature fusion, followed by convolutional upsampling to obtain a target feature image. This process fully extracts image features, further enhancing defect identification accuracy. Finally, edge detection and defect location are performed on the target feature image to identify whether the captured image contains defects and to pinpoint their location, achieving high accuracy. This invention employs a convolutional neural network and various convolutional layers, upsampling, edge detection, and defect detection techniques to construct a defect identification network. Because image preprocessing is also performed, the network structure is simple, resulting in high accuracy and fast detection speed. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of a glass bottle defect identification method provided in an embodiment of the present invention;

[0069] Figure 2 This is a schematic diagram of the acquisition of three views of a glass bottle provided in an embodiment of the present invention;

[0070] Figure 3 This is a diagram showing the relative positions of the glass bottle, the monocular camera, and the reflector provided in an embodiment of the present invention.

[0071] Figure 4 This is a schematic diagram of adding a light source to the bottle mouth according to an embodiment of the present invention;

[0072] Figure 5 This is a schematic diagram of the three views of the glass bottle collected according to an embodiment of the present invention. Detailed Implementation

[0073] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are given for illustrative purposes only and should not be construed as limiting the present invention. The accompanying drawings are for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0074] This invention provides a method for identifying defects in glass bottles, such as... Figure 1 The flowchart shown includes the following steps:

[0075] S1, photographing the bottle mouth, bottle bottom and bottle body of the glass bottle to obtain corresponding bottle mouth view, bottle bottom view and bottle body view;

[0076] S2, preprocessing the bottle mouth view, bottle bottom view and bottle body view to obtain preprocessed bottle mouth view, preprocessed bottle bottom view and preprocessed bottle body view;

[0077] S3, fusing the preprocessed bottle mouth view, preprocessed bottle bottom view and preprocessed bottle body view to obtain a fused image;

[0078] S4, inputting the fused image into the trained defect recognition network to locate the defect position and output a defect recognition result image.

[0079] (1) for step S1

[0080] As shown in Figure 2 , the present example adopts a monocular camera to first collect bottle mouth data, then rotates the bottle body to collect bottle bottom data, and then rotates the bottle body to collect bottle body data.

[0081] In order to successfully collect three views of a glass bottle, especially the bottle body, the present application further provides two mirrors located on both sides of the glass bottle, and the positional relationship among the monocular camera (camera), glass bottle and mirror is as shown in Figure 3 . Figure 3 (a), (b) and (c) of are respectively a top view, a front view and a side view (left or right) of the monocular camera (camera), glass bottle and mirror when the glass bottle is located on the detection station.

[0082] Figure 4 In addition, the present example adds a light source at the bottle mouth part when collecting images. The bottle mouth part will become bright due to the light, while the defect part will become dark due to the principle of light propagation. The light source selects a voltage 0V-20V adjustable annular shadowless incandescent lamp as the illumination light source, which can control the strength of the light source by adjusting the size of the voltage to facilitate the irradiation of objects with different positions and different materials. The specific construction method is as shown in .

[0083] Finally, the corresponding bottle mouth, bottle bottom and bottle body three views are as shown in Figure 5 (a), (b) and (c) respectively.

[0084] (2) for step S2

[0085] This step further preprocesses the three views, which specifically includes the following steps:

[0086] S21, Gaussian filtering is performed on the bottle mouth view, bottle bottom view and bottle body view to obtain respective first intermediate images;

[0087] S22, median filtering is performed on the first intermediate image to obtain respective second intermediate images;

[0088] S23, gray value transformation is performed on the second intermediate image to obtain respective third intermediate images;

[0089] S24, sharpening is performed on the third intermediate image to obtain respective fourth intermediate images;

[0090] S25, frequency domain filtering is performed on the fourth intermediate image to obtain a preprocessed bottle mouth view, a preprocessed bottle bottom view and a preprocessed bottle body view.

[0091] In step S21, Gaussian filtering is performed on the fused image, and the formula is as follows:

[0092]

[0093] Wherein, g1(x,y) represents the output first intermediate image, (x,y) represents the input image coordinates, a smoothing window with a size of z×z is set, a convolution kernel δ conforming to Gaussian distribution is pre-set for the window, the convolution kernel is traversed for each pixel in the image, the weighted average value calculated by convolution is substituted for the center pixel of the convolution kernel, Gaussian smoothing filtering is completed, and the purpose of reducing image noise is achieved.

[0094] In step S22, median filtering is performed on the first intermediate image, and the formula is as follows:

[0095] g2(x,y)=median(g1(x-k,y-l),(k,l∈W)

[0096] g2(x,y) represents the second intermediate image obtained after median filtering, W is a two-dimensional template, and the size is usually 5×5 or 3×3, k and l are horizontal and vertical coordinates of pixel points in the two-dimensional template, and median() represents a median filtering function. The process of median filtering is that the image pixels are sorted according to the size of the pixel value by using W, a monotonically increasing or decreasing two-dimensional data sequence is formed, and the smoothing processing is completed until the sliding window traverses the image.

[0097] In step S23, gray value transformation is performed on the second intermediate image, and the formula is as follows:

[0098]

[0099] Wherein, g2(x,y) is the second intermediate image, the range of the original pixel gray value is [a,b], the gray value range to be reached after image enhancement is [c,d], and the output third intermediate image is g3(x,y). The formula is the transformation relationship of the gray value when the transformation function is a linear function, and the gray value change can make the gray level feature of the image more obvious.

[0100] In step S24, the sharpening is performed by an image sharpening algorithm to highlight the boundary pixels of the target in the image and the details that are desired to be highlighted, and to weaken the phenomenon of image blurring. In the frequency domain, the sharpening effect is achieved by filtering techniques, and in the spatial domain, the differential processing needs to be performed on the pixel values in the image, and the formula is as follows:

[0101]

[0102] The first-order difference in the x direction and the first-order difference in the y direction of the point (x, y) in the third intermediate image g3(x, y) are replaced by the difference equation in the above formula, and g3(x, y) can be expressed as: x g3(x, y) and Δ y g3(x, y) can be expressed as:

[0103]

[0104] The modulus of the gradient vector at the point (x, y) is the fourth intermediate image g4(x, y) and can be expressed as:

[0105] g4(x, y) = |Δ x g3(x, y) | + |Δ y g3(x, y) |

[0106] The gradient value is taken as the sharpening output, and the output image only shows the edge profile with relatively strong gray value changes.

[0107] In step S25, the image is subjected to frequency domain filtering, and the pixel value discrete function in the two-dimensional space of the image is subjected to Fourier transform, so that the image can be analyzed in the frequency domain, the spectrum is processed and analyzed to change the frequency characteristics of the image, and the specific process is as follows:

[0108] The trigonometric Fourier series expansion of a function f(t) with a period T is:

[0109]

[0110] wherein, ω represents the angular frequency, a k and b k are Fourier coefficients, x is the independent variable, n is an integer greater than zero, and a0 is the direct current component.

[0111] After the above image preprocessing, the noise of the fused image can be weakened and suppressed, and the image can be enhanced to highlight the boundary pixels of the target in the image and the details that are desired to be highlighted, and to weaken the phenomenon of image blurring, so as to facilitate the convolutional neural network to quickly extract the required features.

[0112] (3) For step S3

[0113] Step S3 specifically comprises steps of:

[0114] S31, a convolution operation shared by the preprocessed bottle mouth view, the preprocessed bottle bottom view and the preprocessed bottle body view of 256x256x1 is adopted, the convolution kernel size is 3x3, and then a ReLU function is used for activation, the padding mode is same, feature extraction is performed, and three view feature maps are obtained; 256x256 represents the length and width of the image, and 1 represents the number of image channels;

[0115] S32, the three view feature maps are spliced to form a fusion feature by using the feature weights of 0.5 for the bottle body, 0.3 for the bottle mouth and 0.2 for the bottle bottom;

[0116] S33, the fusion feature is further subjected to a convolution operation, the convolution kernel size is 1x1, the number of channels of the output image is adjusted to 3, and then a linear activation function is used, and finally a fusion image of 256x256x3 with the same size as the input image of step S31 is output.

[0117] (4) for step S4

[0118] Reference Figure 1 , step S4 specifically comprises steps of:

[0119] S41, feature extraction is performed on the processed image to generate a target feature image;

[0120] S42, edge detection is performed on the feature image to obtain an edge detection image;

[0121] S43, defect position positioning is performed on the edge detection image to obtain a defect recognition result image.

[0122] Step S41 specifically comprises steps of:

[0123] S411, input the processed image into a convolutional neural network to generate a first feature map;

[0124] S412, input the first feature map into a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a maximum pooling layer respectively to obtain a second feature map, a third feature map, a fourth feature map, a fifth feature map and a sixth feature map, and fuse and splice them to obtain a seventh feature map; the first feature map is subjected to convolution processing with a convolution kernel of 1x1 to obtain an eighth feature map;

[0125] S413, the seventh feature map is subjected to convolution processing with a convolution kernel of 1x1, and then fused and spliced with the eighth feature map after upsampling to obtain a ninth feature map;

[0126] S414, the ninth feature map is subjected to convolution processing with a convolution kernel of 3x3, and then upsampling is performed to obtain a target feature image.

[0127] In step S412:

[0128] The first convolutional layer performs a dilated convolution operation on the input first feature map with a convolution kernel of 1x1 and an expansion coefficient r=1 to obtain a second feature map.

[0129] The second convolutional layer performs a dilated convolution operation on the input first feature map with a convolution kernel of 3x3 and an expansion coefficient r=6 to obtain a third feature map.

[0130] The third convolutional layer performs a dilated convolution operation on the input first feature map with a convolution kernel of 3x3 and an expansion coefficient r=12 to obtain a fourth feature map.

[0131] The fourth convolutional layer performs a dilated convolution operation on the input first feature map with a convolution kernel of 3x3 and an expansion coefficient r=18 to obtain a fifth feature map.

[0132] Step S42 includes steps:

[0133] S421, calculate the amplitude and direction of the gradient of each point of the target feature image using the finite difference of the first-order partial derivative;

[0134] S422, perform detection scanning on the target feature image to determine whether the amplitude of the point on the edge is the largest in the same direction, if so, keep it, otherwise delete it;

[0135] S423, determine the points with pixel values greater than threshold T2 among the retained points as boundary points, and discard the points less than threshold T1, T1<T2, determine whether the points between thresholds T1 and T2 are connected to the true boundary points, if so, determine as boundary points, if not, discard;

[0136] S424, perform Canny edge detection on the target feature image after image enhancement processing in steps S421-S423 to obtain an edge detection image, which includes an outer contour and an inner contour.

[0137] In step S421, the amplitude and the direction of the target feature image are calculated according to the following formulas:

[0138]

[0139]

[0140] are functions after the target feature image is operated with two first-order difference convolution templates.

[0141] Step S43 specifically comprises the steps of:

[0142] S431, retaining the pixels between the outer contour and the inner contour in the edge detection image, and removing the pixels outside the outer contour and inside the inner contour;

[0143] S432, performing threshold segmentation on the detection region, so that the pixel value of the defect is 255 and the pixel value of the rest is 0, to obtain the connected region of the defect part;

[0144] S433, determining whether each connected region is a specified defect according to the size of the connected region, if yes, retaining the connected region and determining that the glass bottle is a defective glass bottle, if not, setting the pixel value of the pixel point in the connected region to 0.

[0145] In step S432, since a light source is added at the bottle opening part when the image is collected, the bottle opening part will become bright due to the light, and the defect part will become dark due to the principle of light propagation. Therefore, threshold segmentation is performed on the detection region image to highlight the difference in pixel value of the defect position, so that the pixel value of the defect is 255. According to the principle of connected domain, the pixel points in the 3x3 region around the point with a pixel value of 255 are detected and extended, thereby obtaining the connected region of the defect part of the glass bottle.

[0146] In step S433, the area coefficient of the defect is set as A (A>0), A is the pixel value of the connected region with a gray value of 255 after threshold segmentation, and a threshold B (B>0) is set. When A>B, the bottle is determined to be a defective bottle, otherwise, the bottle is determined to be a non-defective bottle.

[0147] In addition, it should be noted that the defect recognition network needs to be trained, verified and tested in advance, and a related data set needs to be constructed.

[0148] First, a large number of fusion images are collected according to steps S1 and S2 to obtain a data set I NHB (RGB three-channel color image), and the data set I NHB is represented as: I NHB ={I1, I2, …, In, …, I n-1 , I n}, then the data set I NHB is cropped and labeled with defects to obtain a corresponding label set I Label (RGB single-channel binary image), and the label set I label is represented as: I label ={L1, L2, …, Ln, …, L n-1 , L n}, where In represents the nth fusion image in the data set I n , and L NHB is the label of In. n ​n The corresponding label.

[0149] Then, the images in the data set I NHB and the label set I Label are rotated by 90°, 180° and 270° to obtain the augmented data set I NHB_En and the target label set I Label_En .

[0150] Then, the augmented data set I NHB_En and the target label set I Label_En are randomly split in a ratio of 7:3, and the target label set I NHB_En is processed to be consistent according to the augmented data set I Label_En to form the target training sample data set the target test data set and the target validation sample data set are not used for training, validation and testing.

[0151] In this embodiment, the image size in the target training sample data set the target test data set and the target validation sample data set is 256x256x3, where 256 represents the width and height of the image, and 3 represents a three-channel image. The annotation size in the target label set I Label_En is 256x256x1, where 256 represents the width and height of the image, and 1 represents a single-channel image.

[0152] In the training process, the loss function is designed as:

[0153] Loss=0.4*Loss obj +0.3*Loss rect +0.3*Loss cls

[0154] Wherein, Loss obj represents the confidence loss, Loss rect represents the rectangular frame loss, and Loss cls represents the classification loss, which are calculated by the following formula respectively:

[0155]

[0156]

[0157]

[0158] Wherein, p o represents the target confidence score in the predicted frame, and piou IOU value of a prediction box and a target box corresponding thereto, w obj weight of a positive sample when calculating o binary cross-entropy loss between p iou and p gt IOU represents the overlap ratio of a prediction box and an actual box, b represents the center point of the prediction box, b gt w represents the width of the minimum rectangle of the prediction box and the actual box, h represents the height of the prediction box, h gt h represents the width of the minimum rectangle of the prediction box and the actual box, p() represents the Euclidean distance; c p gt represents the true class, w cls weight of a positive sample when calculating p binary cross-entropy loss between c gt and c

[0159] The process of training the defect recognition network is specifically:

[0160] Step 1: input the target training sample data set and the target label set I Label En into the model for training at the same time;

[0161] Step 2: calculate the model loss function value and record the current model parameters;

[0162] Step 3: determine whether the current model is better than the saved optimal model, if so, save the current model as the optimal model, otherwise do not store;

[0163] Step 4: repeat steps 1, 2 and 3 for epoch times;

[0164] Step 5: load the optimal model, and input the target test data set and the target verification sample data set into the model for testing; step 5 specifically includes the following steps:

[0165] Step 51, load the optimal model;

[0166] Step 52, input the target test data set into the model and save the result; ​​​​​​​​

[0167] Step 53, verifying the target sample data set Input the model and save the results;

[0168] Step 54, respectively, and label set I Label En Perform a comparative classification comparison;

[0169] Step 55, calculate the classification accuracy.

[0170] In step 3, the model performance comparison compares the model loss value, the lower the loss value, the better the model performance, and vice versa.

[0171] As shown in Table 1, the defect recognition network constructed by the present application can accurately identify defective glass bottles, and the average recognition accuracy (Acc) is as high as 96.67%. As shown in Table 2, compared with other defect recognition methods, the average recognition accuracy of the defect recognition network constructed by the present application is the highest, reaching 95.80%, which is more than 2% higher than other algorithms. Because the present method can simultaneously extract features from pictures in three directions, fuse them into a three-channel feature map according to different weights, and perform defect detection, it can improve the defect detection accuracy and complete the detection task at a faster speed. Moreover, the running speed of the present method is still fast. As shown in Table 2, if no image preprocessing is performed, the recognition accuracy of the present application is 90.88%, and after image preprocessing, the recognition accuracy of the present application is improved to 95.80%. It can be seen that image preprocessing has an important contribution to improving the recognition accuracy.

[0172] Table 1

[0173] Sample Type Number T (Positive Samples) F (Negative Samples) Acc / % Error / % Good Bottles 200 195 5 97.50 2.50 Defective Bottles 100 95 5 95.00 5.0 Total 300 290 10 96.67 3.33

[0174] Table 2

[0175]

[0176] ​In summary, the glass bottle defect recognition method and system provided by the embodiment of the application can improve the defect detection precision and complete the detection task at a faster speed by imaging the bottle mouth, the bottle bottom and the bottle body of the glass bottle, pre-processing the corresponding three views to weaken and inhibit the noise of the collected three views to obtain a pre-processed image, fusing the single-channel pre-processed image, extracting features of the images in three directions, fusing the features into a three-channel feature map according to different weights for defect detection, using the computing capability of the convolutional neural network to collect and process image features of different degrees, performing feature fusion by convolution and pooling of different degrees, and then performing convolution upsampling to obtain a target feature image, which can fully extract image features and help improve the defect recognition precision. Finally, the target feature image is subjected to edge detection and defect position positioning to identify whether the collected image has defects and position the defects, and the recognition precision is high. The defect recognition network is constructed by using the convolutional neural network and some convolutional layers, upsampling, edge detection and defect detection, and the network structure is simple, the recognition precision is high, and the recognition speed is fast.

[0177] Example 2

[0178] The embodiment provides a glass bottle defect recognition system, which comprises an image collection module, an image preprocessing module and a defect recognition module. The image collection module, the image preprocessing module, the image fusion module and the defect recognition module are respectively used for performing steps S1, S2, S3 and S4 in the method of the embodiment 1.

[0179] Specifically, as shown in Figure 3 The image collection module comprises a mechanical arm, a monocular camera and two mirrors. The monocular camera is arranged in front of the glass bottle on the detection station, and the two mirrors are symmetrically arranged at the left rear and right rear of the glass bottle. The mechanical arm is used for:

[0180] The glass bottle is vertically placed on the detection station, and the monocular camera captures the bottle body of the glass bottle to obtain a bottle body image;

[0181] The middle part of the vertically placed glass bottle is clamped and rotated so that the bottle mouth of the glass bottle faces the monocular camera, and the camera captures the bottle mouth of the glass bottle to obtain a bottle mouth image;

[0182] The middle part of the vertically placed glass bottle is clamped and rotated so that the bottle bottom of the glass bottle faces the monocular camera, and the camera captures the bottle bottom of the glass bottle to obtain a bottle bottom image;

[0183] The middle part of the glass bottle is clamped to take the glass bottle away from the detection station.

[0184] The specific shooting sequence can be set by the user and is not limited to the above process. The work completed by the image acquisition module, the image preprocessing module, the image fusion module, and the defect identification module is described in detail in Embodiment 1, and thus is not described again here.

[0185] The above embodiments are preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications, and the like made without departing from the spirit and principles of the present application should be equivalent replacement manners and should be included in the protection scope of the present application.

Claims

1. A glass bottle defect recognition method characterized by, The method comprises the steps of: S1, photographing the bottle mouth, bottle bottom and bottle body of the glass bottle to obtain a corresponding bottle mouth view, bottle bottom view and bottle body view; S2, preprocessing the bottle mouth view, bottle bottom view and bottle body view to obtain a preprocessed bottle mouth view, preprocessed bottle bottom view and preprocessed bottle body view; S3, fusing the preprocessed bottle mouth view, preprocessed bottle bottom view and preprocessed bottle body view to obtain a fused image; The step S3 specifically comprises the steps of: S31, a convolution operation shared by the single-channel pre-processed bottle mouth view, the pre-processed bottle bottom view and the pre-processed bottle body view is adopted, and the convolution kernel size is After being activated by a ReLU function, a padding mode is same, feature extraction is performed, and three view feature maps are obtained. S32, adopt bottle body , bottle mouth , bottle bottom The feature weight of the three view feature map is spliced to form a fusion feature; S33, the fusion features are further subjected to a convolution operation, the convolution kernel size is , the channel number of the output image is adjusted to 3, and then subjected to a linear activation function, and finally an output of a fusion image with the same size as the input image of step S31 is output channel. S4, inputting the fused image into a trained defect recognition network to locate a defect position and output a defect recognition result image; In the step S4, the defect recognition network locates a defect position and outputs a defect recognition result image, specifically comprising the steps of: S41, extracting features of the fused image to generate a target feature image; S42, performing edge detection on the target feature image to obtain an edge detection image; S43, locating a defect position on the edge detection image to obtain a defect recognition result image; The step S42 comprises the steps of: S421, calculating the amplitude and direction of the gradient of each point of the target feature image by using the finite difference of the first-order partial derivative; S422, performing detection scanning on the target feature image to determine whether the amplitude of a point on the edge is the maximum amplitude in the same direction, if yes, the point is retained, otherwise, the point is deleted; S423, judging the pixel value of the reserved point as a boundary point if the pixel value is greater than the threshold value ​​​​​​ S424, performing Canny edge detection on the target feature image after the image enhancement processing of steps S421-S423 to obtain an edge detection image, which includes an outer contour and an inner contour; The step S43 specifically comprises the steps of: S431, retaining the pixels in the detection region between the outer contour and the inner contour of the edge detection image and removing the pixels outside the outer contour and inside the inner contour; S432, performing threshold segmentation on the detection region to make the pixel value of the defect part 255 and the pixel value of the remaining part 0 to obtain a connected region of the defect part; S433, determining whether each connected region is a specified defect according to the size of the connected region, if yes, retaining the connected region and determining that the glass bottle is a defective glass bottle, if not, setting the pixel value of the connected region to 0.

2. The glass bottle defect recognition method according to claim 1, wherein The step S41 specifically comprises the steps of: S411, inputting the fused image into a convolutional neural network to generate a first feature map; S412, input the first feature map into a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a maximum pooling layer respectively to obtain a second feature map, a third feature map, a fourth feature map, a fifth feature map and a sixth feature map and fuse and splice to obtain a seventh feature map; perform convolution kernel as convolution processing on the first feature map to obtain an eighth feature map; S413, performing convolution processing on the seventh feature map by using a convolution kernel with a size of 3*3 and a stride of 1, and then performing upsampling on the seventh feature map to obtain an eighth feature map; S413, performing convolution processing on the seventh feature map by using a convolution kernel with a size of 3*3 and a stride of 1, and then performing upsampling on the seventh feature map to obtain an eighth feature map; S414, the ninth feature map is subjected to convolution kernel convolution processing, and then is subjected to upsampling to obtain a target feature map.

3. The glass bottle defect recognition method according to claim 2, wherein, In the step S412: The first convolutional layer performs a convolution kernel of , and a dilated coefficient of a cavity convolution processing on the input first feature map to obtain the second feature map. The second convolutional layer performs a convolution kernel , and an expansion coefficient of a cavity convolution processing on the input first feature map to obtain the third feature map. The third convolutional layer performs a convolution kernel of , and a dilated coefficient of a cavity convolution processing on the input first feature map to obtain the fourth feature map. The fourth convolutional layer performs a convolution kernel of , and a dilated coefficient of a cavity convolution processing on the input first feature map to obtain the fifth feature map.

4. The glass bottle defect recognition method according to claim 1, wherein, In the step S421, the amplitude of the target feature image is calculated and the direction of the formula as follows: ， ， , are functions of the target feature image and two first-order difference convolution templates, respectively.

5. The glass bottle defect recognition method according to claim 4, wherein In the training process, the loss function of the defect recognition network is designed as: ， wherein, denotes a confidence loss, denotes a bounding box loss, denotes a classification loss, respectively calculated by: ， ， ， in, This represents the target confidence score in the prediction box. This represents the Interchange of Union (IOU) value between the predicted bounding box and its corresponding target bounding box. Indicates calculation The weight of the sample at the time of birth, express and Binary cross-entropy loss between them; Indicates the overlap ratio between the predicted bounding box and the actual bounding box. Indicates the center point of the prediction box. Indicates the center point of the actual box. This represents the diagonal length of the smallest rectangle containing both the predicted and actual bounding boxes. Indicates the width of the prediction box. This indicates the actual width of the frame. This represents the width of the smallest rectangle between the predicted bounding box and the actual bounding box. Indicates the height of the predicted bounding box. Indicates the actual height of the bounding box. This represents the height of the smallest rectangle between the predicted bounding box and the actual bounding box. Indicates Euclidean distance; Indicates the prediction category, Represents the true category, Indicates calculation The weight of the sample at the time of birth, express and The binary cross-entropy loss between them.

6. The glass bottle defect recognition method according to any one of claims 1 to 5, characterized in that, The step S2 of preprocessing specifically comprises the steps of: S21, performing Gaussian filtering on the bottle mouth view, bottle bottom view and bottle body view to obtain a respective first intermediate image; S22, performing median filtering on the first intermediate image to obtain a respective second intermediate image; S23, performing gray value transformation on the second intermediate image to obtain a respective third intermediate image; S24, performing sharpening on the third intermediate image to obtain a respective fourth intermediate image; S25, performing frequency domain filtering on the fourth intermediate image to obtain the preprocessed bottle mouth view, the preprocessed bottle bottom view and the preprocessed bottle body view; In the step S23, the formula of the gray value transformation is: ， Wherein, The original pixel point gray value range is , the gray value range to be reached after image enhancement is , and the output third intermediate image is ; In the step S24, the gradient value is taken as the sharpening output, wherein the third intermediate image point The modulus point of the gradient vector at the point, i.e. the fourth intermediate image is represented as: ， , These represent the third intermediate image. point place First-order difference sum of directions The first-order difference in direction is expressed as: 。 7. A glass bottle defect recognition system characterized by: The image acquisition module, the image preprocessing module, the image fusion module and the defect identification module are respectively used for performing the steps S1, S2, S3 and S4 in any one of claims 1-6. The image acquisition module comprises a mechanical arm, a monocular camera and two mirrors, the monocular camera is arranged in front of the glass bottle on the detection work station, and the two mirrors are symmetrically arranged at the left and right rear of the glass bottle; the mechanical arm is used for: Placing the glass bottle vertically on the detection work station, at this time the monocular camera captures the bottle body of the glass bottle to obtain a bottle body image; Clamping the middle part of the bottle body of the glass bottle to rotate, so that the bottle mouth of the glass bottle faces the monocular camera, at this time the camera captures the bottle mouth of the glass bottle to obtain a bottle mouth image; Clamping the middle part of the bottle body of the glass bottle to rotate, so that the bottle bottom of the glass bottle faces the monocular camera, at this time the camera captures the bottle bottom of the glass bottle to obtain a bottle bottom image; Clamping the middle part of the bottle body of the glass bottle to take the glass bottle away from the detection work station.

Images