Foreign Object Detection Method, Device and Storage Medium in a Glass Bottle

Abstract

The present invention relates to the field of foreign object detection in canned foods, and particularly to a method, device and storage medium for detecting foreign objects in glass bottles. The method includes: Step S1: For the glass bottles to be detected, obtain X-ray images at multiple different voltages from the same perspective; Step S2: Align all the obtained X-ray images and synthesize them to obtain a multi-channel first image; Step S3: After performing convolution and upsampling on the first image to obtain a low-frequency feature map, subtract the low-frequency feature map from the first image to obtain a high-frequency feature map; Step S4: Enhance the high-frequency feature map; Step S5: After fusing the enhanced high-frequency feature map and the upsampled low-frequency feature map, perform convolution to obtain an output image; Step S6: Input the output image into a target detection model to obtain a foreign object detection result. Compared with the prior art, the present invention can detect suspended foreign objects at different positions without long-term static settlement after filling.

Description

Technical Field

[0001] The present invention relates to the field of foreign object detection in filled foods, and in particular to a method, device and storage medium for detecting foreign objects in glass bottles. Background Art

[0002] With the continuous development of industrial automation, the quality inspection technology of bottled products has become an important link in industries such as food and medicine. As a common packaging material, the detection of foreign objects inside glass bottles is of great significance for ensuring product quality and consumer safety.

[0003] Although some existing technologies provide detection solutions for glass bottle defects. For example, Chinese Patent CN117351314A discloses a method and system for identifying glass bottle defects. However, the light source used in this glass bottle defect identification method is a visible light source, which can only be used for detecting the defects of glass bottles with relatively large light transmittance. When the glass bottle is filled with relevant products, or when the bottle body of the glass bottle is pasted with a label, it cannot penetrate the glass bottle to form an image, so it cannot realize the detection of foreign objects inside the glass bottle. In addition, Chinese Patent CN118090743A provides a multi-modal based quality inspection system for porcelain wine bottles. Similarly, its purpose is also to detect the defects of the bottle itself, so it also relies on visible light images.

[0004] In response to this, some improved existing technologies improve the penetration ability by replacing the light source with an X-ray source, so as to realize the detection of the inside of thicker glass bottles. For example, Chinese Patent CN112508930A discloses a method and device for detecting foreign objects in food based on deep learning, or Chinese Patent CN 113567478 A discloses a new type of X-ray foreign object detection system with a single light source and three perspectives, both of which use X-ray sources to realize the detection of foreign objects in food packaging. In particular, Chinese Patent CN 113567478 A mainly aims at the detection of foreign objects in glass cans. However, this solution still has the following problems: Due to the complex structural characteristics of glass bottles, such as the thickness and density of the bottle body, bottle cap and bottle bottom are different, traditional single-voltage X-ray imaging technology is difficult to effectively detect foreign objects in each part in the same image. Existing X-ray detection methods usually identify foreign objects through a single voltage or simple image processing algorithms. This method may perform well when detecting foreign objects in the thinner area of the bottle body, but when dealing with areas with higher density such as the bottle bottom or bottle cap, the contrast and resolution of the image are insufficient, which easily leads to a decrease in detection accuracy. In addition, excessive voltage increase may cause overexposure in the bottle body area, thus masking fine foreign objects. Therefore, how to obtain clear images in different density areas and accurately detect foreign objects has become an urgent technical problem to be solved.

[0005] In this regard, although Chinese Patent CN 113567478 A attempts to take pictures from multiple perspectives, and different travel paths corresponding to different perspectives can achieve the effect of reducing blind spots, and certain results have been achieved in detecting foreign objects at the bottom of the bottle, it still has the following defects:

[0006] 1. For some foreign objects suspended in the solution, effective detection cannot be achieved. Therefore, on the one hand, the types of detectable foreign objects are limited, and on the other hand, after the product is filled, it needs to be fully static before foreign object detection can be carried out, which undoubtedly reduces the turnover efficiency of the entire production line;

[0007] 2. The deviation of the three-view drawings adopted is large, and the image registration is difficult. If forced fusion processing is carried out, there are defects in misleading the model, which easily leads to the model being unable to converge or specialize. Therefore, only the images from different perspectives can be preprocessed and feature extracted separately, and then classified, which undoubtedly reduces the efficiency. Summary of the Invention

[0008] The purpose of the present invention is to provide a method, device and storage medium for detecting foreign objects in a glass bottle to solve the problem that foreign object detection can only be carried out after a long-time static state after filling in the prior art. By taking X-ray images with the same perspective but different voltages, the perspectives of all single-channel X-ray images obtained are the same, the registration difficulty is low, and no new errors are introduced due to scaling and fusion. In addition, due to the use of different voltages, the imaging requirements of different density regions can be effectively covered. Subsequently, the output pictures obtained by sequentially separating high and low frequencies, enhancing the high frequencies, and then splicing and convolving can make the foreign objects suspended in the glass bottle be clearly displayed. While improving the detection accuracy and reliability, the foreign object detection process can be placed after the filling process, and there is no need for static state after filling, which greatly improves the turnover efficiency of the entire production line.

[0009] The purpose of the present invention can be realized through the following technical solutions:

[0010] A method for detecting foreign objects in a glass bottle, comprising:

[0011] Step S1: For the glass bottle to be detected, obtain X-ray pictures with multiple different voltages from the same perspective respectively;

[0012] Step S2: Align all the obtained X-ray pictures and synthesize them to obtain a multi-channel first picture;

[0013] Step S3: After performing convolution and upsampling on the first picture to obtain a low-frequency feature map, subtract the low-frequency feature map from the first picture to obtain a high-frequency feature map;

[0014] Step S4: Enhance the high-frequency feature map;

[0015] Step S5: After fusing the enhanced high-frequency feature map and the upsampled low-frequency feature map, perform convolution to obtain the output image;

[0016] Step S6: Input the output image into the target detection model to obtain the foreign object detection result.

[0017] In step S1, three X-ray images with different voltages are obtained from the same perspective, and the first image is a three-channel image.

[0018] In the convolution process of step S3, the size of the first convolution kernel is 3*3 and the stride is 2.

[0019] In the upsampling process of step S3, the bilinear interpolation method is adopted, and the size of the low-frequency feature map is the same as that of the first image.

[0020] In the enhancement process of step S4, the enhanced high-frequency feature map is:

[0021] F EP =Concat( F H * Conv 4 ,( F H * Conv2*CBAM *Conv3))

[0022] Where: F H is the high-frequency feature map before enhancement, F EP is the enhanced high-frequency feature map, Conv2 is the second convolution kernel, Conv3 is the third convolution kernel, Conv 4 is the fourth convolution kernel, CBAM is the attention module operator, and Concat is the splicing operation.

[0023] Step S5 includes:

[0024] Step S5-1: Splice the enhanced high-frequency feature map and the upsampled low-frequency feature map along the channel dimension to obtain the first intermediate image;

[0025] Step S5-2: Perform convolution operation on the first intermediate image using the fifth convolution kernel to obtain the output image.

[0026] All the convolution kernels and operators in steps S2 to S5 are obtained by deep learning training, and the training process is adjusted based on the first loss function, and the first loss function is:

[0027]

[0028] Wherein: L is the first loss function, is the normalized pixel value of the pixel at the i th row j and th column in the output image obtained from step S5, i is the normalized pixel value of the pixel at the j th row γ and th column in the output image of the training sample,

[0029] The normalized pixel value of any pixel is:

[0030]

[0031] Wherein: is the value of the i th channel in the normalized pixel value of the pixel at the j th row k and th column, i is the value of the j th channel in the pixel value before normalization of the pixel at the k th row and i th column, j is the minimum value of all channels of the pixel at the th row i and j th column before normalization,

[0032] A device for detecting foreign objects in a glass bottle, comprising a memory, a processor, and a program stored in the memory, wherein when the processor executes the program, the method as described above is implemented.

[0033] A storage medium, on which a program is stored, and when the program is executed, the method as described above is implemented.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. Use X-ray light sources with the same viewing angle but different voltages to shoot, so that all single-channel X-ray images have the same viewing angle, low registration difficulty, and no new errors will be introduced due to scaling and fusion. In addition, due to the use of different voltages, the imaging needs of different density areas can be effectively covered. The output image obtained by subsequent high- and low-frequency separation, high-frequency enhancement, and convolution can clearly display foreign matter suspended in the glass bottle. While improving detection accuracy and reliability, the foreign matter detection process can be placed close to the filling process, without the need for static placement after filling, which greatly improves the flow efficiency of the entire production line.

[0036] 2. Compared with the conventional convolution and pooling enhancement methods, the high-frequency features are enhanced by three convolutions and one attention combined with splicing, which can highlight the advantages of details and clarity and completely ignore the contours and background. Figure 2 The second splicing and re-convolution can make up for the lack of contour in the high-frequency feature map, thereby identifying foreign objects suspended in various positions of the glass bottle.

[0037] 3. The first loss function adopted can give higher weights to pixels with larger prediction errors, so as to better handle areas with inaccurate predictions. At the same time, normalizing each pixel based on the linear mapping of the three channels can fully integrate the information of different channels, so that the details and contours of the imaging of each voltage can be fully displayed. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic flow chart of the main steps of the method of the present invention;

[0039] Figure 2 Schematic diagram of an X-ray image under a single voltage. DETAILED DESCRIPTION

[0040] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0041] Example 1

[0042] A method for detecting foreign matter in a glass bottle, such as Figure 1 As shown, including:

[0043] Step S1: for the glass bottle to be inspected, a plurality of X-ray images of different voltages at the same viewing angle are obtained respectively;

[0044] Different from the foreign object detection methods such as the prior art CN 113567478 A, the technical solution adopted in this application is a single perspective. The single perspective method has advantages such as low material cost, simple on-site layout, and convenient maintenance. In this application, three different voltages are used to take pictures of a glass bottle from the same perspective. Since the shooting time is very short, at the normal conveying speed of the conveyor belt, the deviation of the three X-ray pictures is extremely small. The specific shooting interval needs to be determined according to the device performance of the on-site electronic control part. The processing speed of the CCD will not be a limiting factor. Generally, if the switching speed supported by the X-ray source and its driver is faster, the interval can be smaller, and thus the position deviation of the three X-ray pictures is also smaller. Generally, if the deviation is less than the physical length corresponding to one pixel, it is considered that there is no deviation.

[0045] Specifically, in this embodiment, a total of X-ray pictures with three different voltages from the same perspective are obtained. As Figure 2 shown, each X-ray picture is a single-channel picture taken at a single voltage. After synthesis, a three-channel first picture can be obtained.

[0046] Among them, due to the use of different voltages, the energy of the light emitted by the X-ray source at different voltages is different, which can effectively cover the imaging requirements of different density regions.

[0047] Step S2: Align all the obtained X-ray pictures and synthesize them to obtain a multi-channel first picture. Generally, since the shooting perspectives are the same, if the deviation is less than the physical length corresponding to one pixel, the single pictures are default aligned. Of course, in some embodiments, if the deviation is too large, alignment can be performed by translation.

[0048] In addition, in this embodiment, the alignment process also includes cropping, and the glass bottle part is cropped as much as possible while keeping the size of the overall picture smaller. This can effectively reduce the computing power requirements in the subsequent processing process and also reduce noise. In this cropping process, the specific cropping area can be determined by using the imaging diagram of the medium voltage. In the imaging diagram of the medium voltage, the distinction between the background and the foreground is more obvious, which can improve the cropping accuracy. Of course, in other embodiments, other cropping methods can also be used.

[0049] Step S3: After performing convolution and upsampling on the first picture to obtain a low-frequency feature map, subtract the low-frequency feature map from the first picture to obtain a high-frequency feature map;

[0050] In this embodiment, the size of the first convolution kernel in the convolution process is 3*3, the stride is 2, the bilinear interpolation method is used in the upsampling process, and the size of the low-frequency feature map is the same as that of the first picture.

[0051] Among them, the low-frequency information can reflect the general outline and background of the image, thus cooperating with the subsequent high-frequency enhancement method.

[0052] Step S4: Enhance the high-frequency feature map. In this embodiment, during the enhancement process, the enhanced high-frequency feature map is:

[0053] F EP =Concat( F H * Conv 4 ,( F H * Conv2*CBAM *Conv3))

[0054] Where: F H is the high-frequency feature map before enhancement, F EP is the high-frequency feature map after enhancement, Conv2 is the second convolution kernel, Conv3 is the third convolution kernel, Conv 4 is the fourth convolution kernel, CBAM is the attention module operator, and Concat is the splicing operation.

[0055] Compared with the enhancement method of conventional convolution and pooling, using the method of three convolutions, one attention combined with splicing to enhance the high-frequency features can highlight the advantages of its details and clarity, completely ignoring the outline and background. Subsequently, the enhanced high-frequency feature map and the upsampled low-frequency feature Figure 2 are spliced and convolved again, which can make up for the problem of insufficient outline of the high-frequency feature map, so as to identify foreign objects suspended at various positions in the glass bottle.

[0056] In this embodiment, the sizes of the above-mentioned second convolution kernel, third convolution kernel and fourth convolution kernel are all 3*3.

[0057] Step S5: After fusing the enhanced high-frequency feature map and the upsampled low-frequency feature map, perform convolution to obtain the output image. In this embodiment, it includes:

[0058] Step S5-1: Splice the enhanced high-frequency feature map and the upsampled low-frequency feature map along the channel dimension to obtain the first intermediate image;

[0059] Step S5-2: Use the fifth convolution kernel to perform convolution operation on the first intermediate image to obtain the output image. In this embodiment, the size of the fifth convolution kernel is also 3*3.

[0060] In addition, in some embodiments, a 1*1 convolution can be used to adjust the output channels.

[0061] Step S6: Input the output image into the target detection model to obtain the foreign object detection result.

[0062] All the convolutional kernels and operators in the above steps S2 to S5 are obtained by deep learning training, and the training process is adjusted based on the first loss function. The first loss function is:

[0063]

[0064] Where: L is the first loss function, is the normalized pixel value of the pixel at the i th row j and th column in the output image obtained from step S5, i is the normalized pixel value of the pixel at the j th row γ and th column in the output image of the training sample,

[0065] is the dynamic weight amplitude control parameter, which is set to be greater than 1 in this embodiment,

[0066]

[0067] Where: is the value of the i th channel in the normalized pixel value of the pixel at the j th row k and th column, i is the value of the j th channel in the pixel value before normalization of the pixel at the k th row and i th column, j is the minimum value of all channels of the pixel at the th row i and j th column before normalization,

[0068] In summary, by using an X-ray source with different voltages from the same perspective to take pictures, the perspectives of all single-channel X-ray images obtained are the same, the registration difficulty is low, and no new errors are introduced due to scaling and fusion. In addition, by using different voltages, the imaging requirements of different density regions can be effectively covered. Subsequently, the output pictures obtained by sequentially performing high-low frequency separation, high-frequency enhancement, and then splicing and convolving can make the foreign objects suspended in the glass bottle be clearly displayed. While improving the detection accuracy and reliability, the foreign object detection process can be placed close to the position after the filling process, without the need for static settlement after filling, greatly improving the turnover efficiency of the entire production line.

[0069] Example 2

[0070] This example is generally similar to Example 1, with only a few differences. Specifically, the difference between this example and Example 1 is that in step S4 of this example, the high-frequency features Figure 1 are enhanced twice in total, and the two enhancement methods are the same.

[0071] In addition, in order to further verify the effect of this application, the following comparative examples are also set:

[0072] Comparative Example 1

[0073] This comparative example is designed with reference to Example 1. The difference from Example 1 is that in step S4 of this comparative example, only convolution and pooling operations are used, without splicing, that is, the size of the high-frequency feature map after enhancement is the same as that before enhancement, and the rest is the same as Example 1.

[0074] Comparative Example 2

[0075] This comparative example is designed with reference to Chinese Patent CN117351314A in combination with Chinese Patent CN112508930A. A total of three different perspectives are arranged to synchronously collect the glass bottles to be tested. The light source used is an X-ray source. The arrangement directions of the three perspectives are the same as those in Chinese Patent CN117351314A, and different voltages are used. The rest is arranged with reference to CN117351314A.

[0076] By testing the data of the glass beverage filling production line, in the production line, the time interval between the filling process and the foreign object detection process is about 6 seconds, and there is not enough time to wait for static settlement. A total of 1000 groups of samples are obtained. Each group of samples includes three single-channel X-ray pictures, of which 700 groups are used as training set samples, and the remaining 300 groups are used as test sets. Among them, the foreign object detection rate of Example 2 exceeds 97%, that of Example 1 also exceeds 91%, the detection rate of Comparative Example 2 is lower than 10%, and the detection rate of Comparative Example 1 is about 60%.

[0077] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

Claims

1. A method for detecting foreign objects in a glass bottle, characterized in that, Including: Step S1: For the glass bottles to be detected, obtain X-ray images at multiple different voltages from the same perspective respectively; Step S2: Align all the obtained X-ray images and synthesize them to obtain a multi-channel first image; Step S3: After performing convolution and upsampling on the first image to obtain a low-frequency feature map, subtract the low-frequency feature map from the first image to obtain a high-frequency feature map; Step S4: Enhance the high-frequency feature map; Step S5: After fusing the enhanced high-frequency feature map and the upsampled low-frequency feature map, perform convolution to obtain an output image; Step S6: Input the output image into the target detection model to obtain the foreign object detection result; During the enhancement process in Step S4, the enhanced high-frequency feature map is: F EP =Concat( F H * Conv 4 ,( F H * Conv2*CBAM *Conv3)) Wherein: F H is the high-frequency feature map before enhancement, F EP is the high-frequency feature map after enhancement, Conv2 is the second convolutional kernel, and Conv3 is the third convolutional kernel, Conv 4 is the fourth convolutional kernel, CBAM is the attention module operator, and Concat is the concatenation operation; All the convolution kernels and operators in Steps S2 to S5 are obtained by deep learning training, and the training process is adjusted based on the first loss function, and the first loss function is: ， in: L is the first loss function, is the output image obtained by step S5 i OK j The normalized pixel value of the pixel point in the column, is the output image in the training sample i OK j The normalized pixel value of the pixel point in the column, γ It is the dynamic weight amplitude control parameter.

2. The foreign object detection method in a glass bottle according to claim 1, wherein In Step S1, three X-ray images at different voltages from the same perspective are obtained in total, and the first image is a three-channel image.

3. A method for detecting foreign objects in a glass bottle according to claim 1, characterized in that, The size of the first convolution kernel in the convolution process in Step S3 is 3*3, and the stride is 2.

4. A method for detecting foreign objects in a glass bottle according to claim 1, characterized in that, The upsampling process in Step S3 adopts the bilinear interpolation method, and the size of the low-frequency feature map is the same as that of the first image.

5. A method for detecting foreign objects in a glass bottle according to claim 1, characterized in that, Step S5 includes: Step S5-1: Concatenate the enhanced high-frequency feature map and the upsampled low-frequency feature map along the channel dimension to obtain a first intermediate image; Step S5-2: Perform a convolution operation on the first intermediate image using the fifth convolution kernel to obtain an output image.

6. The foreign object detection method in a glass bottle according to claim 1, wherein, The pixel value after normalization of any pixel point is: ， Wherein: is the value of the i th row j and k th channel among the normalized pixel values of the pixel at the th column; i is the value of the j th row k and th channel among the pixel values before normalization of the pixel at the i th column; j is the minimum value of all channels of the pixel at the th row i and j th column before normalization; is the maximum value of all channels of the pixel at the th row and th column before normalization.

7. A foreign object detection device inside a glass bottle, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1-6.

8. A storage medium, on which a program is stored, characterized in that, When the program is executed, it implements the method described in any one of claims 1-6.

Images