Transparent package foreign matter detection method and system based on multispectral fusion and adaptive threshold, and medium

Abstract

The invention provides a transparent package foreign matter detection method and system based on multispectral fusion and a self-adaptive threshold value and a medium, and the method comprises the steps: obtaining a visible light image and a near-infrared image of a target object at the same time based on a multispectral imaging technology, and carrying out the preprocessing of the visible light image and the near-infrared image, carrying out fusion processing on the pre-processed visible light image and the near-infrared image to obtain a fused image; constructing a deep learning discrimination model, inputting the fused image into the deep learning discrimination model, and outputting a foreign matter recognition result; the feature value of the foreign matter recognition result is extracted, the feature value of the foreign matter recognition result is analyzed based on a detection threshold value, foreign matter parameter information is obtained, the foreign matter parameter information is compared with set condition information, alarm information is generated, and the spectral difference between the foreign matter and the background is remarkably improved through multispectral fusion. And complex textures and reflective features can be automatically learned through a deep learning discrimination model, so that the false alarm rate is greatly reduced.

Description

Technical Field

[0001] This application relates to the field of foreign object detection technology, and more specifically, to a method, system, and medium for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold. Background Technology

[0002] In modern food, pharmaceutical, and cosmetic industries, transparent plastic or glass packaging is widely used. However, this has led to increasingly prominent issues related to contamination of the packaging contents. Especially during filling, sealing, and transportation, tiny foreign objects (such as metal dust and glass shards) can easily get into the product. These foreign objects can cause harm to consumers, resulting in significant safety hazards and economic losses.

[0003] Existing visual detection technologies, such as those based on traditional image features like grayscale, texture, and edges, suffer from low accuracy and high false alarm rates when processing images with highly transparent backgrounds and complex textures. They are particularly difficult to distinguish from smooth, semi-transparent, and reflective foreign objects under complex background conditions.

[0004] While technologies such as infrared imaging and X-rays can improve detection capabilities to some extent, they generally suffer from drawbacks such as high cost, slow detection speed, and radiation, making them unsuitable for food processing.

[0005] Therefore, there is an urgent need for a detection system that combines multispectral fusion, efficient identification algorithms, and dynamic adaptability to break through existing technological bottlenecks and achieve high-precision identification of tiny foreign objects inside transparent packaging. Summary of the Invention

[0006] The purpose of this application is to provide a method, system and medium for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold. By using multispectral fusion, the spectral difference between the foreign object and the background is significantly improved, and by using a deep learning discrimination model, complex textures and reflective features can be automatically learned, which greatly reduces the false alarm rate.

[0007] This application also provides a method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding, including: Based on multispectral imaging technology, visible light and near-infrared images of the target object are acquired simultaneously. The visible light and near-infrared images are preprocessed to obtain preprocessed visible light and near-infrared images. The preprocessed visible light image and the near-infrared image are fused to obtain a fused image; Construct a deep learning discrimination model, input the fused image into the deep learning discrimination model, and output the foreign object recognition result; The feature values ​​of the foreign object identification results are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification results are analyzed based on the detection threshold to obtain foreign object parameter information, which includes the location, shape and type of the foreign object. The foreign object parameter information is compared with the set condition information to generate alarm information, which is then transmitted to the terminal in real time.

[0008] Optionally, in the transparent packaging foreign object detection method based on multispectral fusion and adaptive threshold described in the embodiments of this application, the visible light image and near-infrared image of the target object are acquired simultaneously based on multispectral imaging technology, and the visible light image and near-infrared image are preprocessed, specifically including: Acquire a visible light image of the target object, and remove Gaussian noise from the visible light image based on a Gaussian filtering algorithm; Edge noise in a visible light image is removed based on an edge filtering algorithm to obtain a denoised visible light image. Near-infrared images of the target object are acquired, and salt-and-pepper noise in the image is removed based on the median filtering algorithm to obtain a denoised near-infrared image. The denoised visible light image and near-infrared image are grayscaled to obtain visible light grayscale image and near-infrared grayscale image; The visible light grayscale image and the near-infrared grayscale image are subjected to grayscale normalization processing to obtain the preprocessed visible light image and near-infrared image.

[0009] Optionally, in the transparent packaging foreign object detection method based on multispectral fusion and adaptive threshold described in the embodiments of this application, the preprocessed visible light image and near-infrared image are fused to obtain a fused image, specifically including: The preprocessed CAR image and near-infrared image are subjected to two-dimensional discrete wavelet transform and wavelet decomposition to obtain low-frequency sub-image and high-frequency sub-image. A low-frequency fused image is obtained by fusing low-frequency sub-images using a weighted average method. The high-frequency sub-images are fused using the maximum absolute value method to obtain a high-frequency fused image; The low-frequency fused image and the high-frequency fused image are then fused again to obtain the final fused image; The final fused image is reconstructed based on inverse wavelet transform to obtain the reconstructed fused image.

[0010] Optionally, in the foreign object detection method for transparent packaging based on multispectral fusion and adaptive threshold described in the embodiments of this application, a deep learning discriminant model is constructed, the fused image is input into the deep learning discriminant model, and the foreign object recognition result is output, specifically including: Obtain image samples without foreign objects and image samples of several different types of foreign objects to construct a training set; A joint loss function is used, which includes binary cross-entropy and Dice coefficient; The loss value of the joint loss function is calculated based on the training set, and the model parameters are dynamically adjusted based on the loss value to obtain the deep learning model. The fused image is input into a deep learning model to generate a foreign object presence probability map, and the foreign object identification result is obtained based on the foreign object presence probability map.

[0011] Optionally, in the foreign object detection method for transparent packaging based on multispectral fusion and adaptive threshold described in the embodiments of this application, feature values ​​of the foreign object identification result are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification result are analyzed based on the detection threshold to obtain foreign object parameter information, specifically including: Features are extracted based on the foreign object identification results, including morphological features, location features, and texture features; Feature values ​​are obtained by weighted fusion of morphological features, positional features, and texture features. Set a detection threshold, and analyze the category of foreign objects by comparing the detection threshold with feature values; The confidence level of the foreign object category is analyzed, and based on the confidence level of the foreign object category, foreign object parameter information is obtained, including the foreign object location, foreign object shape, and foreign object size.

[0012] Optionally, in the transparent packaging foreign object detection method based on multispectral fusion and adaptive threshold described in the embodiments of this application, the foreign object parameter information is compared with the set condition information to generate alarm information, and the alarm information is transmitted to the terminal in real time, specifically including: Obtain foreign object parameter information, including foreign object size and shape, and analyze the foreign object area based on the foreign object size and shape; The ratio of the foreign object area to the outer surface area of ​​the target object is calculated based on the foreign object area. Compare the foreign object area ratio with the set condition information; If the area ratio of the foreign object is less than the set condition information, a first warning message will be generated; If the area ratio of the foreign object is greater than or equal to the set condition information, a second warning message will be generated; The first or second early warning information is transmitted to the terminal in real time.

[0013] Secondly, embodiments of this application provide a foreign object detection system for transparent packaging based on multispectral fusion and adaptive thresholding. The system includes a memory and a processor. The memory includes a program for a method of detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding. When the program is executed by the processor, it performs the following steps: Based on multispectral imaging technology, visible light and near-infrared images of the target object are acquired simultaneously. The visible light and near-infrared images are preprocessed to obtain preprocessed visible light and near-infrared images. The preprocessed visible light image and the near-infrared image are fused to obtain a fused image; Construct a deep learning discrimination model, input the fused image into the deep learning discrimination model, and output the foreign object recognition result; The feature values ​​of the foreign object identification results are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification results are analyzed based on the detection threshold to obtain foreign object parameter information, which includes the location, shape and type of the foreign object. The foreign object parameter information is compared with the set condition information to generate alarm information, which is then transmitted to the terminal in real time.

[0014] Optionally, in the transparent packaging foreign object detection system based on multispectral fusion and adaptive threshold described in this application embodiment, the visible light image and near-infrared image of the target object are acquired simultaneously based on multispectral imaging technology, and the visible light image and near-infrared image are preprocessed, specifically including: Acquire a visible light image of the target object, and remove Gaussian noise from the visible light image based on a Gaussian filtering algorithm; Edge noise in a visible light image is removed based on an edge filtering algorithm to obtain a denoised visible light image. Near-infrared images of the target object are acquired, and salt-and-pepper noise in the image is removed based on the median filtering algorithm to obtain a denoised near-infrared image. The denoised visible light image and near-infrared image are grayscaled to obtain visible light grayscale image and near-infrared grayscale image; The visible light grayscale image and the near-infrared grayscale image are subjected to grayscale normalization processing to obtain the preprocessed visible light image and near-infrared image.

[0015] Optionally, in the transparent packaging foreign object detection system based on multispectral fusion and adaptive threshold described in this application embodiment, the preprocessed visible light image and near-infrared image are fused to obtain a fused image, specifically including: The preprocessed CAR image and near-infrared image are subjected to two-dimensional discrete wavelet transform and wavelet decomposition to obtain low-frequency sub-image and high-frequency sub-image. A low-frequency fused image is obtained by fusing low-frequency sub-images using a weighted average method. The high-frequency sub-images are fused using the maximum absolute value method to obtain a high-frequency fused image; The low-frequency fused image and the high-frequency fused image are then fused again to obtain the final fused image; The final fused image is reconstructed based on inverse wavelet transform to obtain the reconstructed fused image.

[0016] Thirdly, embodiments of this application also provide a computer-readable storage medium, which includes a program for a method of detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold. When the program for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold is executed by a processor, it implements the steps of the method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold as described in any of the preceding claims.

[0017] As can be seen from the above, the method, system, and medium for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold provided in this application embodiment simultaneously acquires visible light and near-infrared images of the target object using multispectral imaging technology. The visible light and near-infrared images are preprocessed to obtain preprocessed visible light and near-infrared images. The preprocessed visible light and near-infrared images are then fused to obtain a fused image. A deep learning discriminant model is constructed, and the fused image is input into the deep learning discriminant model to output the foreign object identification result. Feature values ​​of the foreign object identification result are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification result are analyzed based on the detection threshold to obtain foreign object parameter information. The foreign object parameter information is compared with set condition information to generate alarm information, which is then transmitted to the terminal in real time. Multispectral fusion significantly improves the spectral difference between the foreign object and the background, and the deep learning discriminant model can automatically learn complex textures and reflective features, greatly reducing the false alarm rate. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart of a foreign object detection method for transparent packaging based on multispectral fusion and adaptive threshold provided in the embodiments of this application; Figure 2 A flowchart of an image preprocessing method for a transparent packaging foreign object detection method based on multispectral fusion and adaptive threshold provided in an embodiment of this application; Figure 3 A flowchart of an image fusion processing method for a transparent packaging foreign object detection method based on multispectral fusion and adaptive threshold provided in an embodiment of this application; Figure 4A schematic diagram of visible light and near-infrared images of a transparent packaging foreign object detection system based on multispectral fusion and adaptive threshold provided in an embodiment of this application; Figure 5 A block diagram of a transparent packaging foreign object detection system based on multispectral fusion and adaptive threshold provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding, as described in some embodiments of this application. This method, used in a terminal device, includes the following steps: S101, based on multispectral imaging technology, simultaneously acquires visible light and near-infrared images of the target object, preprocesses the visible light and near-infrared images to obtain preprocessed visible light and near-infrared images; S102, the preprocessed visible light image and near-infrared image are fused to obtain a fused image; S103, Construct a deep learning discrimination model, input the fused image into the deep learning discrimination model, and output the foreign object recognition result; S104, extract the feature values ​​of the foreign object identification results, set the detection threshold, analyze the feature values ​​of the foreign object identification results based on the detection threshold, and obtain foreign object parameter information, including foreign object location, foreign object shape and foreign object type; S105 compares the foreign object parameter information with the set condition information, generates alarm information, and transmits the alarm information to the terminal in real time.

[0023] It should be noted that this application integrates the multispectral foreign object detection system of visible light and near-infrared spectroscopy, and combines image fusion technology, deep neural networks and adaptive threshold segmentation algorithm based on local statistics to achieve high-precision and robust real-time identification of foreign objects inside packaging.

[0024] Please refer to Figure 2 , Figure 2 This is a flowchart of an image preprocessing method for a transparent packaging foreign object detection method based on multispectral fusion and adaptive thresholding, as described in some embodiments of this application. According to embodiments of the present invention, based on multispectral imaging technology, visible light and near-infrared images of the target object are simultaneously acquired. Preprocessing of the visible light and near-infrared images specifically includes: S201, acquire the visible light image of the target object, and remove Gaussian noise from the visible light image based on the Gaussian filtering algorithm; S202, remove edge noise from the visible light image based on the edge filtering algorithm to obtain the denoised visible light image; S203, acquire the near-infrared image of the target object, remove salt-and-pepper noise from the image based on the median filtering algorithm, and obtain the denoised near-infrared image; S204, perform grayscale processing on the denoised visible light image and near-infrared image to obtain visible light grayscale image and near-infrared grayscale image; S205, perform grayscale normalization processing on the visible light grayscale image and the near-infrared grayscale image to obtain the preprocessed visible light image and near-infrared image.

[0025] It should be noted that the image preprocessing is as follows: Spectral image registration: Due to the differences in imaging angle and resolution between visible light and near-infrared cameras, image spatial alignment is first required. A registration method based on frequency domain phase correlation is used. , in: Represents a two-dimensional Fourier transform; Indicates its inverse transformation; This is complex number multiplication; This is a conjugate operation.

[0026] Obtain the translation vector The near-infrared image was then resampled and transformed.

[0027] Illumination normalization: Background light suppression and detail enhancement are achieved using guided filtering. , in: This represents the pixel value of the original input image at coordinates (x, y) (visible light or near-infrared image); This represents the output pixel value after illumination normalization; This is a Gaussian blurred version of the image; To prevent small constants from being divided by zero.

[0028] Please refer to Figure 3 , Figure 3 This is a flowchart of an image fusion processing method for foreign object detection in transparent packaging based on multispectral fusion and adaptive thresholding, as described in some embodiments of this application. According to embodiments of the present invention, the preprocessed visible light image and near-infrared image are fused to obtain a fused image, specifically including: S301, Perform two-dimensional discrete wavelet transform and wavelet decomposition on the preprocessed card light image and near-infrared image to obtain low-frequency sub-image and high-frequency sub-image; S302, The low-frequency sub-images are fused based on the weighted average method to obtain a low-frequency fused image; S303, high-frequency sub-images are fused based on the maximum absolute value method to obtain a high-frequency fused image; S304, perform a second fusion of the low-frequency fused image and the high-frequency fused image to obtain the final fused image; S305, the final fused image is reconstructed based on inverse wavelet transform to obtain the reconstructed fused image.

[0029] It should be noted that the multispectral image fusion algorithm is as follows: A wavelet domain-based fusion strategy is adopted to integrate complementary information from different spectra.

[0030] Based on wavelet decomposition, the two images and Perform two-dimensional discrete wavelet transform (DWT): ; It is the pixel value at pixel coordinates (x,y) of the k-th image (e.g., visible light image I1, near-infrared image I2, etc.).

[0031] in The wavelet subbands represent visible light and near-infrared images, respectively: : Low-frequency subgraph (structural information); : High-frequency sub-image (edges and textures).

[0032] The fusion rules are as follows: The low-frequency component uses an energy-weighted average method: ; Wherein the fusion weight: ; This represents the energy or image feature value of a visible light image; This represents the energy or image feature value of a near-infrared image; This represents the fusion energy of a sub-band in the fused image or the weight parameters of the fused image. The high-frequency portion uses the maximum absolute value method: ; The fused image is reconstructed using inverse wavelet transform: ; This indicates a fused image.

[0033] According to an embodiment of the present invention, a deep learning discriminant model is constructed, the fused image is input into the deep learning discriminant model, and the foreign object recognition result is output, specifically including: Obtain image samples without foreign objects and image samples of several different types of foreign objects to construct a training set; A joint loss function is used, which includes binary cross-entropy and Dice coefficient; The loss value of the joint loss function is calculated based on the training set, and the model parameters are dynamically adjusted based on the loss value to obtain the deep learning model. The fused image is input into a deep learning model to generate a foreign object presence probability map, and the foreign object identification result is obtained based on the foreign object presence probability map.

[0034] It should be noted that the deep learning discrimination model uses a structurally improved U-Net network, combined with an attention mechanism module (SE-block) to enhance the model's ability to identify small foreign objects.

[0035] The network inputs and outputs are as follows: Input: fused image ; Output: Probability graph of foreign object presence : ; in: This is a neural network function with parameters. ; This is the sigmoid activation function.

[0036] A joint loss function is used, including binary cross-entropy (BCE) and the Dice coefficient: ; Represents the joint loss function; Specifically, it appears in the "image registration" stage of the image preprocessing section, and is used to represent the two input images in the mutual information registration algorithm. Its meaning is as follows: P: Represents a visible light image (usually a standard reference image); Y: Represents the near-infrared image (needs to be aligned with the reference image).

[0037] In this application, the goal of the registration process is to determine whether P and Y are spatially aligned by calculating their mutual information. If the mutual information is higher than a certain set threshold, the image registration is considered successful.

[0038] in For the real label mask, , This represents the weighting coefficients of the loss function.

[0039] According to an embodiment of the present invention, feature values ​​of the foreign object identification result are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification result are analyzed based on the detection threshold to obtain foreign object parameter information, specifically including: Features are extracted based on the foreign object identification results, including morphological features, location features, and texture features; Feature values ​​are obtained by weighted fusion of morphological features, positional features, and texture features. Set a detection threshold, and analyze the category of foreign objects by comparing the detection threshold with feature values; The confidence level of the foreign object category is analyzed, and based on the confidence level of the foreign object category, the foreign object parameter information is obtained, including the foreign object location, foreign object shape and foreign object size.

[0040] It should be noted that the adaptive threshold segmentation is as follows: The segmentation threshold is dynamically determined using local statistical information. ; in: Indicates an adaptive threshold; :by Centered The mean of the region; Regional standard deviation; K: Empirical coefficient, usually set to K=1.5.

[0041] Generate the final detection mask: , This represents a binary detection image.

[0042] According to an embodiment of the present invention, foreign object parameter information is compared with set condition information to generate alarm information, and the alarm information is transmitted to the terminal in real time, specifically including: Obtain foreign object parameter information, including foreign object size and shape, and analyze the foreign object area based on foreign object size and shape; The ratio of the foreign object area to the outer surface area of ​​the target object is calculated based on the foreign object area. Compare the foreign object area ratio with the set condition information; If the area ratio of the foreign object is less than the set condition information, a first warning message will be generated; If the area ratio of the foreign object is greater than or equal to the set condition information, a second warning message will be generated; The first or second early warning information is transmitted to the terminal in real time.

[0043] It should be noted that by analyzing the proportion of foreign object area, different levels of early warning information are generated to distinguish the severity of foreign objects.

[0044] According to an embodiment of the present invention, image spatial alignment is performed before image fusion processing, as follows: The visible light image and the near-infrared image are spatially aligned to obtain the alignment result. The mutual information between the visible light image and the near-infrared image is calculated based on the alignment result, and it is determined whether the mutual information is greater than or equal to the set mutual threshold. If the cross threshold is greater than or equal to the set threshold, the image alignment is deemed valid, and the pixel deviation after alignment is less than 1 pixel. If the threshold is less than the set threshold, the visible light image and the near-infrared image will be recalibrated.

[0045] It should be noted that the images are corrected before image fusion to ensure that pixel deviations are small and to improve fusion accuracy.

[0046] like Figures 4-5As shown, in a second aspect, embodiments of this application provide a transparent packaging foreign object detection system based on multispectral fusion and adaptive thresholding. The system includes a memory and a processor. The memory includes a program for a transparent packaging foreign object detection method based on multispectral fusion and adaptive thresholding. When the program is executed by the processor, it performs the following steps: Based on multispectral imaging technology, visible light and near-infrared images of the target object are acquired simultaneously. The visible light and near-infrared images are preprocessed to obtain preprocessed visible light and near-infrared images. The preprocessed visible light image and the near-infrared image are fused to obtain a fused image; Construct a deep learning discrimination model, input the fused image into the deep learning discrimination model, and output the foreign object recognition result; The feature values ​​of the foreign object identification results are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification results are analyzed based on the detection threshold to obtain foreign object parameter information, including foreign object location, foreign object shape, and foreign object type. The foreign object parameter information is compared with the set condition information to generate alarm information, which is then transmitted to the terminal in real time.

[0047] It should be noted that the system consists of the following modules: Multispectral imaging module: Simultaneously acquires visible light images of the target image. and near-infrared images ; Image preprocessing module: includes image registration, background modeling, noise suppression, etc. Multispectral fusion module: Generates fused images using multi-scale fusion methods. ; Deep learning detection module: Identifies foreign objects based on an improved convolutional neural network; Adaptive threshold module: Dynamically generates detection thresholds based on local statistical information; Output module: Outputs a binary mask image and an alarm signal.

[0048] According to an embodiment of the present invention, a visible light image and a near-infrared image of a target object are simultaneously acquired based on multispectral imaging technology. Preprocessing of the visible light image and the near-infrared image specifically includes: Acquire a visible light image of the target object, and remove Gaussian noise from the visible light image based on a Gaussian filtering algorithm; Edge noise in a visible light image is removed based on an edge filtering algorithm to obtain a denoised visible light image. Near-infrared images of the target object are acquired, and salt-and-pepper noise in the image is removed based on the median filtering algorithm to obtain a denoised near-infrared image. The denoised visible light image and near-infrared image are grayscaled to obtain visible light grayscale image and near-infrared grayscale image; The visible light grayscale image and the near-infrared grayscale image are subjected to grayscale normalization processing to obtain the preprocessed visible light image and near-infrared image.

[0049] According to an embodiment of the present invention, a fused image is obtained by fusing a preprocessed visible light image with a near-infrared image, specifically including: The preprocessed CAR image and near-infrared image are subjected to two-dimensional discrete wavelet transform and wavelet decomposition to obtain low-frequency sub-image and high-frequency sub-image. A low-frequency fused image is obtained by fusing low-frequency sub-images using a weighted average method. The high-frequency sub-images are fused using the maximum absolute value method to obtain a high-frequency fused image; The low-frequency fused image and the high-frequency fused image are then fused again to obtain the final fused image; The final fused image is reconstructed based on inverse wavelet transform to obtain the reconstructed fused image.

[0050] It should be noted that the multispectral image fusion algorithm is as follows: A wavelet domain-based fusion strategy is adopted to integrate complementary information from different spectra.

[0051] Based on wavelet decomposition, the two images and Perform two-dimensional discrete wavelet transform (DWT): ; in The wavelet subbands represent visible light and near-infrared images, respectively: : Low-frequency subgraph (structural information); : High-frequency sub-image (edges and textures).

[0052] The fusion rules are as follows: The low-frequency component uses an energy-weighted average method: ; Wherein the fusion weight: ; The high-frequency portion uses the maximum absolute value method: ; The fused image is reconstructed using inverse wavelet transform: ; This indicates a fused image.

[0053] According to an embodiment of the present invention, a deep learning discriminant model is constructed, the fused image is input into the deep learning discriminant model, and the foreign object recognition result is output, specifically including: Obtain image samples without foreign objects and image samples of several different types of foreign objects to construct a training set; A joint loss function is used, which includes binary cross-entropy and Dice coefficient; The loss value of the joint loss function is calculated based on the training set, and the model parameters are dynamically adjusted based on the loss value to obtain the deep learning model. The fused image is input into a deep learning model to generate a foreign object presence probability map, and the foreign object identification result is obtained based on the foreign object presence probability map.

[0054] It should be noted that the deep learning discrimination model uses a structurally improved U-Net network, combined with an attention mechanism module (SE-block) to enhance the model's ability to identify small foreign objects.

[0055] The network inputs and outputs are as follows: Input: fused image ; Output: Probability graph of foreign object presence : ; in: This is a neural network function with parameters. ; This is the sigmoid activation function.

[0056] A joint loss function is used, including binary cross-entropy (BCE) and the Dice coefficient: ; in For the real label mask, , This represents the weighting coefficients of the loss function.

[0057] According to an embodiment of the present invention, feature values ​​of the foreign object identification result are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification result are analyzed based on the detection threshold to obtain foreign object parameter information, specifically including: Features are extracted based on the foreign object identification results, including morphological features, location features, and texture features; Feature values ​​are obtained by weighted fusion of morphological features, positional features, and texture features. Set a detection threshold, and analyze the category of foreign objects by comparing the detection threshold with feature values; The confidence level of the foreign object category is analyzed, and based on the confidence level of the foreign object category, the foreign object parameter information is obtained, including the foreign object location, foreign object shape and foreign object size.

[0058] It should be noted that the adaptive threshold segmentation is as follows: The segmentation threshold is dynamically determined using local statistical information. ; in: Indicates an adaptive threshold; :by Centered The mean of the region; Regional standard deviation; K: Empirical coefficient, usually set to K=1.5.

[0059] Generate the final detection mask: , This represents a binary detection image.

[0060] According to an embodiment of the present invention, foreign object parameter information is compared with set condition information to generate alarm information, and the alarm information is transmitted to the terminal in real time, specifically including: Obtain foreign object parameter information, including foreign object size and shape, and analyze the foreign object area based on foreign object size and shape; The ratio of the foreign object area to the outer surface area of ​​the target object is calculated based on the foreign object area. Compare the foreign object area ratio with the set condition information; If the area ratio of the foreign object is less than the set condition information, a first warning message will be generated; If the area ratio of the foreign object is greater than or equal to the set condition information, a second warning message will be generated; The first or second early warning information is transmitted to the terminal in real time.

[0061] It should be noted that by analyzing the area ratio of foreign objects, different levels of early warning information are generated to distinguish the severity of foreign objects.

[0062] According to an embodiment of the present invention, image spatial alignment is performed before image fusion processing, as follows: The visible light image and the near-infrared image are spatially aligned to obtain the alignment result. The mutual information between the visible light image and the near-infrared image is calculated based on the alignment result, and it is determined whether the mutual information is greater than or equal to the set mutual threshold. If the cross threshold is greater than or equal to the set threshold, the image alignment is deemed valid, and the pixel deviation after alignment is less than 1 pixel. If the threshold is less than the set threshold, the visible light image and the near-infrared image will be recalibrated.

[0063] It should be noted that the images are corrected before image fusion to ensure that pixel deviations are small and to improve fusion accuracy.

[0064] A third aspect of the present invention provides a computer-readable storage medium including a program for a method of detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold. When the program is executed by a processor, it implements the steps of the method of detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold as described above.

[0065] This invention discloses a method, system, and medium for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding. It simultaneously acquires visible light and near-infrared images of the target object using multispectral imaging technology. The visible light and near-infrared images are preprocessed to obtain preprocessed visible light and near-infrared images. These preprocessed images are then fused to obtain a fused image. A deep learning discriminant model is constructed, and the fused image is input into the model to output the foreign object identification result. Feature values ​​of the foreign object identification result are extracted, a detection threshold is set, and the feature values ​​are analyzed based on the detection threshold to obtain foreign object parameter information. This foreign object parameter information is compared with set condition information to generate alarm information, which is transmitted to the terminal in real time. Multispectral fusion significantly improves the spectral difference between the foreign object and the background, and the deep learning discriminant model can automatically learn complex textures and reflective features, greatly reducing the false alarm rate.

[0066] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0067] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0068] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0069] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0070] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding, characterized in that, include: Based on multispectral imaging technology, visible light and near-infrared images of the target object are acquired simultaneously. The visible light and near-infrared images are preprocessed to obtain preprocessed visible light and near-infrared images. The preprocessed visible light image and the near-infrared image are fused to obtain a fused image; Construct a deep learning discrimination model, input the fused image into the deep learning discrimination model, and output the foreign object recognition result; The feature values ​​of the foreign object identification results are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification results are analyzed based on the detection threshold to obtain foreign object parameter information, which includes the location, shape and type of the foreign object. The foreign object parameter information is compared with the set condition information to generate alarm information, which is then transmitted to the terminal in real time.

2. The method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding according to claim 1, characterized in that, Based on multispectral imaging technology, visible light and near-infrared images of the target object are acquired simultaneously. The visible light and near-infrared images are then preprocessed, specifically including: Acquire a visible light image of the target object, and remove Gaussian noise from the visible light image based on a Gaussian filtering algorithm; Edge noise in a visible light image is removed based on an edge filtering algorithm to obtain a denoised visible light image. Near-infrared images of the target object are acquired, and salt-and-pepper noise in the image is removed based on the median filtering algorithm to obtain a denoised near-infrared image. The denoised visible light image and near-infrared image are grayscaled to obtain visible light grayscale image and near-infrared grayscale image; The visible light grayscale image and the near-infrared grayscale image are subjected to grayscale normalization processing to obtain the preprocessed visible light image and near-infrared image.

3. The method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding according to claim 2, characterized in that, The preprocessed visible light image and the near-infrared image are fused to obtain a fused image, specifically including: The preprocessed CAR image and near-infrared image are subjected to two-dimensional discrete wavelet transform and wavelet decomposition to obtain low-frequency sub-image and high-frequency sub-image. A low-frequency fused image is obtained by fusing low-frequency sub-images using a weighted average method. The high-frequency sub-images are fused using the maximum absolute value method to obtain a high-frequency fused image; The low-frequency fused image and the high-frequency fused image are then fused again to obtain the final fused image; The final fused image is reconstructed based on inverse wavelet transform to obtain the reconstructed fused image.

4. The method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding according to claim 3, characterized in that, Construct a deep learning discrimination model, input the fused image into the deep learning discrimination model, and output the foreign object recognition result, specifically including: Obtain image samples without foreign objects and image samples of several different types of foreign objects to construct a training set; A joint loss function is used, which includes binary cross-entropy and Dice coefficient; The loss value of the joint loss function is calculated based on the training set, and the model parameters are dynamically adjusted based on the loss value to obtain the deep learning model. The fused image is input into a deep learning model to generate a foreign object presence probability map, and the foreign object identification result is obtained based on the foreign object presence probability map.

5. The method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding according to claim 4, characterized in that, The feature values ​​of the foreign object identification results are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification results are analyzed based on the detection threshold to obtain foreign object parameter information, specifically including: Features are extracted based on the foreign object identification results, including morphological features, location features, and texture features; Feature values ​​are obtained by weighted fusion of morphological features, positional features, and texture features. Set a detection threshold, and analyze the category of foreign objects by comparing the detection threshold with feature values; The confidence level of the foreign object category is analyzed, and based on the confidence level of the foreign object category, foreign object parameter information is obtained, including the foreign object location, foreign object shape, and foreign object size.

6. The method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive thresholding according to claim 5, characterized in that, The foreign object parameter information is compared with the set condition information to generate an alarm message, which is then transmitted to the terminal in real time. Specifically, this includes: Obtain foreign object parameter information, including foreign object size and shape, and analyze the foreign object area based on the foreign object size and shape; The ratio of the foreign object area to the outer surface area of ​​the target object is calculated based on the foreign object area. Compare the foreign object area ratio with the set condition information; If the area ratio of the foreign object is less than the set condition information, a first warning message will be generated; If the area ratio of the foreign object is greater than or equal to the set condition information, a second warning message will be generated; The first or second early warning information is transmitted to the terminal in real time.

7. A foreign object detection system for transparent packaging based on multispectral fusion and adaptive threshold, characterized in that, The system includes a memory and a processor. The memory contains a program for a foreign object detection method for transparent packaging based on multispectral fusion and adaptive thresholding. When the program for the foreign object detection method for transparent packaging based on multispectral fusion and adaptive thresholding is executed by the processor, it performs the following steps: Based on multispectral imaging technology, visible light and near-infrared images of the target object are acquired simultaneously. The visible light and near-infrared images are preprocessed to obtain preprocessed visible light and near-infrared images. The preprocessed visible light image and the near-infrared image are fused to obtain a fused image; Construct a deep learning discrimination model, input the fused image into the deep learning discrimination model, and output the foreign object recognition result; The feature values ​​of the foreign object identification results are extracted, a detection threshold is set, and the feature values ​​of the foreign object identification results are analyzed based on the detection threshold to obtain foreign object parameter information, which includes the location, shape and type of the foreign object. The foreign object parameter information is compared with the set condition information to generate alarm information, which is then transmitted to the terminal in real time.

8. The transparent packaging foreign object detection system based on multispectral fusion and adaptive threshold as described in claim 7, characterized in that, Based on multispectral imaging technology, visible light and near-infrared images of the target object are acquired simultaneously. The visible light and near-infrared images are then preprocessed, specifically including: Acquire a visible light image of the target object, and remove Gaussian noise from the visible light image based on a Gaussian filtering algorithm; Edge noise in a visible light image is removed based on an edge filtering algorithm to obtain a denoised visible light image. Near-infrared images of the target object are acquired, and salt-and-pepper noise in the image is removed based on the median filtering algorithm to obtain a denoised near-infrared image. The denoised visible light image and near-infrared image are grayscaled to obtain visible light grayscale image and near-infrared grayscale image; The visible light grayscale image and the near-infrared grayscale image are subjected to grayscale normalization processing to obtain the preprocessed visible light image and near-infrared image.

9. The transparent packaging foreign object detection system based on multispectral fusion and adaptive threshold as described in claim 8, characterized in that, The preprocessed visible light image and the near-infrared image are fused to obtain a fused image, specifically including: The preprocessed CAR image and near-infrared image are subjected to two-dimensional discrete wavelet transform and wavelet decomposition to obtain low-frequency sub-image and high-frequency sub-image. A low-frequency fused image is obtained by fusing low-frequency sub-images using a weighted average method. The high-frequency sub-images are fused using the maximum absolute value method to obtain a high-frequency fused image; The low-frequency fused image and the high-frequency fused image are then fused again to obtain the final fused image; The final fused image is reconstructed based on inverse wavelet transform to obtain the reconstructed fused image.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a program for a method of detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold. When the program is executed by a processor, it implements the steps of the method for detecting foreign objects in transparent packaging based on multispectral fusion and adaptive threshold as described in any one of claims 1 to 6.

Images