Quality inspection method for packaged products based on fusion of X-ray and visible light

Abstract

The present invention relates to the technical field of packaging product detection, and specifically to a quality detection method for packaging products based on the fusion of X-rays and visible light. In the present invention, X-ray images and visible light images are collected; feature engineering processing is performed on the X-ray images and visible light images to generate X-ray feature data and visible light feature data, wherein the data processed by feature engineering includes spatial and structural information; the X-ray feature data is used to detect the internal product quality using an internal quality detection model, and different output results are determined according to different internal product quality detection results; and the visible light feature data is enhanced according to the abnormal position to generate a visible light abnormal image, and the visible light abnormal image is used to detect the external product quality using an external quality detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of packaged product detection, and in particular to a packaged product quality detection method based on the fusion of X-rays and visible light. Background Art

[0002] Traditional quality inspection methods for packaged products usually rely on a single imaging technology, such as using only visible light images or X-ray images for inspection. The visible light image inspection method mainly focuses on the appearance inspection of the product's exterior, and identifies surface defects such as scratches and stains through simple image processing algorithms. However, this method can only detect external defects, but is powerless against internal quality problems and cannot provide internal structural information of the product.

[0003] On the other hand, although X-ray images alone can reveal internal structural defects of products, they are prone to overlooking external defects due to the complexity of X-ray images and the lack of color and appearance information. At the same time, processing and interpreting these images requires more expertise and equipment. Traditional methods also have limitations in feature extraction. They usually use relatively basic image processing technology and cannot fully utilize the spatial and structural information in the image, resulting in insufficient accuracy and reliability of the detection results.

[0004] In addition, traditional inspection methods mostly rely on manual or semi-automated processing, which makes it difficult to conduct large-scale efficient inspections, and their ability to detect complex defects is limited. Traditional inspection methods lack comprehensive coverage of internal and external quality, and cannot achieve in-depth utilization of image space and structural information. The overall inspection effect and efficiency are obviously backward, and it is difficult to meet the needs of modern industry for high-precision and high-efficiency quality control. Summary of the invention

[0005] The purpose of the present invention is to provide a quality inspection method for packaged products based on the fusion of X-rays and visible light to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a quality inspection method for packaged products based on the fusion of X-rays and visible light, the method steps comprising:

[0007] S1, collect X-ray images and visible light images;

[0008] S2. Perform feature engineering processing on the X-ray image and the visible light image to generate X-ray feature data and visible light feature data, wherein the feature engineering processed data includes spatial and structural information;

[0009] S3. Using the internal quality inspection model to inspect the internal product quality of the X-ray feature data, wherein:

[0010] If the internal product quality test result is normal, the external quality test model is used to perform external product quality test on the visible light feature data, wherein: when the external product quality test is abnormal, the product external abnormal result is output; when the external product quality test is normal, the product internal and external normal results are output;

[0011] If the internal product quality test result is abnormal, based on the spatial and structural information of the X-ray feature data, the abnormal position corresponding to the X-ray feature data in the abnormal detection result is extracted, and the visible light feature data is enhanced according to the abnormal position to generate a visible light abnormal image; and the external product quality is tested on the visible light abnormal image using the external quality detection model, wherein: when the external product quality test is abnormal, the internal and external abnormal results of the product are output; when the external product quality test is normal, the internal abnormal result of the product is output;

[0012] The internal quality detection model and the external quality detection model are composed of different convolutional neural network models.

[0013] As a further improvement of the present technical solution, the X-ray image is an internal structure image obtained by scanning the packaged product to be inspected by an X-ray device; the visible light image is an external structure image obtained by photographing the external surface of the packaged product by a visible light camera.

[0014] As a further improvement of the present technical solution, the feature engineering includes preprocessing, which specifically includes:

[0015] Resizing: resize all images to a uniform size by interpolating to the target size;

[0016] Normalization: adjust the image pixel values ​​to the range of (0,1) by subtracting the minimum value of the image pixel value from each pixel value and then dividing it by the range of the original image pixel value;

[0017] Noise processing: Remove noise from the image, apply a Gaussian kernel function to the image, and perform a convolution operation.

[0018] As a further improvement of the present technical solution, the feature engineering includes feature extraction, and the feature extraction includes X-ray image feature extraction and visible light image feature extraction. The X-ray image feature extraction specifically includes:

[0019] The preprocessed X-ray image is input into an X-ray residual network model, which includes residual blocks, which are connected across layers to capture complex features; the X-ray residual network model has a deep network structure consisting of multiple convolution operations, regularization processing and activation operations;

[0020] Select the convolutional layer output after the last residual block and before the fully connected layer in the last X-ray residual network model as the X-ray feature layer, which contains spatial and structural information, obtain a feature map from the X-ray feature layer, and use the feature map as X-ray feature data;

[0021] Visible light image feature extraction specifically includes:

[0022] Inputting the preprocessed visible light image into a visible light residual network model, which is another residual network model, and the visible light residual network model is composed of a series of convolutional layers, batch normalization layers and activation function layers;

[0023] The output of the convolutional layer after the last residual block and before the fully connected layer in the last visible light residual network model is selected as the visible light feature layer, which contains spatial and structural information. A feature map is obtained from the visible light feature layer, and the feature map is used as visible light feature data.

[0024] As a further improvement of the technical solution, the internal quality detection model specifically includes:

[0025] Extract a large number of labeled X-ray images from a public database, and perform feature engineering in S2 on the labeled X-ray images to generate labeled X-ray feature data;

[0026] The convolutional neural network algorithm is used to train the labeled X-ray feature data. The training goal is to distinguish normal and abnormal samples. During the training process, the cross entropy loss function and optimization algorithm are used to update the parameters until the model converges.

[0027] The trained internal quality inspection model takes the X-ray feature data in S2 as input to output the inspection results of the internal product quality.

[0028] As a further improvement of the technical solution, the specific process of the external quality detection model is as follows:

[0029] Extract a large number of labeled visible light images from a public database, and perform feature engineering in S2 on the labeled visible light images to generate labeled visible light images;

[0030] A convolutional neural network algorithm different from the internal quality detection model is used to train the labeled X-ray feature data. During the training process, the cross entropy loss function and optimization algorithm are also used to update the parameters until the external quality detection model converges.

[0031] The trained external quality inspection model takes the visible light feature data in S2 as input to output the inspection results of the external product quality.

[0032] As a further improvement of the technical solution, when the internal product quality detection result is abnormal, the abnormal position corresponding to the X-ray feature data in the abnormal detection result is extracted, and the visible light feature data is enhanced according to the abnormal position to generate a visible light abnormal image, which specifically includes:

[0033] According to the spatial and structural information in the feature map of the last convolutional layer in the internal quality detection model, the area that contributes the most to the anomaly detection result is determined as the anomaly location by back-propagating the gradient size;

[0034] According to the determined abnormal position, the corresponding area in the visible light feature data is highlighted by superimposing a semi-transparent color mask on the visible light feature data;

[0035] Perform Gaussian blur processing on the non-abnormal positions in the visible light feature data, apply the Gaussian blur convolution kernel, perform convolution operation on the non-abnormal positions, and blur these areas;

[0036] The visible light feature data after highlight processing and Gaussian blur processing are integrated to generate a visible light abnormal image.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The quality inspection method for packaging products based on the fusion of X-rays and visible light combines two different imaging technologies, makes full use of the spatial and structural information in the image, and realizes efficient and accurate quality inspection; X-ray images provide internal structural information of packaging products, can identify internal defects and anomalies, and the feature map extracted by convolutional neural network contains detailed spatial layout and structural relationship. The visible light image provides the visual features of the product's exterior, and the residual network extracts the external surface details to ensure the appearance is lossless; the feature map generated by this method through feature engineering preprocessing and feature extraction not only retains the spatial layout of the image, but also captures the key structural information; when inspecting the internal quality, the spatial and structural information in the X-ray feature map is used to determine the abnormal position, and the visible light feature data is enhanced when necessary to generate a visible light abnormal image; finally, the abnormal position determined by the back-propagation gradient enables the detection model to accurately locate and analyze the abnormal area, thereby improving the reliability and accuracy of the detection results. Therefore, this fusion method makes full use of the spatial and structural information, realizes the comprehensive inspection of the internal and external quality of packaging products, and significantly improves the effect and efficiency of quality control.

[0039] 2. The quality inspection method for packaging products based on the fusion of X-rays and visible light performs the same feature engineering processing on X-ray images and visible light images, which not only realizes preprocessing and feature extraction, but also makes subsequent enhancement processing feasible; and when training internal quality inspection models and external quality inspection models, feature engineering is used to unify different data formats and structures. This consistent feature engineering processing includes scale adjustment, normalization and noise processing to ensure that the data structure of different types of images is consistent, which facilitates model training and detection. This unified processing method not only simplifies the complexity of the algorithm, but also improves the efficiency and accuracy of feature extraction. After feature engineering processing, the extracted feature data can better reflect the spatial and structural information of the image, provide high-quality input data for internal and external quality inspection models, and enhance the detection effect and robustness of the model.

[0040] 3. Overall, the quality inspection method for packaging products based on the fusion of X-rays and visible light, firstly, uses the residual network model for feature extraction, which solves the gradient vanishing problem in the deep network, making it have stronger feature extraction capabilities when processing complex images; secondly, the convolutional neural network model is used for internal and external quality inspection, and the high precision and high reliability of the model are ensured through training with large-scale labeled data; in addition, through the combination of X-ray images and visible light images, it can comprehensively cover the internal and external quality inspection of the product, reducing the limitations of a single inspection method; especially in anomaly detection, the abnormal position is determined by back propagation and the visible light image is enhanced, which greatly improves the accuracy and interpretability of anomaly detection; overall, this quality inspection scheme that integrates multiple technologies and methods not only improves the comprehensiveness and accuracy of detection, but also significantly improves the detection efficiency, providing strong technical support for industrial product quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the method steps of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] See also Figure 1 The present invention provides a technical solution: a quality inspection method for packaged products based on the fusion of X-rays and visible light, comprising the following method steps:

[0044] S1, collect X-ray images and visible light images;

[0045] S2. Perform the same feature engineering processing on the X-ray image and the visible light image to generate X-ray feature data and visible light feature data, wherein the feature engineering processed data includes spatial and structural information;

[0046] S3. Using the internal quality inspection model to inspect the internal product quality of the X-ray feature data, wherein:

[0047] If the internal product quality test result is normal, the external quality test model is used to perform external product quality test on the visible light feature data, wherein: when the external product quality test is abnormal, the product external abnormal result is output; when the external product quality test is normal, the product internal and external normal results are output;

[0048] If the internal product quality test result is abnormal, based on the spatial and structural information of the X-ray feature data, the abnormal position corresponding to the X-ray feature data in the abnormal detection result is extracted, and the visible light feature data is enhanced according to the abnormal position to generate a visible light abnormal image; and the external product quality is tested on the visible light abnormal image using the external quality detection model, wherein: when the external product quality test is abnormal, the internal and external abnormal results of the product are output; when the external product quality test is normal, the internal abnormal result of the product is output;

[0049] The internal quality detection model and the external quality detection model are composed of different convolutional neural network models.

[0050] Specifically include:

[0051] In S1, an X-ray device is used to scan the packaged product to be inspected to obtain an X-ray image of the internal structure; at the same time, a visible light camera is used to photograph the external surface of the packaged product to obtain a visible light image; that is, the X-ray image is an internal structure image of the packaged product, and the visible light image is an external structure image of the packaged product.

[0052] In S2, the same feature engineering processing is performed on the X-ray image and the visible light image to generate X-ray feature data and visible light feature data. The feature engineering includes preprocessing and feature extraction. The preprocessing specifically includes:

[0053] Resizing: resize all images to a uniform size by interpolating to the target size;

[0054] Normalization: adjust the image pixel values ​​to the range of (0,1) by subtracting the minimum value of the image pixel value from each pixel value and then dividing it by the range of the original image pixel value (maximum value minus minimum value);

[0055] Noise processing: Remove noise from the image, apply the Gaussian kernel function to the image, and perform convolution operations. The Gaussian kernel parameters (standard deviation and kernel size) determine the smoothing effect.

[0056] Feature extraction includes X-ray image feature extraction and visible light image feature extraction. X-ray image feature extraction specifically includes:

[0057] The preprocessed X-ray image is input into the X-ray residual network model. The residual network model effectively solves the gradient vanishing problem in the deep network due to the introduction of residual blocks, so that it has a strong feature extraction ability when processing complex images. The residual blocks are connected across layers to capture complex features. The X-ray residual network model is a deep network structure composed of multiple convolution operations, regularization processing and activation operations.

[0058] The convolutional layer output after the last residual block and before the fully connected layer in the last X-ray residual network model is selected as the X-ray feature layer, which contains spatial and structural information, and a feature map is obtained from the X-ray feature layer, and the feature map is used as X-ray feature data.

[0059] Visible light image feature extraction specifically includes:

[0060] Inputting the preprocessed visible light image into a visible light residual network model, which is another residual network model, and the visible light residual network model is composed of a series of convolutional layers, batch normalization layers and activation function layers;

[0061] The output of the convolutional layer after the last residual block and before the fully connected layer in the last visible light residual network model is selected as the visible light feature layer, which contains spatial and structural information. A feature map is obtained from the visible light feature layer, and the feature map is used as visible light feature data.

[0062] The residual block effectively solves the gradient vanishing problem, enabling the network to have stronger feature extraction capabilities when processing complex images; the feature extraction of X-ray images and visible light images uses the same network architecture, which facilitates subsequent enhancement processing.

[0063] In S3, the internal quality detection model is used to detect the internal product quality of the X-ray feature data. The internal quality detection model is a convolutional neural network model, which specifically includes:

[0064] Extract a large number of labeled X-ray images from the public database, and perform feature engineering in S2 on the labeled X-ray images to generate labeled X-ray feature data so that different data structures remain consistent;

[0065] The convolutional neural network algorithm is used to train the labeled X-ray feature data. The training goal is to distinguish normal and abnormal samples so that the internal quality inspection model can accurately detect the quality of internal products. During the training process, the cross entropy loss function and optimization algorithm are used to update the parameters until the model converges.

[0066] The trained internal quality inspection model takes the X-ray feature data in S2 as input to output the inspection results of the internal product quality; where:

[0067] If the internal product quality test result is normal, the external quality test model is used to test the visible light feature data for external product quality. The external quality test model is another convolutional neural network model, which is different from the above internal quality test model. The specific algorithm process of the external quality test model is as follows:

[0068] Extract a large number of labeled visible light images from the public database, and perform feature engineering in S2 on the labeled visible light images to generate labeled visible light images so that different data structures remain consistent;

[0069] A convolutional neural network algorithm different from the internal quality detection model is used to train the labeled X-ray feature data. During the training process, the cross entropy loss function and optimization algorithm are also used to update the parameters until the external quality detection model converges.

[0070] The trained external quality detection model takes the visible light feature data in S2 as input to output the detection result of the external product quality; wherein: when the external product quality detection is abnormal, it indicates that there is an abnormality on the outside of the product, and the product external abnormal result is output; when the external product quality detection is normal, it indicates that the internal and external quality of the product are normal at this time, and the product internal and external normal result is output;

[0071] If the internal product quality test result is abnormal, based on the spatial and structural information of the X-ray feature data, the abnormal position corresponding to the X-ray feature data in the abnormal detection result is extracted, and the visible light feature data is enhanced according to the abnormal position to generate a visible light abnormal image, which specifically includes:

[0072] According to the spatial and structural information in the feature map of the last convolutional layer in the internal quality detection model, the area that contributes the most to the anomaly detection result is determined as the anomaly location by back-propagating the gradient size;

[0073] According to the determined abnormal position, the corresponding area in the visible light feature data is highlighted by superimposing a semi-transparent color mask on the visible light feature data to highlight the area;

[0074] Perform Gaussian blur processing on the non-abnormal positions in the visible light feature data, apply a Gaussian blur convolution kernel (e.g., a kernel size of 5x5 or 7x7), and perform convolution operations on the non-abnormal positions to blur these areas; only retain the original clarity of the abnormal positions to ensure that the abnormal positions are clearly visible in the image, so that they can be analyzed and processed in the subsequent external quality inspection;

[0075] The visible light feature data after highlight processing and Gaussian blur processing are integrated to enhance the visible light feature data and generate a visible light abnormal image.

[0076] The external quality detection model is used to detect the external product quality of the visible light abnormal image, so as to focus on analyzing the external abnormal situations of the product that may be caused by internal abnormal positions. Among them: when the external product quality detection is abnormal, it means that the product is not only abnormal inside, but also abnormal outside, and the internal and external abnormal results of the product are output; when the external product quality detection is normal, it means that the abnormality of the product is caused by the inside, and the internal abnormality result of the product is output.

[0077] Therefore, when conducting internal and external quality inspections, if the internal product quality inspection result is abnormal, the external quality inspection model is used to inspect the external product quality of the visible light abnormal image, so as to focus on analyzing the external abnormalities of the product that may be caused by the internal abnormal position. At this time, when the external product quality inspection is abnormal, it means that the product is not only abnormal inside, but also abnormal outside, and the internal and external abnormal results of the product are output; this process enables the external quality inspection model to more accurately capture external problems caused by internal abnormalities through enhanced processing of visible light feature data. This targeted inspection method ensures the accuracy and effectiveness of the inspection results, and can detect and identify potential quality problems at an early stage, thereby improving the overall level of product quality control.

[0078] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A quality inspection method for packaged products based on the fusion of X-rays and visible light, characterized in that: The method steps are as follows: S1, collect X-ray images and visible light images; S2. Perform feature engineering processing on the X-ray image and the visible light image to generate X-ray feature data and visible light feature data, wherein the feature engineering processed data includes spatial and structural information; S3. Using the internal quality inspection model to inspect the internal product quality of the X-ray feature data, wherein: If the internal product quality test result is normal, the external quality test model is used to perform external product quality test on the visible light feature data, wherein: when the external product quality test is abnormal, the product external abnormal result is output; when the external product quality test is normal, the product internal and external normal results are output; If the internal product quality test result is abnormal, based on the spatial and structural information of the X-ray feature data, the abnormal position corresponding to the X-ray feature data in the abnormal detection result is extracted, and the visible light feature data is enhanced according to the abnormal position to generate a visible light abnormal image; and the external product quality is tested on the visible light abnormal image using the external quality detection model, wherein: when the external product quality test is abnormal, the internal and external abnormal results of the product are output; when the external product quality test is normal, the internal abnormal result of the product is output; The internal quality detection model and the external quality detection model are composed of different convolutional neural network models, and the internal quality detection model specifically includes: Extract a large number of labeled X-ray images from a public database, and perform feature engineering in S2 on the labeled X-ray images to generate labeled X-ray feature data; The convolutional neural network algorithm is used to train the labeled X-ray feature data. The training goal is to distinguish normal and abnormal samples. During the training process, the cross entropy loss function and optimization algorithm are used to update the parameters until the model converges. The trained internal quality inspection model takes the X-ray feature data in S2 as input to output the inspection results of the internal product quality; When the internal product quality detection result is abnormal, the abnormal position corresponding to the X-ray feature data in the abnormal detection result is extracted, and the visible light feature data is enhanced according to the abnormal position to generate a visible light abnormal image, which specifically includes: According to the spatial and structural information in the feature map of the last convolutional layer in the internal quality detection model, the area that contributes the most to the anomaly detection result is determined as the anomaly location by back-propagating the gradient size; According to the determined abnormal position, the corresponding area in the visible light feature data is highlighted by superimposing a semi-transparent color mask on the visible light feature data; Perform Gaussian blur processing on the non-abnormal positions in the visible light feature data, apply the Gaussian blur convolution kernel, perform convolution operation on the non-abnormal positions, and blur these areas; The visible light feature data after highlight processing and Gaussian blur processing are integrated to generate a visible light abnormal image.

2. The method for quality inspection of packaged products based on the fusion of X-rays and visible light according to claim 1, characterized in that: The X-ray image is an internal structure image obtained by scanning the packaged product to be inspected by an X-ray device; the visible light image is an external structure image obtained by photographing the external surface of the packaged product by a visible light camera.

3. The method for quality inspection of packaged products based on the fusion of X-rays and visible light according to claim 1, characterized in that: The feature engineering includes preprocessing, which specifically includes: Resizing: resize all images to a uniform size by interpolating to the target size; Normalization: adjust the image pixel values ​​to the range of (0,1) by subtracting the minimum value of the image pixel value from each pixel value and then dividing it by the range of the original image pixel value; Noise processing: Remove noise from the image, apply a Gaussian kernel function to the image, and perform a convolution operation.

4. The method for quality inspection of packaged products based on the fusion of X-rays and visible light according to claim 3, characterized in that: The feature engineering includes feature extraction, which includes X-ray image feature extraction and visible light image feature extraction. X-ray image feature extraction specifically includes: The preprocessed X-ray image is input into an X-ray residual network model, which includes residual blocks, which are connected across layers to capture complex features; the X-ray residual network model has a deep network structure consisting of multiple convolution operations, regularization processing and activation operations; Select the convolutional layer output after the last residual block and before the fully connected layer in the last X-ray residual network model as the X-ray feature layer, which contains spatial and structural information, obtain a feature map from the X-ray feature layer, and use the feature map as X-ray feature data; Visible light image feature extraction specifically includes: Inputting the preprocessed visible light image into a visible light residual network model, which is another residual network model, and the visible light residual network model is composed of a series of convolutional layers, batch normalization layers and activation function layers; The output of the convolutional layer after the last residual block and before the fully connected layer in the last visible light residual network model is selected as the visible light feature layer, which contains spatial and structural information. A feature map is obtained from the visible light feature layer, and the feature map is used as visible light feature data.

5. The method for quality inspection of packaged products based on the fusion of X-rays and visible light according to claim 1, characterized in that: The specific process of the external quality detection model is as follows: Extract a large number of labeled visible light images from a public database, and perform feature engineering in S2 on the labeled visible light images to generate labeled visible light images; A convolutional neural network algorithm is trained on labeled X-ray feature data, which is different from the internal quality detection model. During the training process, the cross entropy loss function and optimization algorithm are also used to update the parameters until the external quality detection model converges; The trained external quality inspection model takes the visible light feature data in S2 as input to output the inspection results of the external product quality.

Images