Food package air leakage detection method based on multi-dimensional visual angle visual converter
Through multi-dimensional viewing converter and deep learning algorithm, the problem of time and cost in food packaging leak detection is solved, and the rapid and accurate identification of leaked packaging bags is achieved, with an identification rate of 17 milliseconds/bag, with an accuracy rate of 97.5%. , significantly reduces time costs and avoids human errors.
Patent Information
- Application Number
- CN202410922395.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art has problems of time-consuming, high cost and human error in the detection of air leakage in food packaging. Especially for packaging bags of different shapes and volumes, the critical voltage needs to be adjusted, and the chemical sensor is sensitive to humid environments and is difficult to adapt to the continuous assembly line. Operation.
Using a deep learning algorithm based on a multi-dimensional viewing perspective converter, three-view photos are collected through ordinary cameras, multi-dimensional viewing angle fusion vision converter model is established, outline features of packaging bags are extracted and fused, and morphological features are analyzed using BiSeNet network and Transformer encoder. , quickly identify air leak packaging bags.
It realizes rapid and accurate identification of leaky packaging bags, reduces the cost of testing time, improves the identification accuracy and economic benefits, and avoids the influence of human subjective factors. The identification rate is 17 milliseconds/bag, with an accuracy rate of 97.5%.
Smart Images

Figure CN118967557A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of food industry packaging detection, in particular to a food packaging air leakage detection method based on a multi-dimensional viewing angle visual converter. Background Art
[0002] In recent years, artificial intelligence has been gradually applied to various fields as a hot technology. Intelligent technologies such as machine learning and deep learning have been widely used in the inspection and grading of agricultural products, screening of the best fruit shape, and inspection of snack quality. At the same time, the demand for intelligent online non-destructive testing has increased year by year. However, there is little research on online non-destructive testing of leaking food modified atmosphere pillow packaging based on deep learning technology.
[0003] Traditional methods for online nondestructive testing of leaking food modified atmosphere pillow-shaped packaging include manual inspection, dielectric barrier discharge, vacuum electroacoustic and disposable chemical sensor methods. The average rate of the manual inspection method is 2.5 seconds per package. The dielectric barrier discharge method refers to treating the package to be inspected as an insulating medium, placing it in a high-voltage electric field parallel plate electrode, observing the dielectric barrier discharge phenomenon occurring in the circuit based on the principle that the insulation strength of the leaking package is low, collecting the discharge waveform through an oscilloscope, and completing the detection of leaking packages by combining wavelet packet transform, empirical mode decomposition and overall average empirical mode decomposition. The vacuum electroacoustic method refers to placing the package to be tested in a vacuum container, using the principle of sound emission from package leakage in a vacuum environment, and using a high-sensitivity microphone to collect the airflow sound pressure, thereby detecting the leaking package. This method can determine the size of the leak through sound pressure level and audio spectrum analysis. The disposable chemical sensor method refers to implanting a disposable oxygen sensor or carbon dioxide sensor in the package, and judging the leakage of the packaging bag by observing the color change of this chemical sensor.
[0004] Traditional existing technologies have many shortcomings. Manual inspection increases production costs, and high-intensity assembly line operations make it difficult to avoid human errors. Although the dielectric barrier discharge method can replace manual inspection to a certain extent, the critical voltage needs to be readjusted to test packaging bags of different shapes and volumes, and improper adjustment may break through the bag body, which is not conducive to non-destructive testing. The vacuum electroacoustic method requires repeated pressurization and depressurization of the container, which is not conducive to continuous assembly line operation. The chemical materials of disposable chemical sensors are sensitive to humid environments, so there are high requirements for the storage conditions of the materials. Even if the additional packaging costs are not considered, the storage costs of sensor materials are difficult to ignore.
[0005] Therefore, there is an urgent need for a food packaging leakage detection method based on a multi-dimensional perspective visual converter to solve the above problems. Summary of the invention
[0006] To solve the above problems, the present application provides a food packaging leakage detection method based on a multi-dimensional perspective visual converter. Through a deep learning algorithm, it can quickly and accurately identify leaking packaging bags, solving the shortcomings of traditional methods such as long time consumption and high cost, and greatly reducing the time cost of testing.
[0007] In the present application, a method for detecting air leakage in food packaging based on a multi-dimensional viewing angle visual converter is provided, comprising the following steps:
[0008] S1. Using a common camera to collect three-view photos of the appearance of the controlled atmosphere pillow-shaped packaging bag, and processing the three-view photos of the controlled atmosphere pillow-shaped packaging bag to obtain sample image data, wherein the sample image data includes front image data, side image data, and top view image data;
[0009] S2. Establish a multi-dimensional perspective fusion visual converter model;
[0010] S3, feeding the front image data, side image data and top view image data of the packaging bag into corresponding channels of the multi-dimensional perspective fusion visual converter model for image recording to obtain recording data;
[0011] S4, extracting the contour of the packaging bag in the input data to obtain target contour data;
[0012] S5, the multi-dimensional perspective fusion visual converter model extracts the contour features of the target contour data of different channels, and fuses the contour features of the contour data of different channels to obtain a fusion result;
[0013] S6. Determine whether the packaging bag is leaking or not based on the fusion result, and then obtain the morphological information of the food packaging bag to obtain the classification result.
[0014] Preferably, the specific contents of establishing the multi-dimensional perspective fusion visual converter model in S2 are as follows:
[0015] Establish a perspective fusion visual converter model;
[0016] The view fusion visual converter model is added with two channels to obtain a multi-dimensional view fusion visual converter model;
[0017] Establish the feature fusion layer of the multi-dimensional view fusion visual converter model;
[0018] Establish the self-attention mechanism and multi-layer perception mechanism of the multi-dimensional perspective fusion visual converter model;
[0019] Building slice and position encoding embedding modules;
[0020] Incorporate the BiSeNet network into the multi-dimensional view fusion visual transformer model.
[0021] Preferably, the expression of the feature fusion layer for establishing the multi-dimensional perspective fusion visual converter model is:
[0022] F=P fi ∪P si ∪P ti ,i∈(0,N)
[0023] Where F is the fused feature set, i is the corresponding food packaging number, Pf, Ps and Pt are the packaging morphological features of the same package i under different viewing angles (front view, side view, and top view).
[0024] Preferably, the specific content of extracting the contour of the packaging bag in the input data to obtain the target contour data in S4 includes:
[0025] Use the BiSeNet network to extract the contour of the packaging bag in the input data to obtain the initial contour data;
[0026] The initial contour data is scaled to obtain the target contour data.
[0027] Preferably, the expression for obtaining the initial contour data by extracting the contour of the packaging bag in the input data using the BiSeNet network is:
[0028] Z=BiSeNet(P j )={x0,y0;x1,y1;...;x N ,y N};
[0029] Among them, Pj is the jth picture in the dataset, and Z is a set of closed curves (xi, yi) consisting of N nodes obtained through BiSeNet semantic segmentation, i.e., the contour.
[0030] Preferably, the expression for obtaining contour data by performing variable scaling on the initial contour data is:
[0031]
[0032] Where P is the closed curve point set formed by multiple points on the edge of the figure; LN is the normalization method; W and H are the pixel height and pixel width of the output image respectively; K is the ratio of the contour image to the output image area. Where (xi, yi) is the coordinate in the set Z; zi is the element in the set Z. Finally, the pixel size of the contours of different packaging bags is kept consistent.
[0033] Preferably, the specific content of S5 is:
[0034] The target contour data is segmented by using a BiSeNet network, and the target contour data is transformed to obtain morphological feature data, wherein the morphological feature data includes front view morphological feature data, side view morphological feature data and top view morphological feature data;
[0035] The morphological feature data is divided into three groups of slices in the form of pictures. The three groups of slices correspond to the morphological feature data of the front view, the morphological feature data of the side view and the morphological feature data of the top view, and are processed simultaneously in the form of three parallel channels.
[0036] The three groups of slices enter the slice and position encoding embedding module, and three groups of input arrays with position information are obtained for the Transformer encoder;
[0037] The features of the three groups of input arrays are fused through the feature fusion layer to obtain the fused record.
[0038] Preferably, the fusion result in S6 is judged using a multi-layer perception mechanism.
[0039] In summary, compared with the traditional food industry packaging detection technology, the food packaging leakage detection method based on the multi-dimensional visual converter of the present invention can quickly and accurately identify leaking packaging bags through deep learning algorithms, which solves the shortcomings of other methods such as long time consumption and high cost, and uses computer artificial intelligence algorithms to avoid the influence of human subjective factors, significantly improve economic benefits, greatly reduce the time cost of testing, and quickly and accurately identify leaking packaging automatically. The final recognition accuracy rate reached 97.5%, and the recognition rate was 17 milliseconds / bag.
[0040] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 The present invention is a flow chart of a method for detecting air leakage in food packaging based on a multi-dimensional viewing angle visual converter. DETAILED DESCRIPTION
[0042] The technical solution of the present invention is further described below by means of the accompanying drawings and embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values described in these embodiments do not limit the scope of the present application.
[0043] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.
[0044] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.
[0045] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0046] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.
[0047] like Figure 1 As shown, in this application, a food packaging leakage detection method based on a multi-dimensional visual converter is provided, which can quickly, simply and efficiently replace manual leakage detection and be applied to factory production. Figure 1 The model diagram of this method. Among them: BiSeNet is the contour extraction network, Morphology Extraction is the morphology extraction process, Linear Projection of Flattened Patches is the linear mapping layer of image slices, Position + Patch Embedding is the image slice + position encoding embedding, Transformer Encoder is the Transformer encoder, MLP HEAD is the multi-layer perceptron, and Classification is the classification result, that is, leakage or no leakage.
[0048] The food packaging air leakage method comprises the following steps:
[0049] S1. Use an ordinary camera to collect three-view photos of the appearance of the controlled atmosphere pillow-shaped packaging bag, and process the three-view photos of the controlled atmosphere pillow-shaped packaging bag to obtain sample image data, wherein the sample image data includes front image data, side image data and top view image data.
[0050] S2. Establish a multi-dimensional perspective fusion visual converter model;
[0051] Among them, the content of establishing the multi-dimensional perspective fusion visual converter model in S2 is specifically as follows:
[0052] Establish a perspective fusion visual converter model;
[0053] The view fusion visual converter model is added with two channels to obtain a multi-dimensional view fusion visual converter model;
[0054] Establish the feature fusion layer of the multi-dimensional view fusion visual converter model;
[0055] Establish the self-attention mechanism and multi-layer perception mechanism of the multi-dimensional perspective fusion visual converter model;
[0056] Building slice and position encoding embedding modules;
[0057] Incorporate the BiSeNet network into the multi-dimensional view fusion visual transformer model.
[0058] It can be understood that the expression of the feature fusion layer of the multi-dimensional perspective fusion visual converter model is: F = P fi ∪P si ∪P ti ,i∈(0,N).
[0059] Where F is the fused feature set; i is the corresponding food packaging number; Pf, Ps and Pt are the packaging morphological features of the same package i under different viewing angles (front view, side view, and top view).
[0060] S3. Feed the front image data, side image data and top view image data of the packaging bag into corresponding channels of the multi-dimensional perspective fusion visual converter model for image input to obtain input data.
[0061] In order to eliminate the interference of patterns and textures on the surface of packaging bags, improve the generalization ability of the VisionTransformer model, and guide the main model to learn the packaging shape, in the first step, this model uses the BiSeNet network to extract the contour of the packaging bag in the image to obtain the morphological information of the food packaging bag, so that the model can stably identify leaking and deformed packaging in complex situations.
[0062] S4, extracting the contour of the packaging bag in the input data to obtain target contour data;
[0063] The specific contents of extracting the contour of the packaging bag in the input data to obtain the target contour data in S4 include:
[0064] Use the BiSeNet network to extract the contour of the packaging bag in the input data to obtain the initial contour data;
[0065] The initial contour data is scaled to obtain the target contour data.
[0066] BiSeNet is an efficient real-time semantic bilateral segmentation network, which aims to achieve fast and accurate semantic segmentation on high-resolution images. It can accurately extract the contour of the packaging bag in the image. The expression of the initial contour data obtained by using the BiSeNet network to extract the contour of the packaging bag in the input data is:
[0067] Z=BiSeNet(P j )={x0,y0;x1,y1;...;x N ,yN};
[0068] Among them, Pj is the jth picture in the dataset, and Z is a set of closed curves (xi, yi) consisting of N nodes obtained through BiSeNet semantic segmentation, i.e., the contour.
[0069] Furthermore, the expression of the contour data obtained by variable scaling of the initial contour data is:
[0070]
[0071] Where P is the closed curve point set formed by multiple points on the edge of the figure; LN is the normalization method; W and H are the pixel height and pixel width of the output image respectively; K is the ratio of the contour image to the output image area. Where (xi, yi) is the coordinate in the set Z; zi is the element in the set Z. Finally, the pixel size of the contours of different packaging bags is kept consistent.
[0072] S5, the multi-dimensional perspective fusion visual converter model extracts the contour features of the target contour data of different channels, and fuses the contour features of the contour data of different channels to obtain the fusion result;
[0073] The specific contents of S5 are:
[0074] The target contour data is segmented by using a BiSeNet network, and the target contour data is transformed to obtain morphological feature data, wherein the morphological feature data includes front view morphological feature data, side view morphological feature data and top view morphological feature data;
[0075] The morphological feature data is divided into three groups of slices in the form of pictures. The three groups of slices correspond to the morphological feature data of the front view, the morphological feature data of the side view and the morphological feature data of the top view, and are processed simultaneously in the form of three parallel channels.
[0076] The three groups of slices enter the slice and position encoding embedding module, and three groups of input arrays with position information are obtained for the Transformer encoder;
[0077] The features of the three groups of input arrays are fused through the feature fusion layer to obtain the fused record.
[0078] S6. Determine whether the packaging bag is leaking or not based on the fusion result, and then obtain the morphological information of the food packaging bag to obtain the classification result, wherein the fusion result is judged by a multi-layer perception mechanism.
[0079] The specific implementation process is that this method uses VisionTransformer (ViT) as the basic model. The processed packaging bag image will be divided into fixed-size slices (patches), and then these slices are linearly embedded into one-dimensional vectors, and then these vectors are used as the input of the Transformer. The local and global features in the image are captured through the self-attention mechanism and position encoding, thereby achieving efficient image classification.
[0080] The main logical contents of ViT are:
[0081]
[0082] The encoding method of the Transformer encoder is expressed as follows. The data is first normalized (layernormalization, LN) and y is the feature obtained by normalizing the slice:
[0083] z′ l =MSA(LN(z l-1 ))+z l-1 .
[0084] z l =MLP(LN(z′ l ))+z′ l .
[0085] Where z0 is the set of slices that the image is divided into, xclass is the start of the set, xipE is the i-th slice in the image, Epos is the position information carried by each slice, p is the height and width of the slice, C is the number of channels of the image, and D is the number of slices after the image is segmented. The multi-headed self-attention mechanism (MSA) and the multi-layer perceptron (MLP) are used to calculate the output data z of the l-1 layer. l-1 Alternate encoding, z′ l Nested loops as intermediate variables.
[0086] Then, the single-dimensional ViT model is improved into a multidimensional model by adding two channels. The feature quantity is expanded from different visual angles to improve the accuracy and stability of the model, and the feature fusion of food modified atmosphere pillow packaging under three perspectives is completed.
[0087] like Figure 1As shown in the figure, the three perspectives of the same packaging bag, namely the front view, side view and top view, are first imported into the contour extraction, and the BiSeNet network is used to segment the food packaging semantics, and then the semantic information is transformed into morphological features to obtain the corresponding morphological data of the three perspectives, and finally it is passed into the Vision Transformer model with multi-dimensional perspective fusion (multi-dimentional fusion ViT, MdF-ViT). Compared with the single-dimensional model, the morphological data is divided into three groups of slices in the form of pictures, and the three groups of slices correspond to the front view, side view and top view of the original package, which are processed simultaneously in three parallel channels.
[0088] The three groups of slices then enter the slice and position encoding embedding module (position+patch embedding), and obtain three groups of Transformer encoder input arrays with position information.
[0089] Here, the Transformer encoder is also expanded to three channels, and the operation is performed in parallel. The three sets of feature data of the three views are fused through the newly added feature fusion layer. The fused features enter the multi-layer perceptron HEAD (MLP HEAD) to obtain the classification result, that is, leakage or no leakage.
[0090] To verify the results of the multi-dimensional perspective fusion model MdF-ViT, the corresponding data sets were used for training and testing, and the ResNet-18 and VGG-16 models were added for comparison. The test results are shown in Table 1. The accuracy of ResNet-18 for leakage detection of three types of packages was 87.50%, 62.50%, and 85.00%, respectively, and the accuracy of VGG-16 for three types of packages was 85.00%, 87.50%, and 90.00%, respectively, which were all lower than the 97.50% of the MdF-ViT model. In addition, the highest F1 value of the first two models was only 90.50%, which was lower than the 97.60%, 97.60%, and 97.40% of MdF-ViT. The recognition accuracy and stability of MdF-ViT are better than those of ResNet-18 and VGG-16.
[0091] Table 1 Verification results of the multi-dimensional perspective fusion ViT model compared with other traditional models
[0092]
[0093] In summary, compared with the traditional food industry packaging detection technology, the food packaging leakage detection method based on the multi-dimensional visual converter of the present invention can quickly and accurately identify leaking packaging bags through deep learning algorithms, which solves the shortcomings of other methods such as long time consumption and high cost, and uses computer artificial intelligence algorithms to avoid the influence of human subjective factors, significantly improve economic benefits, greatly reduce the time cost of testing, and quickly and accurately identify leaking packaging automatically. The final recognition accuracy rate reached 97.5%, and the recognition rate was 17 milliseconds / bag.
[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A food packaging air leakage detection method based on a multi-dimensional visual converter, characterized in that: The following steps are involved: S1. Using a common camera to collect three-view photos of the appearance of the controlled atmosphere pillow-shaped packaging bag, and processing the three-view photos of the controlled atmosphere pillow-shaped packaging bag to obtain sample image data, wherein the sample image data includes front image data, side image data, and top view image data; S2. Establish a multi-dimensional perspective fusion visual converter model; S3, feeding the front image data, side image data and top view image data of the packaging bag into corresponding channels of the multi-dimensional perspective fusion visual converter model for image recording to obtain recording data; S4, extracting the contour of the packaging bag in the input data to obtain target contour data; S5, the multi-dimensional perspective fusion visual converter model extracts the contour features of the target contour data of different channels, and fuses the contour features of the contour data of different channels to obtain the fusion result; S6. Determine whether the packaging bag is leaking or not based on the fusion result, and then obtain the morphological information of the food packaging bag to obtain the classification result.
2. The method for detecting air leakage in food packaging based on a multi-dimensional visual converter according to claim 1, characterized in that: The specific contents of establishing the multi-dimensional perspective fusion visual converter model in S2 are as follows: Establish a perspective fusion visual converter model; The view fusion visual converter model is added with two channels to obtain a multi-dimensional view fusion visual converter model; Establish the feature fusion layer of the multi-dimensional view fusion visual converter model; Establish the self-attention mechanism and multi-layer perception mechanism of the multi-dimensional perspective fusion visual converter model; Building slice and position encoding embedding modules; Incorporate the BiSeNet network into the multi-dimensional view fusion visual transformer model.
3. The method for detecting air leakage in food packaging based on a multi-dimensional visual converter according to claim 2, characterized in that: The expression of the feature fusion layer for building the multi-dimensional view fusion visual converter model is: F=P fi ∪P si ∪P ti ,i∈(0,N) Where F is the fused feature set, i is the corresponding food packaging number, Pf, Ps and Pt are the packaging morphological features of the same package i under different viewing angles (front view, side view, and top view).
4. The method for detecting gas leakage in food packaging based on a multi-dimensional visual converter according to claim 1, characterized in that: The specific contents of extracting the contour of the packaging bag in the input data to obtain the target contour data in S4 include: Use the BiSeNet network to extract the contour of the packaging bag in the input data to obtain the initial contour data; The initial contour data is scaled to obtain the target contour data.
5. The method for detecting gas leakage in food packaging based on a multi-dimensional visual converter according to claim 4, characterized in that: The expression for extracting the contour of the packaging bag in the input data using the BiSeNet network to obtain the initial contour data is: Z=BySet(P j )={x0,y0;x1,y1;...;x N ,y N }; Among them, Pj is the jth picture in the dataset, and Z is a set of closed curves (xi, yi) consisting of N nodes obtained through BiSeNet semantic segmentation, i.e., the contour.
6. The method for detecting gas leakage in food packaging based on a multi-dimensional visual converter according to claim 4, characterized in that: The expression for obtaining contour data by variable scaling of the initial contour data is: Among them, P is the closed curve point set surrounded by multiple points on the edge of the figure, LN is the normalization method, W and H are the pixel height and pixel width of the output image respectively, K is the ratio of the contour image to the output image area, (xi,yi) is the coordinate in the set Z, and zi is the element in the set Z.
7. The method for detecting gas leakage in food packaging based on a multi-dimensional visual converter according to claim 1, characterized in that: The specific contents of S5 are: The target contour data is segmented by using a BiSeNet network, and the target contour data is transformed to obtain morphological feature data, wherein the morphological feature data includes front view morphological feature data, side view morphological feature data and top view morphological feature data; The morphological feature data is divided into three groups of slices in the form of pictures. The three groups of slices correspond to the morphological feature data of the front view, the morphological feature data of the side view and the morphological feature data of the top view, and are processed simultaneously in the form of three parallel channels. The three groups of slices enter the slice and position encoding embedding module, and three groups of input arrays with position information are obtained for the Transformer encoder; The features of the three groups of input arrays are fused through the feature fusion layer to obtain the fused record.
8. The method for detecting gas leakage in food packaging based on a multi-dimensional visual converter according to claim 1, characterized in that: The fusion result described in S6 is judged by using a multi-layer perception mechanism.
