Systems for in-line inspection of seal integrity

Abstract

A process for monitoring the seal quality of a packaged article. In one embodiment, the process for monitoring the seal quality includes sealing a film to form a packaged article and forming a seal area. The seal area is analyzed by a vision system to collect image data of the seal area. The vision inspection engine analyzes the image data to determine the continuity of the seal, the strength of the seal, or both.

Description

Technical Field

[0001] The subject matter disclosed herein relates to systems and processes for inspecting packaged products for quality assurance to ensure the seal integrity of the packaged products. Background Art

[0002] Flexible films are often used in packaging because they can conform to the surface of the product. The quality of the packaging depends on the quality of the film material and the effective closure of any seal. Appropriate sealing quality means the complete fusion of two relative sealing layers. In heat sealing, the long-chain molecules of the sealing layer are allowed to engage to achieve this fusion by applying a specific combination of temperature and pressure within a certain period of time. Sealing defects can damage the seal because of improper combination of sealing parameters or because of water vapor, bubbles, wrinkles or product contamination in the seal. Such sealing defects can be contamination, seal burns, seal gaps and local defects in the seal, cold seal, weak seal, wrinkle, hole, sealing area that are burned. Sealing defects can allow air and microorganisms to penetrate the packaging and make the encapsulated food go bad. They can also negatively affect the sealing appearance, thereby making consumers lose confidence in the integrity of the package.

[0003] Today, the detection of defective seals is still a labor-intensive, offline destructive process. Current seal test methods are described in ASTM F1921 "Standard Test Methods for Hot Seal Strength (Hot Tack) of Thermoplastic Polymers and Blends Comprising the Sealing Surfaces of Flexible Webs", ASTM F88 "Standard Test Method for Seal Strength of Flexible Barrier Materials", ASTM F2054 "Standard Test Method for Burst Testing of Flexible Package Seals Using Internal Air Pressurization Within Restraining Plates", and ASTM F2095 "Standard Test Methods for Pressure Decay Leak Test for Flexible Packages With and Without Restraining Plates". Traditional test methods are destructive and can only be performed offline. In addition, these processes only allow testing of a limited number of packaged items (such as less than 1%) and cause damage to the packaged items.

[0004] It would be desirable to be able to check the seal of a packaged product in a non-destructive process.It would further be desirable to be able to check the seal concurrently with the manufacture of the packaged product or the packaging of the contents within the packaged product before the packaged product is sent away.

[0005] The above discussion is provided for general background information only and is not intended to be used as an aid in determining the scope of the claimed subject matter. Summary of the invention

[0006] A process for monitoring the seal quality of a packaged article. In one embodiment, the process for monitoring the seal quality includes sealing a film to form a packaged article and forming a seal area. The seal area is analyzed by a vision system to collect image data of the seal area. The vision inspection engine analyzes the image data to determine the continuity of the seal, the strength of the seal, or both.

[0007] In one exemplary embodiment, a process for monitoring the sealing quality of a packaged item is disclosed. The process for monitoring the sealing quality of a packaged item includes

[0008] A) sealing the film to itself, another film or a packaging support to form a packaged article by forming at least one sealed area;

[0009] B) acquiring image data of at least one sealing area using a vision system including an image capture device;

[0010] C) evaluating the image data of the seal area using a visual inspection engine to verify the continuity of the seal, the strength of the seal, or both the continuity and strength of the seal area.

[0011] In an exemplary embodiment, the formation of the at least one sealing area is formed by heat generated sealing.

[0012] In an exemplary embodiment, the heat generated seal is a non-linear seal.

[0013] In an exemplary embodiment, the heat generated seal is selected from the group consisting of impulse sealing, ultrasonic sealing, laser sealing, and heat sealing.

[0014] In an exemplary embodiment, the film includes at least one layer comprising a fluorescence-based indicator.

[0015] In an exemplary embodiment, the visual system further includes a blue light bandpass filter.

[0016] In an exemplary embodiment, the vision system is an ultraviolet vision system further comprising an ultraviolet light source.

[0017] In an exemplary embodiment, the ultraviolet vision system further includes a white light source.

[0018] In an exemplary embodiment, the UV vision system illuminates a white light source and a UV light source in a variable pattern.

[0019] In an exemplary embodiment, the image capture device is capable of capturing visible images in the visible spectrum.

[0020] In an exemplary embodiment, the process for monitoring the seal quality of a packaged item further comprises the following steps:

[0021] A) exposing the packaged article to incident radiation to excite the fluorescence-based indicator such that the fluorescence-based indicator fluoresces;

[0022] B) While the indicator is fluorescing, image data of the fluorescence emitted by the packaged item from the seal area is collected.

[0023] In an exemplary embodiment, the fluorescence-based indicator includes at least one selected from the group consisting of an ultraviolet indicator, an infrared indicator, a dye, a pigment, an optical brightener, a fluorescent whitening agent, 2,2'-(2,5-thiophenediyl)bis(5-tert-butylbenzoxazole), hydroxy-4-(p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dye.

[0024] In an exemplary embodiment, the fluorescence-based indicator is present in at least one layer of the film, wherein the fluorescence-based indicator is present at a level of from 0.5 to 1000 ppm based on the weight of the layer.

[0025] In an exemplary embodiment, the fluorescence-based indicator is present in at least one layer of the film, wherein the fluorescence-based indicator is present at a level of from 5 to 10 ppm based on the weight of the layer.

[0026] In an exemplary embodiment, a visual inspection engine includes a computing device including computer-executable instructions configured to determine whether fluorescent electromagnetic energy emitted by an excited fluorescence-based indicator indicates a defective seal.

[0027] In an exemplary embodiment, the computer-executable instructions include an artificial intelligence algorithm.

[0028] In an exemplary embodiment, an artificial intelligence algorithm is generated using a training set of training packaging items prior to analyzing the image data.

[0029] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to a second area of ​​the seal.

[0030] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to an expected level of fluorescence.

[0031] In an exemplary embodiment, determining that the seal is defective includes determining at least one of: (i) the membrane exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to a second area of ​​the seal, (ii) the membrane exhibits a higher or lower fluorescence intensity in the first area of ​​the seal relative to an expected level of fluorescence, or (iii) both (i) and (ii).

[0032] In an exemplary embodiment, the image data is time-lapse thermal imaging data captured at a time subsequent to formation of the at least one sealing area.

[0033] In an exemplary embodiment, the vision system is a thermal imaging vision system that includes an infrared imaging device capable of capturing a temperature distribution based on the amount of infrared radiation emitted from the sealing area.

[0034] In an exemplary embodiment, the image data is acquired between 2 and 30 seconds after forming the at least one sealing area.

[0035] In an exemplary embodiment, the image data is acquired between 5 and 20 seconds after forming the at least one sealing area.

[0036] In an exemplary embodiment, the process further includes an active cooling system.

[0037] In an exemplary embodiment, a visual inspection engine includes a computing device including computer-executable instructions configured to determine whether captured thermal image data of a seal indicates a defective seal.

[0038] In an exemplary embodiment, the computer-executable instructions include an artificial intelligence algorithm.

[0039] In an exemplary embodiment, the process further includes generating an artificial intelligence algorithm using a training set of training packaging items prior to analyzing the image data.

[0040] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower thermal temperature in a first area of ​​the seal relative to a second area of ​​the seal.

[0041] In an exemplary embodiment, the vision system is a photoelastic vision system comprising:

[0042] (i) a first linear polarizer having a polarization direction oriented in a first direction;

[0043] (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction;

[0044] (iii) light source; and

[0045] (iv) Imaging equipment.

[0046] In an exemplary embodiment, the first linear polarizer is integrated into the light source.

[0047] In an exemplary embodiment, the second linear polarizer is integrated into the imaging device.

[0048] In an exemplary embodiment, the light source is a white light source having a wavelength spectrum from 400 nm to 700 nm.

[0049] In an exemplary embodiment, the white light source is a diffuse white light source.

[0050] In an exemplary embodiment, the process further includes positioning a light diffuser between the light source and the packaged item.

[0051] In an exemplary embodiment, the acquisition of the photoelastic image data is performed synchronously with the forming of the packaged article.

[0052] In an exemplary embodiment, the vision system is a first vision system and further includes a second vision system different from the first vision system.

[0053] In an exemplary embodiment, the first vision system is a thermal imaging vision system including an infrared imaging device; and the second vision system is a photoelastic vision system including:

[0054] (i) a first linear polarizer having a polarization direction oriented in a first direction;

[0055] (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction;

[0056] (iii) light source; and

[0057] (iv) Image capture device.

[0058] In an exemplary embodiment, the process further includes a third vision system, the third vision system being an ultraviolet vision system including an ultraviolet light source.

[0059] In an exemplary embodiment, the first vision system is a thermal imaging vision system including an infrared imaging device; and the second vision system is an ultraviolet vision system including an ultraviolet light source.

[0060] In an exemplary embodiment, the process further includes a third vision system, the third vision system is a photoelastic vision system, the photoelastic vision system including:

[0061] (i) a first linear polarizer having a polarization direction oriented in a first direction;

[0062] (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction;

[0063] (iii) light source; and

[0064] (iv) Image capture device.

[0065] In an exemplary embodiment, the first vision system is an ultraviolet vision system including an ultraviolet light source; and the second vision system is a photoelastic vision system including:

[0066] (i) a first linear polarizer having a polarization direction oriented in a first direction;

[0067] (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction;

[0068] (iii) light source; and

[0069] (iv) Image capture device.

[0070] In an exemplary embodiment, the process further includes a third vision system that is a thermal imaging vision system including an infrared imaging device.

[0071] In an exemplary embodiment, a visual inspection engine includes a computing device including computer-executable instructions configured to determine whether captured visual image data of a seal indicates a defective seal.

[0072] In exemplary embodiments, the film has a total free shrink in the machine or cross direction of less than 10% at 85°C.

[0073] In exemplary embodiments, the film has a total free shrink of at least 10% in the machine or cross direction at 85°C.

[0074] In an exemplary embodiment, determining that the seal is defective indicates at least one of a gap in the seal of the packaged item, a wrinkle in the seal of the packaged item, a weak seal, or a cold seal.

[0075] In an exemplary embodiment, the product is a food product.

[0076] In an exemplary embodiment, the food product includes at least one selected from the group consisting of meat and cheese.

[0077] In an exemplary embodiment, the film is a single layer film.

[0078] In exemplary embodiments, the film is a multi-layer film.

[0079] In an exemplary embodiment, the multilayer film includes a functional layer; and wherein the fluorescence-based indicator is present in the functional layer.

[0080] In an exemplary embodiment, the functional layer is one selected from the group consisting of: an oxygen barrier layer, a sensory barrier layer, a moisture barrier layer, a harmful chemical barrier layer, a microbial barrier layer, an acid layer, an acid salt layer, a bacteriocin layer, a bacteriophage layer, a metal layer, a metal salt layer, a natural oil layer, a natural extract layer, a layer containing polyhexamethylene biguanide hydrochloride, a layer containing parabens, a layer containing grafted silane-quaternary ammonium, a layer containing triclosan, and a layer containing zeolite containing silver, copper and / or zinc.

[0081] In an exemplary embodiment, the functional layer is an oxygen barrier layer including at least one selected from the group consisting of vinylidene chloride copolymer, saponified ethylene / vinyl acetate copolymer, polyamide, polyester, oriented polypropylene, and ethylene homopolymer.

[0082] In an exemplary embodiment, the multilayer film includes:

[0083] A) a first layer, which is a first outer film layer configured to function as a heat seal layer;

[0084] B) a second layer, which is a second outer layer configured to function as an outer layer;

[0085] C) a third layer between the first layer and the second layer, wherein the third layer is configured to function as a functional layer;

[0086] D) a fourth layer between the first layer and the third layer, wherein the fourth layer is configured to function as a first bonding layer; and

[0087] E) a fifth layer between the second layer and the third layer, wherein the fifth layer is configured to function as a second bonding layer.

[0088] In an exemplary embodiment, the packaging article is selected from the group consisting of an end seal bag, a side seal bag, a pouch, or a back seam packaging article.

[0089] In an exemplary embodiment of the preceding claim, wherein, in an instance in which a defective seal is detected, the process further includes generating a signal comprising at least one selected from the group consisting of: an alarm, a package marking, an image showing the defective seal, generating a report, identifying a packaged item, and diverting a packaged item.

[0090] In an exemplary embodiment, at least a portion of the vision system is contained within a housing configured to block at least 50% of ambient light.

[0091] In an exemplary embodiment, at least a portion of the vision system is contained within a housing configured to block at least 85% of ambient light.

[0092] In an exemplary embodiment, the film is a single layer film.

[0093] In an exemplary embodiment, the image data is selected from the group consisting of thermal image data, photoelastic image data, and ultraviolet fluorescence emission image data.

[0094] In an exemplary embodiment, in an instance where a defective seal is detected, the process further includes generating a signal comprising at least one selected from the group consisting of: an alarm, a package marking, an image showing the defective seal, generating a report, identifying a packaged item, and diverting a packaged item.

[0095] In an exemplary embodiment, image data of the at least one sealing area is captured by an image capture device at a rate of at least 5 images per second.

[0096] In an exemplary embodiment, image data of the at least one sealing area is captured by the image capture device at a rate of at least 30 images per second.

[0097] In an exemplary embodiment, image data of the at least one sealing area is captured by an image capture device at a rate of at least 100 images per second.

[0098] In an exemplary embodiment, image data of the at least one sealing area is captured by the image capture device at a rate of at least 250 images per second.

[0099] In an exemplary embodiment, image data of at least one seal area is captured by a vision system before the packaged product is sent away.

[0100] In an exemplary embodiment, the visual inspection engine assigns a seal score value to the image data of the seal area.

[0101] In an exemplary embodiment, the visual inspection engine compares the seal score value of the image data of the seal area to a threshold value.

[0102] In an exemplary embodiment, if the seal score value exceeds a threshold, the visual inspection engine sends instructions to a seal defect mechanism.

[0103] In an exemplary embodiment, the visual inspection engine assigns a confidence level to the seal score value.

[0104] In one exemplary embodiment, a system for detecting a defective seal of a packaged item is disclosed. The system for detecting a defective seal of a packaged item includes

[0105] A) a sealing mechanism configured to seal the film to itself, another film, or a packaging support to form a packaged article by forming at least one sealed area;

[0106] B) a vision system comprising an image capture device configured to acquire image data of at least one sealing area using the vision system;

[0107] C) evaluating the image data of the seal area using a visual inspection engine to verify the continuity of the seal, the strength of the seal, or both the continuity and strength of the seal area.

[0108] In an exemplary embodiment, the sealing mechanism forms a heat seal.

[0109] In an exemplary embodiment, the film includes at least one layer comprising a fluorescence-based indicator.

[0110] In an exemplary embodiment, the system further includes a blue light bandpass filter.

[0111] In an exemplary embodiment, the vision system is an ultraviolet vision system further comprising an ultraviolet light source.

[0112] In an exemplary embodiment, the system further comprises:

[0113] A) exposing the packaged article to incident radiation to excite the fluorescence-based indicator such that the fluorescence-based indicator fluoresces;

[0114] B) While the indicator is fluorescing, image data of the fluorescence emitted by the packaged item from the seal area is collected.

[0115] In an exemplary embodiment, the fluorescence-based indicator includes at least one selected from the group consisting of an ultraviolet indicator, an infrared indicator, a dye, a pigment, an optical brightener, a fluorescent whitening agent, 2,2'-(2,5-thiophenediyl)bis(5-tert-butylbenzoxazole), hydroxy-4-(p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dye.

[0116] In an exemplary embodiment, a visual inspection engine includes a computing device including computer-executable instructions configured to determine whether fluorescent electromagnetic energy emitted by an excited fluorescence-based indicator indicates a defective seal.

[0117] In an exemplary embodiment, the computer-executable instructions include an artificial intelligence algorithm.

[0118] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to a second area of ​​the seal.

[0119] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to an expected level of fluorescence.

[0120] In an exemplary embodiment, determining that the seal is defective includes determining at least one of: (i) the membrane exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to a second area of ​​the seal, (ii) the membrane exhibits a higher or lower fluorescence intensity in the first area of ​​the seal relative to an expected level of fluorescence, or (iii) both (i) and (ii).

[0121] In an exemplary embodiment, the image data is time-lapse thermal imaging data captured at a time subsequent to formation of the at least one sealing area.

[0122] In an exemplary embodiment, the vision system is a thermal imaging vision system that includes an infrared imaging device capable of capturing a temperature distribution based on the amount of infrared radiation emitted from the sealing area.

[0123] In an exemplary embodiment, the image data is acquired between 2 and 30 seconds after forming the at least one sealing area.

[0124] In an exemplary embodiment, a visual inspection engine includes a computing device including computer-executable instructions configured to determine whether captured thermal image data of a seal indicates a defective seal.

[0125] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower thermal temperature in a first area of ​​the seal relative to a second area of ​​the seal.

[0126] In an exemplary embodiment, the vision system is a photoelastic vision system comprising:

[0127] (i) a first linear polarizer having a polarization direction oriented in a first direction;

[0128] (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction;

[0129] (iii) light source; and

[0130] (iv) Imaging equipment.

[0131] In an exemplary embodiment, the image data is selected from the group consisting of thermal image data, photoelastic image data, and ultraviolet fluorescence emission image data.

[0132] In an exemplary embodiment, image data of the at least one sealing area is captured by the image capture device at a rate of at least 30 images per second.

[0133] In an exemplary embodiment, the visual inspection engine assigns a seal score value to the image data of the seal area.

[0134] In an exemplary embodiment, the visual inspection engine compares the seal score value of the image data of the seal area to a threshold value.

[0135] In an exemplary embodiment, the visual inspection engine assigns a confidence level to the seal score value.

[0136] This brief description of the invention is intended only to provide a brief overview of the subject matter disclosed herein according to one or more illustrative embodiments, and is not intended to be used as a guide to interpreting the claims or to define or limit the scope of the invention, which is limited only by the appended claims. This brief description is provided to introduce a series of illustrative concepts in a simplified form, which are further described in the detailed description below. This brief description is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that address any or all of the shortcomings noted in the background technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0137] In order that the features of the present invention can be understood, the present invention may be described in detail by reference to certain embodiments, some of which are illustrated in the accompanying drawings. However, it will be noted that the accompanying drawings illustrate only certain embodiments of the present invention and are therefore not to be considered as limiting the scope of the present invention, as the scope of the present invention encompasses other equally effective embodiments. The accompanying drawings are not necessarily drawn to scale, and emphasis is generally placed on illustrating the features of certain embodiments of the present invention. In the accompanying drawings, the same numbers are used to indicate the same parts throughout the various views. Therefore, in order to further understand the present invention, reference may be made to the following detailed description read in conjunction with the accompanying drawings, in which:

[0138] Figure 1 is a schematic diagram of a web production process for extruding an endless web to produce an endless strip according to some embodiments herein;

[0139] Figure 2 According to some embodiments of the present invention, Figure 1 Schematic diagram of another web production process for converting the endless strip produced in into an endless film tube;

[0140] Figure 3 is a block diagram of an in-line seal defect detection system including a thermal vision system of an embodiment;

[0141] Figure 4 is a block diagram of an online seal defect detection system including a photoelastic vision system of an embodiment;

[0142] Figure 5 is a block diagram of an online seal defect detection system including an ultraviolet vision system of an embodiment;

[0143] Figure 6 is a block diagram of an online seal defect detection system including a thermal vision system, a photoelastic vision system, and an ultraviolet vision system of an embodiment;

[0144] Figure 7is a block diagram of a seal defect detection system including a thermal vision system, a photoelastic vision system, and an ultraviolet vision system of an embodiment;

[0145] Figure 8 is a schematic diagram of a thermal imaging vision system of an embodiment;

[0146] Fig. 9 is a schematic diagram of a photoelastic vision system of an embodiment;

[0147] Fig.10 is a schematic diagram of the principle of operation of a photoelastic vision system according to an embodiment;

[0148] Fig.11 is a schematic diagram of an ultraviolet vision system of an embodiment;

[0149] Fig.12 is a block diagram of a visual system according to an embodiment;

[0150] Fig.13 is a block diagram of a system for evaluating the integrity of a seal according to an embodiment;

[0151] Fig.14 is a block diagram of a system for evaluating the integrity of a seal according to an embodiment;

[0152] Fig.15 is a flow chart of an algorithm according to an embodiment;

[0153] Fig.16 is a flow chart of a method of developing a trained image classification model according to an embodiment;

[0154] Fig.17 is a flow chart of a method for developing a trained image classification model based on multiple parameters according to an embodiment;

[0155] Fig.18 is a flow chart of a method for developing a trained image classification model based on multiple parameters according to an embodiment;

[0156] Fig.19 is a flow chart of a method for an image classification system that both trains a model to classify a seal and applies an artificial intelligence algorithm to classify the state of the seal according to an embodiment;

[0157] Fig. 20 is a schematic diagram of a neural network according to an embodiment, the neural network being a multi-layer neural network;

[0158] Fig.21 is a flow chart of a method for classifying a state of a seal according to an embodiment;

[0159] Fig. 22 is a thermographic image showing an acceptable seal;

[0160] Fig.23 is an image view showing a thermographic image of a defective seal;

[0161] Fig.24 is an image view showing a thermographic image of a defective seal;

[0162] Fig.25 is an image view of a thermographic image showing an acceptable seal;

[0163] Fig.26 is an image view of a thermographic image showing an acceptable seal;

[0164] Fig. 27 is an image view showing a thermographic image of a defective sealing area;

[0165] Fig.28 is an image view showing a thermographic image of a defective sealing area;

[0166] Fig.29 is an image view of a photoelastic image showing an acceptable seal;

[0167] Fig.30 is an image view showing a photoelastic image of a defective seal;

[0168] Fig.31 is an image view showing a photoelastic image of a defective seal; and

[0169] Fig.32 is an image view showing a UV image of a defective seal. DETAILED DESCRIPTION

[0170] Example embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which some but not all embodiments are shown. In fact, embodiments may take many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure will satisfy applicable legal requirements. The same reference numerals refer to the same elements throughout. According to some example embodiments, the terms "data," "content," "information," and similar terms may be used interchangeably to refer to data that can be transmitted, received, manipulated, and / or stored. In addition, the term "exemplary," as may be used herein, is not provided to convey any qualitative assessment, but rather, is merely used to convey an illustration of an example. Therefore, the use of any such terms should not be viewed as limiting the spirit and scope of embodiments of the present invention.

[0171] refer to Figure 1 to Figure 2, showing an exemplary embodiment of forming a packaging article. In some embodiments, the packaging article can be made using any known method, as long as the indicator as described herein is incorporated into the packaging article. In some embodiments, solid polymer beads (not shown) are supplied to multiple extruders 28 (for simplicity, only one extruder is shown). Inside the extruder 28, the polymer beads are pushed, melted and degassed, after which the resulting bubble-free melt is pushed into a die head 30 and extruded through an annular die, thereby forming an annular strip 32.

[0172] After being cooled and quenched by water spray from cooling ring 34, annular strip 32 is flattened into a lay-flat configuration by nip rollers 36. Annular strip 32 in the lay-flat configuration then passes through irradiation chamber 38 surrounded by shield 40, where annular strip 32 is irradiated with high energy electrons (i.e., ionizing radiation) from core transformer accelerator 42. Annular strip 32 is guided through irradiation chamber 38 on rollers 44.

[0173] After irradiation, the irradiated annular strip 46 is directed through a pre-coating nip roller 48, after which the irradiated annular strip 46 is slightly inflated, thereby creating trapped bubbles 50. The irradiated annular strip 46 is not significantly drawn longitudinally at the trapped bubbles 50 because the surface speed of the post-coating nip roller 52 is approximately the same as the surface speed of the pre-coating nip roller 48. In addition, the irradiated strip 46 is only inflated enough to place the annular strip in a substantially circular configuration without significant transverse orientation, i.e., without transverse stretching.

[0174] The irradiated annular strip 46 slightly inflated by the bubbles 50 passes through a vacuum chamber 54 and is thereafter pushed through a coating die 56. An annular coating stream 58 is melt extruded from the coating die 56 and is coated onto the inflated irradiated annular strip 46 to form a coated annular strip 60. The coating stream 58 includes an O2 barrier layer made of PVDC and additional layers, all of which are not permeable to ionizing radiation. Further details of the coating steps described above are generally as set forth in U.S. Pat. No. 4,278,738 to BRAX et al., which is incorporated herein by reference in its entirety. After irradiation and coating, the coated annular strip 60 is wound onto a winding roll 62.

[0175] After that, go to Figure 2, in the second stage in the process of making the desired heat shrinkable film tube, the winding roll 62 is installed as an unwinding roll 64. The coated endless strip 60 is unwound from the unwinding roll 64 and passes through a guide roll 66, after which the coated endless strip 60 enters a hot water bath 68 containing hot water 70. The coated tubular film 60, still in a lay-flat configuration, is immersed in the hot water 70 for a sufficient time to allow the endless strip 60 to reach its softening point, i.e., the desired temperature for biaxial orientation when the coated endless strip is in a solid state.

[0176] Thereafter, the coated endless strip 60 is guided through the nip roller 72, and the bubble 74 is blown, thereby solid-state stretching the coated endless strip 60 transversely. In addition, when being blown (i.e., stretched transversely), the nip roller 76 pulls the endless strip 60 in the longitudinal direction because the nip roller 76 has a surface speed higher than the surface speed of the nip roller 72. As a result of the transverse stretching and longitudinal drawing, the endless strip 60 is biaxially oriented in the solid state to form a biaxially oriented heat-shrinkable film tube 78. The heat-shrinkable film tube 78 is stretched transversely and drawn longitudinally. When the bubble 74 is held between the pair of nip rollers 72 and 76, the resulting blown film tube 78 is flattened into a flat configuration by a roller 80. Thereafter, the film tube 78 in the flat configuration is conveyed through the nip roller 76 and across the guide roller 82, and then is rolled onto the winding roller 84. The idler roller 86 ensures good winding.

[0177] As used herein, the term "film" includes plastic webs, whether it is a film, a bag or a sheet. A multilayer film as disclosed herein may have a total thickness of up to 0.15 mm. In an embodiment, the multilayer film includes: A) a first layer, which is a first outer film layer and serves as a heat seal layer; B) a second layer, which is a second outer layer and serves as an outer layer; C) a third film layer between the first layer and the second layer, the third layer serving as a functional layer; D) a fourth film layer between the first layer and the third layer, the fourth layer serving as a first bonding layer; and E) a fifth film layer between the second layer and the third layer, the fifth layer serving as a second bonding layer. In some embodiments, the film may include the first layer, the second layer and the third layer as described above, without the first and second bonding layers. Functional layers include, but are not limited to, oxygen barriers, sensory barriers, moisture barriers, harmful chemical barriers, microbial barriers, acid layers, acid salt layers, bacteriocin layers, bacteriophage layers, metal layers, metal salt layers, natural oil layers, natural extract layers, layers containing polyhexamethylene biguanide hydrochloride, layers containing parabens, layers containing grafted silane-quaternary ammonium, layers containing triclosan, and layers containing zeolites containing silver, copper and / or zinc.

[0178] In an embodiment, the film is adhered to a packaging support such as a tray or another film to form a packaged article. In an embodiment, the packaged article is a kind of group selected from the group consisting of end seal bags, side seal bags, pouches or back seam packaged articles. In certain embodiments, the packaged article may be a film container like a bag (e.g., end seal bags, side seal bags, etc.) having a film surrounding the product. The film may include substantially uniformly distributed fluorescent indicators therein so that the packaged product can indicate the gas in the packaged article on all sides of the product. In certain embodiments, the packaged article may include a rigid or semi-rigid shell, wherein a film covering or a seal encapsulates the shell. In an embodiment, the product placed in the packaged article is a food product. In an embodiment, the food product includes at least one selected from the group consisting of meat and cheese.

[0179] Reference now Figure 3 , Figure 3 31 is a block diagram of an exemplary process flow for detecting defects in the sealing area of ​​a packaged article. The film or packaged article is sealed by a sealing mechanism 31 to produce a sealing area by sealing the film to itself or another packaging support. In some embodiments, the packaged article may include a film, a pouch, a bag, a tubing, a rigid or semi-rigid shell or a tray, wherein the film covering or a seal encapsulates the shell or tray. In some embodiments, the packaged article may include a skin bag. The sealing mechanism 31 produces a sealing area in the film. The sealing mechanism 31 is a heat-generated sealing mechanism, such as heating, impulse sealing, ultrasonic sealing, high-speed heat sealing (sealing at least 100 seals per minute) or laser sealing from a sealing bar. The sealing mechanism seals the film to itself. In the example of the film tubing as described above, the sealing mechanism seals one end of the tubing to produce a packaged article with an opening. It should be understood that the sealing mechanism can also seal the side of the packaged article, and can also seal the final opening after the product has been placed in the packaged article.

[0180] After the sealing of the packaged article, the thermal imaging vision system 33 captures a thermal imaging image of the sealing area. The residual heat from the sealing process performed by the sealing mechanism 31 is retained in the sealing area. This allows the thermal imaging image of the sealing area to be captured. The thermal image is captured between 2 and 40 seconds after sealing. In an embodiment, the thermal image is captured between 5 and 30 seconds after sealing. In an embodiment, the thermal image is captured between 10 and 20 seconds after sealing. In an embodiment, the thermal image is captured about 15 seconds after sealing. In an embodiment, the thermal image is captured about 20 seconds after sealing. It should be understood that active cooling systems such as air blowing, cold air, cold rooms, etc. can be used to reduce the image capture time frame.

[0181] Thermography is a non-contact, non-destructive testing method in which the surface temperature distribution (thermogram) is recorded based on the amount of infrared radiation emitted by the inspected site. The inspected object is thermally excited and the response to this excitation is recorded in time. Active thermography data is three-dimensional, as the temperature of each spatial pixel is evaluated in time. Active thermography is used to detect (sub) surface defects based on differences in thermophysical properties.

[0182] The thermal imaging vision system 33 includes at least one thermal imaging image capture device and a visual inspection engine. In some embodiments, the thermal imaging vision system 33 includes a light source, a plurality of light sources (of the same or different types), a filter, a plurality of image capture devices, or a reflector. In an embodiment, the thermal imaging image capture device is a thermal camera. The light source may be a continuous light source or a flash light source. The light source may be white light, filtered light, diffuse light, infrared light, ultraviolet light, within the visible light spectrum, or outside the visible light spectrum. In some embodiments, the thermal imaging vision system 33 is housed in a housing to provide an isolated environment for the image capture device to capture images. The housing may isolate or reduce the effects of external light sources and contaminants such as dust on the imaging area.

[0183] In one embodiment, the thermal imaging image capture device captures an image of at least one seal of each packaged item. In another embodiment, the thermal imaging image capture device captures at least two images of at least one seal of each packaged item. In another embodiment, the thermal imaging image capture device captures at least four images of at least one seal of each packaged item. In one embodiment, the thermal imaging image capture device captures image data at a frame rate of at least 20 frames per second. In one embodiment, the thermal imaging image capture device captures image data at a frame rate of at least 40 frames per second. In one embodiment, the thermal imaging image capture device captures image data at a frame rate of at least 60 frames per second. Having additional image data of the seal provides additional data and can result in a more reliable determination of the seal quality. In an embodiment, the thermal imaging image capture device transmits the image data in a video format, including but not limited to motion JPEG, MPEG, SEQ, and H.264 formats. In an embodiment, the thermal imaging image capture device captures thermal image data. In another embodiment, the thermal imaging image capture device captures multi-spectral dynamic image data.

[0184] The image data collected by the thermal imaging device is sent to a visual inspection engine. As described in more detail herein, the visual inspection engine is capable of analyzing the image and determining whether the seal is acceptable or whether the seal has defects. The visual inspection engine creates a seal score related to the quality of the seal area. The visual inspection engine analyzes at least one of color gradient changes, light intensity, color intensity, changes in data from adjacent pixels, changes in data from adjacent pixel segments, and combinations thereof to create a seal score. In an embodiment, the visual inspection engine analyzes at least one of color gradient changes, light intensity, color intensity, changes in data from adjacent pixels, changes in data from adjacent pixel segments, and combinations thereof to further create a confidence score. The confidence score provides an assessment of the reliability of a specified seal score. The visual inspection engine may be located locally in the thermal imaging vision system 33, integrated into another part of the system, or remote.

[0185] Still reference Figure 3 , the defect mechanism 35 receives instructions to identify packaged items with defective seals or with a confidence level below a certain threshold. In one embodiment, the defect mechanism pulls the defective packaged items from the production line for offline testing, verification, recycling, repackaging or disposal, thereby allowing only packaged items with proper seals to continue beyond the defect mechanism 35. In an embodiment, the defect mechanism ties, marks or prints at least one mark on the packaged items to identify the packaged items as possibly having defective seals. In an embodiment, the detection of defective seals may cause the defect mechanism to activate an alarm, mark defective packaged items, display images of defective seals or packaged items, display data related to defective seals, and generate reports of defective seals or packaged items. In an embodiment, the defect mechanism removes packaged items with defective seals. The defect mechanism may pause the production line when the packaged items are unloaded, or the defect mechanism may remove the packaged items without pausing the production line. In an embodiment, the defect mechanism includes a vacuum suction cup utilized in combination with a robot arm to automatically remove the packaged items. In another embodiment, the defect mechanism moves the package to a defective production line away from the production line of the packaged items without defective seals.

[0186] In an embodiment, a signal responsive to a defective seal is generated and activated while the web remains in motion (i.e., in real time and online), an alarm, a flag, an intermittent image display, intermittent data, a report of the intermittent data, etc. Alternatively, a signal responsive to a defective seal is generated after production is complete (i.e., offline). The signal responsive to the intermittent may include an electronic message, an email, a data log, and a report.

[0187] It should be understood that the vision system and defect mechanism can be implemented after the formation of the packaged article and before the product is packaged. In another embodiment, the vision system and defect mechanism can be implemented after the packaging and sealing of the product in the packaged article.

[0188] Reference now Figure 4 , Figure 4 It is a block diagram of an exemplary process flow for detecting defects in the sealing area of ​​a packaged article. As described above, the film or packaged article is sealed by a sealing mechanism 31. The sealing mechanism 31 produces a sealing area in the film. The sealing mechanism 31 can be a sealing bar heat seal, an impulse seal, an ultrasonic seal, a high-speed heat seal or a laser seal. In one embodiment, the sealing mechanism produces at least 100 seals per minute. In another embodiment, the sealing mechanism produces at least 300 seals per minute. The sealing mechanism seals the film to itself. In the example of the film tubing as described above, the sealing mechanism seals one end of the tubing to produce a packaged article with an opening. It should be understood that the sealing mechanism can also seal the side of the packaged article, and can also seal the final opening after the product has been placed in the packaged article.

[0189] The film is positioned between two crossed polarizing filters that are positioned orthogonal to each other. The first filter passes light oriented in a first direction. The second filter is oriented orthogonal to the first filter to block all light that passes through the first filter. When a piece of transparent material is positioned between the two filters, the light is rotated. The amount of rotation varies with the type of material and the amount of internal strain within the material. In the event that the light is rotated out of the polarization plane of the second filter, some of the light passes through the second filter.

[0190] After sealing of the packaged article, the photoelastic vision system 43 captures an image of the sealed area. The photoelastic vision system 43 includes at least one photoelastic image capture device, a light source, a first polarized light filter, a second polarized light filter, and a visual inspection engine. In some embodiments, the photoelastic vision system 43 includes multiple light sources (of the same or different types), filters, multiple image capture devices, or reflectors. In an embodiment, the photoelastic image capture device is a high-speed camera. The light source may be a continuous light source or a flash light source. The light source may be white light, filtered light, diffuse light, infrared light, ultraviolet light, within the visible light spectrum, or outside the visible light spectrum. In some embodiments, the photoelastic vision system 43 is housed in a housing to provide an isolated environment for the image capture device to capture the image. The housing may isolate or reduce the effects of external light sources and contaminants such as dust on the imaging area.

[0191] In one embodiment, the photoelastic image capture device captures an image of at least one seal of each packaged item. In another embodiment, the photoelastic image capture device captures at least two images of at least one seal of each packaged item. In another embodiment, the photoelastic image capture device captures at least four images of at least one seal of each packaged item. In one embodiment, the photoelastic image capture device captures image data at a frame rate of at least 20 frames per second. In one embodiment, the photoelastic image capture device captures image data at a frame rate of at least 40 frames per second. In one embodiment, the photoelastic image capture device captures image data at a frame rate of at least 60 frames per second. Having additional image data of the seal provides additional data and can result in a more reliable determination of seal quality. In an embodiment, the UV image capture device is an RGB camera capable of capturing images in the visible light spectrum.

[0192] The image data collected by the photoelastic imaging device is sent to a visual inspection engine. As described in more detail herein, the visual inspection engine is capable of analyzing the image and determining whether the seal is acceptable or whether the seal has defects. In an embodiment, the visual inspection engine determines the confidence level and the seal classification. The visual inspection engine may be local to the photoelastic vision system 43, integrated into another part of the system, or remote. The visual inspection engine analyzes the image for changes in color gradients of adjacent pixels, segment comparisons, etc. The visual inspection engine may compare multiple pixels or segments of the image with respect to one or more of the following: mean, variance, skew, minimum, maximum, range, or changes in a sealed area. Changes between pixels or segments of the image may indicate a sealing defect.

[0193] Still reference Figure 4 , the defect mechanism 35 receives instructions to identify packaged items with defective seals or with a confidence level below a certain threshold. In one embodiment, the defect mechanism pulls the defective packaged items from the production line for offline testing, verification, recycling, repackaging or disposal, thereby allowing only packaged items with proper seals to continue beyond the defect mechanism 35. In an embodiment, the defect mechanism ties, marks or prints at least one mark on the packaged items to identify the packaged items as potentially having defective seals. In an embodiment, the detection of a defective seal may cause the defect mechanism to activate an alarm, mark the defective packaged item, display an image of the defective seal or packaged item, display data related to the defective seal, and generate a report on the defective seal or packaged item.

[0194] In an embodiment, a signal responsive to a defective seal is generated and activated while the web remains in motion (i.e., in real time and online), an alarm, a flag, an intermittent image display, intermittent data, a report of the intermittent data, etc. Alternatively, a signal responsive to a defective seal is generated after production is complete (i.e., offline). The signal responsive to the intermittent may include an electronic message, an email, a data log, and a report.

[0195] It should be understood that the vision system and defect mechanism can be implemented after the formation of the packaged article and before the product is packaged. In another embodiment, the vision system and defect mechanism can be implemented after the packaging and sealing of the product in the packaged article.

[0196] Reference now Figure 5 , Figure 5 3 is a block diagram of an exemplary process flow for detecting defects in a seal area of ​​a packaged item. The film or packaged item is sealed by a sealing mechanism 31. In some embodiments, the packaged item may include a film, a pouch, a bag, a tubing, a rigid or semi-rigid shell, wherein a film cover or seal encapsulates the shell. In some embodiments, the film may include a detectable component that allows for detection of seal defects. In some embodiments, the packaged item may include a skin bag. In some embodiments, the film with an indicator based on ultraviolet ("UV") fluorescence may only partially surround the product.

[0197] The film includes detectable components such as fluorescence-based indicators. In an embodiment, the film includes a layer of a blend including a polymer and a fluorescence-based indicator in any functional layer, wherein the concentration of the fluorescence-based indicator is present in the blend in an amount from about 5 to 50 ppm based on the total layer weight, wherein the fluorescence-based indicator is uniformly blended with the polymer. The uniformity of the indicator in the indicator / polymer blend refers to: based on the target concentration of the indicator in the blend, when 10 random samples (each sample having a size of 10 grams) are taken, the concentration of the indicator in the blend is subjected to a change of no more than 20% and no less than 20%. In an embodiment, the polymer includes PVDC (i.e., polyvinylidene chloride), ethylene-vinyl alcohol or nylon.

[0198] UV fluorescent indicator can be present in the functional layer at any level that can be detected by the detector, while allowing the functional layer to maintain its intended function. Too much fluorescent indicator can interfere with the layer function. Too little fluorescent indicator can become unable to be detected by the detector. In an embodiment, the fluorescent indicator can be present at a level of at least 0.5 parts per million (ppm). As used herein, the phrase "parts per million" and the equivalent expression "ppm" refer to the relationship between the weight of the fluorescent indicator and the gross weight of the layer (the weight of the fluorescent indicator + the weight of the remaining components in the layer). Of course, the main component of the layer is one or more thermoplastic polymers that are solid at room temperature. Both the fluorescent indicator and the thermoplastic polymer of the layer can be solid at room temperature. In an embodiment, fluorescent indicator can be present in the layer with a level of at least 1 ppm, or at least 1.5 ppm, or at least 2 ppm, or at least 3 ppm, or at least 5 ppm, or at least 10 ppm, or at least 20 ppm, or at least 40 ppm, or at least 80 ppm, or at least 120 ppm, or at least 160 ppm, or at least 200 ppm, or at least 300 ppm, or at least 500 ppm. In an embodiment, fluorescent indicator can be present in the layer with a level from 0.5 to 40 ppm, or from 1 to 20 ppm, or from 1.5 to 10 ppm, or from 2 to 5 ppm. In order to make film suitable for the final use of food contact, fluorescent indicator is present in the layer with an amount that is no more than 150 ppm.

[0199] UV-based fluorescent indicators are UV-absorbing compounds with unique absorption and / or fluorescence properties. Preferred UV-absorbing fluorescent indicator compositions have unique optical characteristics that do not exist in nature and are not easily confused with signals from natural sources. Preferred UV fluorescent indicators have multiple unique absorption or fluorescence characteristics in their UV spectrum. For example, as used herein, electromagnetic radiation at 375 nanometers is used as the incident radiation to excite a fluorescent indicator known as 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole), which is assigned CAS Registry Number 7128-64-5 and is also known as: 2,2'-(2,5-thiophenediyl)bis[5-tert-butylbenzoxazole]; 2,5-bis-2(5-tert-butyl-benzooxalyl)thiophene; 2,5-bis(5-tert-butyl-2-benzoxazolyl)thiophene; 2,5-bis-(5-tert-butylbenzoxazolyl-[2-yl])-thiophene; 2,5 -bis(5'-tert-butyl-2-benzoxazol-2-yl)thiophene; 2,5-bis(5'-tert-butyl-2'-benzoxazolyl)thiophene; 2,5-bis(5-tert-butyl-2-benzoxazolyl)thiophene; 2,5-bis(5-tert-butyl-benzoxazol-2-yl)thiophene; 2,5-bis(5-tert-butylbenzoxazol-2-yl)thiophene; 2,5-bis(5-tert-butylbenzoxazolyl)-2-thiophene; 2,5-bis(5-tert-butylbenzoxazol-2-yl)thiophene; 2,2'-(2,5-thiophenediyl)bis[5-(1,1-dimethylethyl)-benzoxazole; 2,5-bis(5'-tert-butyl-2-benzoxazolyl)thiophene; and 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole). Absorption of incident radiation at 375 nanometers causes the excited 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) optical brightener detectable component to emit radiation at 435 nanometers. The fluorescent indicator is uniformly blended into the PVDC resin used to produce the oxygen barrier layer of the multilayer film. The resulting endless strip and / or heat shrinkable film tubing is exposed to incident radiation at 375 nm to excite the 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) optical brightener to emit radiation at 435 nanometers. The emitted 435 nm radiation is detected by a machine vision system, which reveals the presence, continuity and thickness of the PVDC barrier layer of the strip and multilayer film tubing. In an embodiment, the UV-based fluorescent indicator includes at least one selected from the group consisting of an ultraviolet indicator, an infrared indicator, a dye, a pigment, an optical brightener, a fluorescent whitening agent, 2,2'-(2,5-thiophenediyl)bis(5-tert-butylbenzoxazole), hydroxy-4-(p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dye.The indicator is of a type that, if exposed to radiation at a first peak wavelength, emits radiation at a second peak wavelength.

[0200] In an embodiment, the detectable component includes at least one selected from the group consisting of: an ultraviolet indicator, an infrared indicator, a dye, a pigment, an optical brightener, a fluorescent whitening agent, and 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole), 2,2'-(2,5-thiophenediyl)bis(5-tert-butylbenzoxazole), hydroxy-4-(p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes. 2,5-Thienyldiylbis(5-tert-butyl-1,3-benzoxazole) is sold as an optical brightener by several suppliers, including BASF (TINOPAL OP® 2,5-Thienyldiylbis(5-tert-butyl-1,3-benzoxazole) fluorescent brightener) and Mayzo, Inc (BENETEX OB PLUS® 2,5-Thienyldiylbis(5-tert-butyl-1,3-benzoxazole) fluorescent brightener). The indicator is of a type that emits radiation at a second peak wavelength if exposed to radiation at a first peak wavelength.

[0201] In an embodiment, a UV-based fluorescent indicator is present in at least one layer of the film, wherein the indicator is present at a level of from 0.5 to 150 ppm based on the layer weight. In another embodiment, the indicator is present in at least one layer of the film, wherein the indicator is present at a level of from 1 to 20 ppm or from 2 to 10 ppm based on the layer weight. In an embodiment, the detectable component is present in the film layer at a level of at least 1 part per million.

[0202] In an embodiment, the UV-based fluorescent indicator is of a type that, if exposed to radiation at a first peak wavelength, emits radiation at a second peak wavelength.

[0203] Still reference Figure 5 , the sealing mechanism 31 produces a sealing area in the film. The sealing mechanism 31 seals the film to itself or another packaged article. In an embodiment, the sealing mechanism is a heat-generated sealing mechanism, such as heating, impulse sealing, ultrasonic sealing or laser sealing from a sealing bar. The sealing mechanism seals the film to itself. In the example of the film tubing as described above, the sealing mechanism seals one end of the tubing to produce a packaged article with an opening. It should be understood that the sealing mechanism can also seal the sides of the packaged article, and can also seal the final opening after the product has been placed in the packaged article.

[0204] After the sealing of the packaged article, the ultraviolet system 53 captures the ultraviolet image of the sealed area. The ultraviolet vision system 53 includes at least one ultraviolet image capture device, an ultraviolet light source, and a visual inspection engine. In an embodiment, the ultraviolet image capture device is an RGB camera capable of capturing images in the visible light spectrum. In some embodiments, the ultraviolet vision system 53 includes multiple light sources (same or different types), filters, multiple image capture devices, or reflectors. The light source can be a continuous light source or a flash light source. The additional light source can be white light, filtered light, diffuse light, infrared light, ultraviolet light, in the visible light spectrum or outside the visible light spectrum. In some embodiments, the ultraviolet vision system 53 is contained in a housing to provide an isolated environment for the image capture device to capture the image. The housing can isolate or reduce the impact of external light sources and contaminants such as dust on the imaging area. The light source of the ultraviolet vision system 53 excites the detectable components in the sealed area of ​​the packaged article. The detectable components allow the image capture device to identify the detectable components.

[0205] In an embodiment, the detectable component is a UV fluorescent indicator and at least one of: (i) a higher intensity of fluorescent electromagnetic energy in the area where the seal defect is located compared to a proper seal, or (ii) a color shift of the fluorescent electromagnetic energy (energy downward, i.e., longer wavelength, lower frequency) in the area where the seal defect is located compared to a proper seal.

[0206] In an embodiment, the intensity of the fluorescent electromagnetic energy is at least 10% higher in the area where the seal defect is located relative to the proper seal. In another embodiment, the intensity of the fluorescent electromagnetic energy is ≥15% higher, or ≥20% higher, or ≥30% higher, or ≥40% higher, or ≥50% higher, or ≥60% higher in the area where the seal defect is located relative to the intensity of the fluorescent electromagnetic energy in the area where the seal is proper. Factors that can affect the fluorescent image emitted by the package include, among others: (i) viewing angle, (ii) focal length between the imaging device and the package area under consideration, (iii) exposure time, (iv) excitation amount, and (v) thickness variations in the film, including increases in thickness at heat seals of the film to itself or to another film.

[0207] In one embodiment, the UV image capture device captures an image of at least one seal of each packaged item. In another embodiment, the UV image capture device captures at least two images of at least one seal of each packaged item. In another embodiment, the UV image capture device captures at least four images of at least one seal of each packaged item. In one embodiment, the UV image capture device captures image data at a frame rate of at least 20 frames per second. In one embodiment, the UV image capture device captures image data at a frame rate of at least 40 frames per second. In one embodiment, the UV image capture device captures image data at a frame rate of at least 60 frames per second. Having additional image data of the seal provides additional data and can result in a more reliable seal quality determination.

[0208] The image data collected by the ultraviolet imaging device is sent to a visual inspection engine. As described in more detail herein, the visual inspection engine is capable of analyzing the image and determining whether the seal is acceptable or whether the seal has defects. In an embodiment, the collection of image data is performed using an ultraviolet imaging device that generates image data of fluorescent electromagnetic energy emitted by an excited indicator, wherein the evaluation of the image data is performed using a visual inspection engine that is programmed with an algorithm that is capable of evaluating the intensity of the fluorescent electromagnetic energy emitted by the excited indicator. In an embodiment, the visual inspection engine determines the confidence level and the seal classification. The visual inspection engine may be located locally in the ultraviolet vision system 33, integrated into another part of the system, or remote.

[0209] Still reference Figure 5 , the defect mechanism 35 receives instructions to identify packaged items with defective seals or with a confidence level below a certain threshold. In one embodiment, the defect mechanism pulls the defective packaged items from the production line for offline testing, verification, recycling, repackaging or disposal, thereby allowing only packaged items with proper seals to continue beyond the defect mechanism 35. In an embodiment, the defect mechanism ties, marks or prints at least one mark on the packaged items to identify the packaged items as potentially having defective seals. In an embodiment, the detection of a defective seal may cause the defect mechanism to activate an alarm, mark the defective packaged item, display an image of the defective seal or packaged item, display data related to the defective seal, and generate a report on the defective seal or packaged item.

[0210] In an embodiment, a signal responsive to a defective seal is generated and activated while the web remains in motion (i.e., in real time and online), an alarm, a flag, an intermittent image display, intermittent data, a report of the intermittent data, etc. Alternatively, a signal responsive to a defective seal is generated after production is complete (i.e., offline). The signal responsive to the intermittent may include an electronic message, an email, a data log, and a report.

[0211] It should be understood that the vision system and defect mechanism can be implemented after the formation of the packaged article and before the product is packaged. In another embodiment, the vision system and defect mechanism can be implemented after the packaging and sealing of the product in the packaged article.

[0212] In some embodiments, multiple vision systems are contemplated. The vision systems may be redundant as vision systems of similar types or different vision systems utilizing different light sources, image capture devices, lighting angles, lighting properties, etc. Referring now to Figure 6 , Figure 6 is a block diagram of an exemplary process flow for detecting defects in the seal area of ​​a packaged item. The process is similar to the process described above with reference to Figures 3 to 5 The described embodiment adds the inclusion of additional vision systems. The thermal imaging vision system 33, the photoelastic vision system 43, and the ultraviolet vision system 53 each send image data to the vision inspection engine. It should be understood that a common shared vision inspection engine or multiple vision inspection engines are envisioned. The vision inspection engine is programmed with one or more algorithms that are capable of evaluating the intensity of the thermal imaging image, the photoelastic image, and the fluorescent electromagnetic energy emitted by the excited indicator. The combined image data from the three vision systems results in a higher confidence level, which is sent to the defect mechanism 35. Although the thermal imaging vision system 33, the photoelastic vision system 43, and the ultraviolet vision system 53 are shown, a system that only utilizes two vision systems is envisioned.

[0213] Reference now Figure 7 , Figure 7 is a block diagram of an exemplary process flow for detecting defects in a sealing area of ​​a packaged article after the product is sealed in the packaged article. A product loading mechanism loads the product into the packaged article. In one embodiment, the product is a food product. In another embodiment, the product is meat or cheese. After the product is loaded, a sealing mechanism 31 seals the product in the packaged article. In some embodiments, the sealing mechanism includes a vacuum chamber to vacuum seal the product in the packaged article. In an embodiment, the packaged article shrinks around the product. In an embodiment, the packaged article is a non-shrink packaged article. One or more seals of the packaged article are then analyzed by a thermal imaging vision system 33, a photoelastic vision system 43, and an ultraviolet vision system 53, and (multiple) visual inspection engines, as described more fully herein. The combined image data from the three vision systems results in a higher confidence level, which is sent to a defect mechanism 35. Although a thermal imaging vision system 33, a photoelastic vision system 43, and an ultraviolet vision system 53 are shown, a system utilizing only two vision systems is envisioned.

[0214] Now go to Figure 8, a thermal imaging vision system 81 is shown according to one embodiment. The film 100 is transported to a sealing mechanism 85 to seal the film 100 to itself or another article. In one embodiment, the sealing mechanism produces at least 10 seals per minute. In an embodiment, the sealing mechanism produces at least 50 seals per minute. In an embodiment, the sealing mechanism produces at least 100 seals per minute. In an embodiment, the sealing mechanism produces at least 250 seals per minute. In one embodiment, the sealing mechanism 85 is a heat-generated sealing mechanism, such as a sealing bar sealer, an impulse seal, an ultrasonic seal, or a laser seal.

[0215] After sealing, the film 100 continues beyond the sealing mechanism 85, where the thermal imaging image capture device 87 captures at least one image of the seal produced by the sealing mechanism 85. The thermal imaging image capture device is an infrared scanner or camera. In an embodiment, the infrared camera is capable of capturing at least two images of each seal while the film is in motion. In an embodiment, the infrared camera is capable of capturing at least four images of each seal while the film is in motion. In an embodiment, the infrared camera is capable of capturing images of each seal at a rate of at least 30 frames per second. In an embodiment, the infrared camera is capable of capturing images of each seal at a rate of at least 60 frames per second. In an embodiment, the thermal imaging image capture device 87 is positioned to capture an image of the seal between 2 and 40 seconds after the seal is produced in the film 100 by the sealing mechanism 85. In an embodiment, the thermal imaging image capture device 87 is positioned to capture an image of the seal 5 to 30 seconds after the seal is produced in the film 100 by the sealing mechanism 85. In an embodiment, the thermal imaging image capture device 87 is positioned to capture an image of the seal 10 to 20 seconds after the seal is produced in the film 100 by the sealing mechanism 85. In an embodiment, the thermal imaging image capture device 87 is positioned to capture an image of the seal about 2 seconds after the seal is created in the film 100 by the sealing mechanism 85. In an embodiment, the thermal imaging image capture device 87 is positioned to capture an image of the seal about 5 seconds after the seal is created in the film 100 by the sealing mechanism 85. In an embodiment, the thermal imaging image capture device 87 is positioned to capture an image of the seal about 10 seconds after the seal is created in the film 100 by the sealing mechanism 85. In an embodiment, the thermal imaging image capture device 87 is positioned to capture an image of the seal about 20 seconds after the seal is created in the film 100 by the sealing mechanism 85. The residual heat from the sealing mechanism 85 allows the thermal imaging image capture device 87 to capture a thermal image of the seal and changes in temperature along the sealing area. Such changes may indicate a defective seal. The image data captured by the thermal image capture device 87 is sent to the visual inspection engine 101, as described in more detail herein.

[0216] Reference now Fig. 9, showing a photoelastic vision system 91 according to one embodiment. The film 100 is transported to a location where a photoelastic image capture device 96 captures an image of light passing through the sealing area of ​​the film 100, the linear polarizers 94 and 95. In an embodiment, the photoelastic image capture device is an RGB camera capable of capturing images in the visible light spectrum. Light from the light source 91 passes through a diffuser plate 93 for diffusing the light, a first linear polarizer 94 having a polarization direction oriented in a first direction to polarize the light from the light source 91. It should be understood that a diffuse light source can be used instead of a diffuser plate, or in addition to a diffuser plate. The polarized light then passes through the film 100, thereby rotating the light. The amount of rotation varies based on variables such as the type of film, the seal strength, and the integrity of the seal. In some cases where the light is rotated, the light then passes through a second linear polarizer 95 having a polarization direction oriented orthogonal to the first direction of the first linear polarizer 94. In the absence of the film 100, no light will be transmitted from the light source 91 to the photoelastic image capture device 96. However, since the film 100 rotates the light, the photoelastic image capture device 96 captures an image of the film 100 .

[0217] In an embodiment, the light source 91 is a white light source having a wavelength spectrum from 400 nm to 700 nm. In an embodiment, the light source is a background light below the film. In another embodiment, the light source is a polarized white light source, thereby eliminating the need for a first linear polarizer.

[0218] Although two linear polarizers are described above, it is contemplated that at least one of the linear polarizers may be replaced by utilizing a polarizing filter film on the light source or lens of the photoelastic image capture device. In another embodiment, the photoelastic image capture device includes a polarizing lens to eliminate the need for a second linear polarizer. In another embodiment, the light source is a polarized light source. In an embodiment, the photoelastic camera is capable of capturing at least two images of each seal while the membrane is in motion. In an embodiment, the photoelastic camera is capable of capturing at least four images of each seal while the membrane is in motion. In an embodiment, the photoelastic camera is capable of capturing images of each seal at a rate of at least 30 frames per second. In an embodiment, the photoelastic camera is capable of capturing images of each seal at a rate of at least 60 frames per second.

[0219] Image data captured by the photoelastic image capture device 97 is sent to a visual inspection engine 101 , as described in greater detail herein.

[0220] Reference now Fig.10 , Fig.101004. It is a schematic diagram of photoelastic operation. The polarization direction of the first polarization filter 1002 is orthogonal to the polarization direction of the second polarization filter 1004. Unpolarized light 1001 passes through the first polarization filter 1002 (i.e., a linear polarizer), thereby converting the unpolarized light 1001 into polarized light 1003. In the absence of the film 1005, the polarized light 1003 is filtered by the second polarization filter 1004, and no or little light passes through the second polarization filter 1004. In contrast, when the polarized light 103 passes through the film 1005, the polarized light 1003 rotates, thereby causing offset polarized light 1006. This rotation allows some light to pass through the second polarization filter 1004. Variables that change the amount of rotation or filtering of the light include the thickness of the film, the stress of the film, and the strain of the film. The amount of change in rotation allows the capture of the image as described above.

[0221] Reference now Fig.11 , shows an ultraviolet vision system 1100 according to one embodiment. The film 100 is transported into a dark housing 1102. An ultraviolet light source 1101 illuminates the film 100, causing detectable components in the film to be excited and emit fluorescent electromagnetic energy. An ultraviolet image capture device 1106 captures at least one image of a sealed area of ​​the film 100. In an embodiment, a blue pass filter is used to filter out light with wavelengths below 450 nm and about 525 nm. The blue pass filter improves use in higher than normal ambient lighting conditions. In one embodiment, the light source or blue pass filter transmits light between 425-495 nm.

[0222] In an embodiment, when the film is in motion, the UV camera can capture at least two images of each seal. In an embodiment, when the film is in motion, the UV camera can capture at least four images of each seal. In an embodiment, the UV camera can capture the image of each seal at a speed of at least 30 frames per second. In an embodiment, the UV camera can capture the image of each seal at a speed of at least 60 frames per second. The nature of the seal allows the UV image capture device 1106 to capture the UV image of the seal and the change in the energy emitted along the seal area. Such a change can indicate a defective seal. The image data captured by the UV image capture device 101 is sent to the visual inspection engine 101, as described in more detail herein.

[0223] Example system architecture

[0224] like Fig.12 , in some embodiments, one or more computing systems, such as computing devices, may be used to control imaging device(s) 1204, light source(s) 1205, and vision system 1201. In some embodiments, one or more computing systems may control or direct other computing systems to control other functions within the packaging environment, such as inline conveyors and material handling equipment.

[0225] Visual inspection engine 101 (such as Figure 8 , Fig. 9 and Fig.11 ) may be a computing device 1203, which may include a processor 1206, a memory 1207, an input / output circuit 1208, a communication circuit 1211, a visual inspection circuit 1210, and an acquisition circuit 1209, and may be configured to perform the functions described herein. In some embodiments, the processor 1206 (and / or a coprocessor or an auxiliary processor or any other processing circuit otherwise associated with the processor) may communicate with the memory 1207 via a bus for transferring information between components of the device. In some embodiments, the computing device 1203 may be a distributed system of remotely located computing devices and / or computing components. In some embodiments, the computing device 1203 may be local or remote. The memory 1207 may be non-temporary and may include, for example, one or more volatile and / or non-volatile memories. In other words, for example, the memory may be an electronic storage device (e.g., a computer-readable storage medium). The memory may be configured to store information, data, content, applications, instructions, etc., for enabling the device to perform a variety of functions according to any example embodiment of the present invention.

[0226] Processor 1206 can be embodied in many different ways, and can include, for example, one or more processing devices configured to execute independently. In addition or alternatively, the processor can include one or more processors configured in series via a bus to achieve independent execution of instructions, pipelining and / or multithreading.

[0227] In an example embodiment, the processor 1206 may be configured to execute instructions stored in the memory 1207 or otherwise accessible to the processor. Additionally or alternatively, the processor may be configured to perform hard-coded functions. As such, whether configured by hardware or software methods, or by a combination thereof, the processor may represent an entity (e.g., physically embodied in a circuit) capable of performing operations according to embodiments of the present invention while being configured accordingly. Alternatively, as another example, when the processor is embodied as an executor of software instructions, the instructions may specifically configure the processor to perform the algorithms and / or operations described herein when the instructions are executed.

[0228] In some embodiments, visual system 1201 may include input / output circuit 1208, which may then communicate with processor 1206 to provide output to the user, and in some embodiments receive an indication of user input. Input / output circuit 1208 may include a user interface, and may include a display, and may include a network user interface, a mobile application, a client device, an information kiosk, etc. In some embodiments, input / output circuit 1208 may also include a keyboard, a mouse, a joystick, a touch screen, a touch area, a soft key, a microphone, a speaker or other input / output mechanism. Processor and / or user interface circuit including processor may be configured to control one or more functions of one or more user interface elements by computer program instructions (e.g., software and / or firmware) stored in a memory (e.g., memory 1207 and / or the like) accessible to the processor.

[0229] Meanwhile, the communication circuit 1211 may be any device, such as a device or circuit embodied in hardware or a combination of hardware and software, configured to receive data from a network and / or any other device or module communicating with the computing device 1203 and / or transmit data to the network and / or any other device or module. In this regard, the communication circuit may include, for example, one or more cables (e.g., USB cables) connecting the (multiple) imaging devices 1204 and (multiple) light sources 1205 to the visual system 1201 for use with the software and hardware configurations described herein. In some embodiments, the communication circuit 1211 may include, for example, an antenna (or multiple antennas) and supporting hardware and / or software for enabling communication with a wireless communication network or one or more wireless devices. Additionally or alternatively, the communication interface may include circuitry for interacting with (multiple) cables and / or (multiple) antennas to cause signal transmission via (multiple) cables and / or (multiple) antennas or processing reception of signals received via (multiple) cables and / or (multiple) antennas. In some environments, the communication interface may additionally or alternatively support wired communication with a network (e.g., Ethernet). As such, for example, the communication interface may include a communication modem and / or other hardware / software for supporting communication via cable, digital subscriber line (DSL), universal serial bus (USB), or other mechanisms.

[0230] The acquisition circuit 1212 may be used to buffer a series of images and data captured at the imaging device(s) 1204 (e.g., camera(s)). In some embodiments, the imaging device(s) 1204 may capture raw data incident on one or more surfaces within the imaging device (e.g., on one or more substrates of an image sensor). The imaging device(s) 1204 may convert the raw data into computer-readable image data via one or more circuits, and may transmit the image data to the acquisition circuit 1212. In some embodiments, the image data may include any sensor data corresponding to the wavelength and / or intensity of electromagnetic energy used to detect defects in packaged products. The image data may include: individual images; image sequences; videos; and the like. The acquisition circuit 1212 may further control the imaging device(s) 1204 and the light source(s) 1205 to trigger and time the capture of corresponding illumination and raw data of packaged products according to any embodiment of the present invention. In some embodiments, the image data may be captured in a manual, autonomous, and partially manual and partially autonomous operation form by any device disclosed herein for generating image data, including but not limited to any imaging device (e.g., camera, sensor, etc.) disclosed herein.

[0231] The visual inspection circuit 1210 may be used to facilitate processing and analysis of image data received from the acquisition circuit 1212. The visual inspection circuit 1210 may further trigger an alarm or other downstream notification or action based on the results of the processing and analysis. The visual inspection circuit may further be connected to one or more remote servers for data mining and storage (e.g., to access models and data from which new models are trained). In some embodiments, the visual system 1201 and the visual inspection circuit 1210 may include a single computing device, or may include multiple devices that are locally connected or interact through a wired and / or wireless network.

[0232] In some embodiments, vision system 1201 may include an operator user interface (e.g., as part of input / output circuitry 1208). Defect data (e.g., from visual inspection circuitry 1210) may be displayed on the interface and archived locally or remotely (e.g., via a local connection or a network connection) in a database. Defect data and images may be displayed on the interface in real time. In some embodiments, instant, historical, and statistical data may also be viewed on demand on the interface. Computing device 1203 may be configured to selectively detect and accurately classify defects in packaged products, including detecting excited fluorescence-based indicators that indicate trapped gases within the packaged items.

[0233] The image of each defect can be classified, stored, displayed, and compared with previous and future images of other defects. The computing device 1203 can capture high-resolution images of each defect in real time. Discrete defect information such as geometric information of a single defect and statistical data of group defects can be provided for immediate decision-making and actions on process improvement and monitoring, such as defect alarms. In some embodiments, each potential defect can be displayed to an operator for manual decision-making on how to deal with defective packaged products. In some embodiments, the screening, marking and / or separation of defective packaged products can be partially or completely completed automatically. In some instances, a human operator can verify the work of an otherwise automatic system.

[0234] Multiple outputs for identification / marking, alarms, and autonomy can be set for different defect severity levels. For example, data can be exported to Microsoft Excel and / or SQL databases located anywhere on the network, where data mining software allows multiple reports to be easily generated automatically and / or on demand. Defect data can be processed on a processing unit such as a digital processing board. Marking can be used in conjunction with separating and / or discarding packaged products with damaged films or films with damage exceeding a predetermined threshold. Labels can be applied to the film at the location of the defect in the film (or at a location corresponding to the defect) for manual or automatic separation (e.g., using a robotic packaging separator). In some embodiments, defective packaging (e.g., a packaged product showing a leak or low vacuum condition) can be unpacked and repacked with a new packaged item.

[0235] In an embodiment, the input / output circuit 1208 may allow external signal inputs, such as new roll or new production process indications and pause inspection indications. Alarm outputs based on user-defined defect alarm criteria are also processed by the input / output circuit 1208 (e.g., user interface). Outputs may also be activated to control downstream marking or identification equipment. Alarms may be activated for defects of different predefined severity or criteria. Alarm and defect information from the computing device 1203 may be sent to a factory network, a programmable logic controller (PLC), or a supervisory control and data acquisition / human-machine interface (SCADA / HMI) via OPC (i.e., a software interface standard).

[0236] In an embodiment, an encoder (not shown) may be used to measure the conveyor speed so that the location of a detected defective packaged product can be determined. A series of pulses from the encoder are received and counted by the system. The count is sent to the processor 1206 to determine the distance along the conveyor at which the detected defective packaged product is located and may be used to time the operation of the defect separator to remove the defective packaged product from the production line before being sent away (when the packaged product moves off the production line).

[0237] Although a local system is depicted, it should be understood that a distributed system can be utilized by connecting multiple computing devices via a network. In some embodiments, the network may include any wired or wireless communication network, including, for example, a wired or wireless local area network (LAN), a personal area network (PAN), a metropolitan area network (MAN), a wide area network (WAN), etc., and any hardware, software, and / or firmware (such as, for example, a network router, etc.) required to implement the network. For example, the network may include a cellular phone, 802.11, 802.16, 802.20, and / or WiMax network. In addition, the network may include a public network such as the Internet, a private network such as an intranet, or a combination thereof, and may utilize a variety of networking protocols currently available or developed later, including but not limited to networking protocols based on TCP / IP.

[0238] In some embodiments, multiple computing devices may collectively include the above Figure 4 Features of computing device 1203 are described. In some embodiments, any number of computing devices may make up a computing device.

[0239] Example detection models and algorithms

[0240] In some embodiments, a variety of algorithms, models, and processes may be implemented to detect defects in product packaging and seals of packaged products. Each of the algorithms, models, and processes may be configured to locate and identify seal defects. In some embodiments, artificial intelligence algorithms (e.g., Fig.13 ), while in other embodiments, an algorithmic approach may be used based on received sensor information (e.g., Fig.14 ). The processes disclosed herein (e.g., algorithmic solutions, model-based solutions, etc.) can receive image data in one or more processed and pre-processed states; can further process, sort, classify, and analyze the image data; and can identify seal defects present in packaged items based on the image data.

[0241] Fig.13 Flowchart (depicted as a functional block diagram) illustrating a model-based process 500 for evaluating whether a seal of a packaged item is defective. Process 500 may be embodied, for example, in the context of Figure 8 , Fig. 9 and Fig.11 As depicted, the system may include one or more imaging devices 210, a data acquisition system 502 (e.g., using Fig.12 ), the visual inspection engine 504 (e.g., using Fig.12 The computing device 1203 and Fig.12), the visual inspection circuit 1210 shown in FIG. 5 ), the online action system 506 , the database management system 508 , the data collection system 510 (e.g., using Fig.12 ) and the model generator 520 (e.g., using Fig.12 The computing device 1203 and Fig.12 The process shown in Figure 1 generates circuit 1209). Fig.13 An example flow diagram is shown for performing a process 500 described in accordance with some embodiments.

[0242] In the depicted embodiment, a series of images are acquired using an appropriate combination of imaging equipment (e.g., including lens and / or sensor selection) and illumination (e.g., via (multiple) light sources 215), and fed into an acquisition system 502, where the data is acquired, buffered, and transmitted to one of the visual inspection engines 504 or data collection systems 510. Fig.13 The depicted embodiment includes two example use cases: a model generation workflow and a model application workflow.

[0243] In a model generation workflow, process 500 can generate a model from multiple images. Images can be captured by (multiple) imaging devices 210 and received by acquisition system 502. The acquisition system can include an image acquisition engine 540, an acquisition routine 542, a storage buffer 544, and a workspace 546. The acquisition system 502 can buffer the images and transmit the images to a data collection system 510, which can label 512 the images (e.g., good / adequate seal or poor / weak seal) and store the images 514. In some embodiments, the model generation workflow can retrieve images directly from storage device 514, or images (e.g., a training set) can be loaded into the system separately (e.g., via Fig.12 1211).

[0244] The images may be tagged 512 using any of the processes described herein. For example, a user may input tags associated with each image into the system. In some embodiments, the packaged item may include a hangtag or label that identifies features of the packaged item to a user or visual system for association with a corresponding image of the packaged item.

[0245] The labeled images from the data collection system 510 may be input to a model generator 520 to generate one or more models, which may receive images in a first step 521 by receiving labeled images in an image database 522 and initializing the images 523 for use in the model generator. The initialized images may then be passed to an image analyzer 524 where the images may be preprocessed 525 and analyzed 526. Model generation 527 may then be performed to create a plurality of models 528 based on the analyzed images using the training process described herein, and in some embodiments the models may be tested and a preferred model selected 529 based on a model evaluation matrix (which includes parameters such as accuracy, false positives, true positives, recall, precision, etc.) of their predictions in the test data.

[0246] Once a model is created and selected in the model generator 520, the model can be deployed to the visual inspection engine 504 in the model application workflow. In the model application workflow, images can be captured by (multiple) imaging devices 210 and fed into the acquisition system 502, where the data is acquired, buffered, and transmitted to the visual inspection engine 504. In the visual inspection engine 504, the model can be applied to unknown images to classify the image (e.g., good / adequate seal or poor / weak seal) based on the artificial intelligence algorithm developed by the model generator 520 in the model generation workflow. The process can include initializing the model 532, and the image can be input into the decision function 534 to receive a decision at the detection model output 536.

[0247] The inspection results 538 can be fed into the online action system 506 to set a predetermined alarm, film mark, display an image of the defective seal (e.g., via a user interface), display data related to one or more defects (including displaying data related to the geometric characteristics of the defective seal area, the location of the defect, the extent of the defect, the severity of the defect), generate a report of the defect data, and / or any other desired output. The data related to the defect can be displayed immediately and online, or after production is completed (i.e., offline or non-online), and the data can be accessed in the offline database management system 508. Using data mining, the data can be manipulated, visualized, and organized into any desired on-demand reports.

[0248] The detection results may be further transmitted as labeled images from the visual inspection engine 504 to the data collection system 510 to be stored in the image storage device 514 and used in a subsequent model generation process to recursively reduce the loss of the objective function and improve the performance of the model based on the model evaluation matrix.

[0249] The data processing software and hardware can be configured to accommodate different seal types while minimizing the need for online adjustments of parameters such as exposure time and light intensity. In some embodiments, portions of process 500 can be completed off-line or remotely. For example, data collection system 510 and / or model generator 520 can be remotely located (e.g., executed on a server) relative to the remainder of the process. Additionally, some or all of the visual inspection process can be performed remotely.

[0250] Go to Fig.14 , shows an example flow chart (depicted as a functional block diagram) of an algorithmic process 550 for evaluating whether a packaged item has a defective seal. Process 550 may be embodied, for example, in the context of Figure 8 , Fig. 9 and Fig.11 As depicted, the system may include one or more imaging devices 210, a data acquisition system 502 (e.g., using Fig.12 ), the visual inspection engine 504 (e.g., using Fig.12 The computing device 1203 and Fig.12 ), the visual inspection circuit 1210 shown in FIG. 5 ), the online action system 506 , the database management system 508 , the data collection system 510 (e.g., using Fig.12 ) and the inference engine 560 (e.g., using Fig.12 The computing device 1203 and Fig.12 The process shown in Figure 1 generates circuit 1209). Fig.14 An example flow diagram is shown for performing a process 550 described in accordance with some embodiments.

[0251] In the depicted embodiment, a series of images are captured using an appropriate combination of imaging device(s) 210 (e.g., including lens and / or sensor selection) and light source(s) 215 (e.g., via illuminator, UV illumination, polarized light) and fed into the acquisition system 502 where the data is captured, buffered, and transmitted to one of the visual inspection engines 504 or data collection systems 510. Fig.14 The depicted embodiment includes two example use cases: an algorithm generation workflow and an algorithm application workflow.

[0252] In an algorithm generation workflow, a process 550 may generate an algorithm from a plurality of images. The images may be captured by the imaging device(s) 210 and received by the acquisition system 502. The acquisition system may include an image acquisition engine 540, an acquisition routine 542, a storage buffer 544, and a workspace 546. The acquisition system 502 may buffer the images and transmit the images to the data collection system 510, which may label 512 the images (e.g., good / adequate seal or poor / weak seal) and store the images 514. In some embodiments, the algorithm generation workflow may retrieve images directly from the storage device 514, or images (e.g., a test set of images) may be loaded into the system individually (e.g., via Fig.12 1211).

[0253] The images may be tagged 512 using any of the processes described herein. For example, a user may input tags associated with each image into the system. In some embodiments, the packaged product may include a hangtag or label that identifies features of the packaged product to a user or visual system for association with a corresponding image of the product. In some embodiments, the training set may be preloaded with tags.

[0254] The labeled images from the data collection system 510 may be input to the reasoning engine 560 to generate and identify one or more algorithms, which may receive the images in a first step 561 by receiving the labeled images in the image database 562 and initializing the images 563 for use in the reasoning engine. The initialized images may then be passed to the image analyzer 564, where the images may be preprocessed 565 and may be analyzed 566. Algorithm determination 567 may then be performed to create one or more algorithms 568 based on the analyzed images using the processes described herein, and in some embodiments the algorithms may be tested and the preferred algorithms selected 569 based on the model evaluation matrices predicted by them in the test data. In some embodiments, one hundred test images are used. In some embodiments, one thousand test images are used. In some embodiments, a user may manually identify desired features in the images and / or image processing parameters to identify features during or prior to algorithm generation. In some embodiments, the system may partially or completely autonomously process images and / or detect desired features in images using the techniques described herein.

[0255] Once an algorithm is created and selected in the inference engine 560, it can be deployed to the visual inspection engine 504 in the algorithm application workflow. In the algorithm application workflow, images can be captured by (multiple) imaging devices 210 and fed into the acquisition system 502, where the data is acquired, buffered, and transmitted to the visual inspection engine 504. In the visual inspection engine 504, the algorithm can be applied to unknown (e.g., unlabeled) images to classify the images (e.g., good / sufficient seal or poor / weak seal) based on the algorithm selected in the inference engine 560 in the algorithm generation workflow. The process can include initializing the algorithm 572, and the image can be input into the decision function 574 to receive a decision at the detection algorithm output 576.

[0256] The inspection results 578 can be fed into the online action system 506 to set a predetermined alarm, film marking, display an image of the seal area (e.g., via a user interface), display data related to one or more defects (including displaying data related to the geometry of the seal, the location of the defect, the extent of the defect, the severity of the defect), generate a report of the defect data, and / or any other desired output. The defect-related data can be displayed immediately and online, or after production is completed (i.e., offline or non-online), and the data can be accessed in the offline database management system 508. Using data mining, the data can be manipulated, visualized, and organized into any desired on-demand reports.

[0257] The detection results can be further transmitted from the visual inspection engine 504 to the data collection system 510 as a labeled image to be stored in the image storage device 514 and used in the subsequent algorithm generation process to improve the evaluation matrix of the algorithm. In some embodiments, the image analysis and algorithm generation in the reasoning engine 560 can be performed autonomously. In some embodiments, the image analysis and algorithm generation in the reasoning engine 560 can be partially performed manually. In some embodiments, the image analysis and algorithm generation in the reasoning engine 560 can be performed manually.

[0258] The data processing software and hardware can be configured to accommodate different concentration levels while minimizing the need for online adjustment of parameters such as exposure time and light intensity. In some embodiments, portions of the process 550 can be completed offline or remotely. For example, the data collection system 510 and / or the inference engine 560 can be remotely located (e.g., executed on a server) relative to the remainder of the process. In addition, some or all of the visual inspection process can be performed remotely.

[0259] In some example embodiments, an algorithmic process may be used to detect the seal quality of a packaged article. For example, in some embodiments, an algorithm based on feature extraction may be used to detect a sealed portion in a packaged article that exhibits an excited fluorescence that is higher than a predetermined threshold intensity or at a predetermined wavelength or wavelength variation from incident light. In some embodiments, an algorithm based on feature extraction may be used to detect a sealed portion in a packaged article that exhibits a thermal gradient different from a baseline thermal gradient. In some embodiments, dark feature detection may be used to identify dark spots and bright spots in captured image data, which may separate areas of low fluorescence from areas of high fluorescence, wherein high fluorescence areas indicate the possibility of defective seals. In some embodiments, bright feature detection may be used in a manner similar to dark feature detection. In some embodiments, dark feature detection and bright feature detection may be calculated based on grayscale value thresholds of image data. In some embodiments, feature extraction may include extracting one or more textures of an image. Image textures may be compared with reference images to identify defective seals. In some embodiments, feature extraction may be further used to exclude undesirable areas, such as backgrounds or areas surrounding seals. For example, in some embodiments, the seal of a packaged article may have inconsistent textures or thicknesses, which may produce false positives on captured images. In some embodiments, bright spots caused by seals may be excluded from the algorithm.

[0260] In some embodiments, the algorithmic process may include streak detection. Streak detection may be based on geometric features extracted from the captured image.

[0261] refer to Fig.15 In an example embodiment, the algorithm 600 (e.g., an algorithm generated by the inference engine 560) and associated image processing may include the following steps: (1) capturing an image 602; (2) applying denoising to the image 604; (3) converting the image to grayscale 606; (4) applying thresholding to reduce the area of ​​the image to be considered 608; (5) in some embodiments, eliminating undesirable areas of the image 610; and (6) identifying features in the remaining image that correspond to defective seals 612. In some embodiments, the algorithm 600 may further include classifying the image based on the detected features 614. In some embodiments, the aforementioned algorithm steps may be eliminated, as long as defective seals can be identified.

[0262] As noted above, in some embodiments, an artificial intelligence algorithm (e.g., Fig.13 ) can be developed to classify sealed image data (e.g., using Fig.13 ). Fig.16An embodiment of a method 700 for developing a trained image classification model is depicted in FIG. At box 702, training image data of a seal area is obtained. In some embodiments, the training image data includes images and / or videos (i.e., sequences of images) of seals with known states. In some embodiments, the visual system used to obtain the training image data is the same as the visual system (e.g., a visual system in a final production line) that will be used to obtain image data of seal areas of packaged items of unknown states after the trained image classification model is created. In some embodiments, a test bench or other experimental configuration may be used to obtain the training image data. At box 704, the training image data is manually labeled with the state of the seal area of ​​the packaged item in the training image data. For example, a user may manually input a state (e.g., the packaged item has a defective seal, has a proper seal, or has a specific defect) for each image and / or video of the seal in the image data. Manually labeling the image data may include physically testing the seal to determine whether a separate seal is proper or defective, and then applying the label to the image data based on the results of the physical test. The training sample may include multiple seals and multiple defective seals. In some embodiments, the number of seals represented in the training image data ranges from tens of seals, hundreds of seals, thousands of seals, or more. At these numbers, the manual labeling process of the training image data can be a labor-intensive and time-intensive process. At block 706, the labeled training image data is input into a training module.

[0263] In some embodiments, the training model is a machine learning module, such as a "deep learning" module. Deep learning is a subset of machine learning that generates a model based on a training data set provided to it. In some embodiments, the training model can use supervised learning techniques, semi-supervised learning techniques, unsupervised learning techniques including clustering, anomaly detection, Hebbian learning, and learning latent variable models, such as expectation maximization algorithms, moment methods (mean, covariance) and blind signal separation techniques, which include principal component analysis, independent component analysis, non-negative matrix decomposition and singular value decomposition. In one example, unsupervised learning techniques are combined with several image pre-processing / post-processing techniques including imaging texture analysis (e.g., calculating leak detection index (Leaker Detection Index) value) to achieve detection and classification. In another embodiment, unsupervised learning technology anomaly detection is combined with several image pre-processing / post-processing techniques including imaging texture analysis to achieve detection and classification.

[0264] In an embodiment, the unsupervised learning technique includes the following steps. 1) Data collection, where images are collected for all types of seals without any category limitations. Best practices are implemented to ensure that only baseline seals are produced. 2) Algorithm development, where autoencoder components (such as encoders, decoders, and latent space vectors) are designed based on input data specifications and expected outputs. 3) Algorithm training, where the autoencoder neural network is trained on a baseline seal dataset with a target to reproduce samples with the finest details. 4) Defect isolation, where the autoencoder is deployed to production after training. Input seals that produce errors that deviate from the baseline examples are isolated. 5) Algorithm retraining, where the isolated defects are verified by humans. A set of false positive samples is prepared and used to retrain the autoencoder to further improve accuracy. The process can be repeated to improve performance.

[0265] In an embodiment, the semi-supervised learning technique includes the following steps. 1) Isolating defective seals from a baseline using an autoencoder. 2) Clustering defective seals by grouping them based on similarity scores using a clustering algorithm. 3) Manual classification, where humans evaluate defective seal clusters and assign categories. 4) Automatic classification, which requires developing a classification algorithm based on manually classified defects.

[0266] At box 708, an artificial intelligence algorithm is developed to classify the seal. In some embodiments, when the artificial intelligence algorithm is developed, one or more learning algorithms are used to create the artificial intelligence algorithm based on the marking state of the seal in the training image data. In some embodiments, the artificial intelligence algorithm is created based on one or more input vectors indicating the characteristics of the seal. In one example, the input vector may be a wrinkle in the seal. In another example, as described above, depending on the specific indicator and illumination used, the input vector may be the property of the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator in the film. For example, the input vector may include one or more of the following: the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator; the intensity of the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator; and / or the change in the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator compared to the incident electromagnetic energy from the illuminator. In some embodiments, the input vector may correspond to the wavelength of fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator (which is one or more colors in the visible light spectrum), the use of invisible electromagnetic energy (e.g., ultraviolet, infrared) to detect additives in the film of the packaged item, thermal imaging scanning, visible light scanning, photoelastic scanning, the presence and number of film folds, or any other number of possible input vectors. In embodiments, the (multiple) light sources 215 may be ultraviolet backlights with software for controlling shutter speed and light intensity. In other embodiments, the (multiple) light sources 215 are white light. In some embodiments, the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator may be at least in the ultraviolet range. In some embodiments, the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator may be at least in the visible light spectrum. In some embodiments, the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator may be at least in the blue or violet range. In some embodiments, the wavelength of the incident light from the illuminator may be at least in the ultraviolet range. In some embodiments, the wavelength of the incident light from the illuminator may define a peak in the ultraviolet range. In some embodiments, the wavelength of the incident light may be in the ultraviolet range, and the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator may be in the visible light range. In embodiments where the process is designed to simultaneously inspect multiple film layers at the same time for the same film, multiple light sources 215 and / or multiple imaging devices 210 may be used with one or more controllers for shutter speed, light intensity, light source, backlight, or highlight light.

[0267] Training with input vectors can help the artificial intelligence algorithm identify a defective seal without identifying the underlying cause. For example, a seal may have a small wrinkle that would be difficult to detect using image data captured while the packaged item is moving on a transport system. The use of input vectors, fluorescence-based indicators, thermal image data, and photoelastic image data detailed herein allow the artificial intelligence algorithm to detect that the seal is defective without identifying the defect itself. After the input vectors are modeled, the artificial intelligence algorithm can be developed as a decision process based on multiple input vectors. Examples of decision processes include decision trees, neural networks, etc. In some embodiments, the decision process of the artificial intelligence algorithm is based on determining acceptable arrangements of the input vectors in the decision process.

[0268] The result of developing the artificial intelligence algorithm in block 708 is the artificial intelligence algorithm depicted at block 710. The artificial intelligence algorithm may be used during normal operation (e.g., operation not used to train the artificial intelligence algorithm) to identify the state of the seal. In some embodiments, the artificial intelligence algorithm includes a neural network having multiple layers. Fig. 20 1 is an example of a neural network 1800 that is a multi-layer neural network. In the depicted embodiment, the neural network 1800 includes a first layer 1802 having three input nodes, a second layer 1804 having five hidden nodes, a third layer 1806 having four hidden nodes, a fourth layer 1808 having four hidden nodes, and a fifth layer 1810 having one output node.

[0269] The neural network 1800 also includes a first set of connections 1812 between each pair of the three input nodes in the first layer 1802 and the five input nodes in the second layer 1804, a second set of connections 1814 between each pair of the five input nodes in the second layer 1804 and the four hidden nodes in the third layer 1806, a third set of connections 1816 between each pair of the four hidden nodes in the third layer 1806 and the four hidden nodes in the fourth layer 1808, and a fourth set of connections 1818 between each pair of the four hidden nodes in the fourth layer 808 and the output nodes in the fifth layer 1810. In some embodiments, the input nodes represent inputs to the artificial intelligence algorithm (e.g., image data, metadata associated with the image data, etc.), one or more of the hidden nodes (e.g., one of the layers of hidden nodes) may represent one of the input vectors determined during development of the model, and the output nodes represent the determined state of the seal.

[0270] Reference now Fig.17 , Fig.17900 is an embodiment of a method for classifying the state of a seal (e.g., wrinkles, weak seals, cold seals, channel wrinkles) using a trained image classification model. At block 902, image data of the seal is collected (e.g., by (multiple) imaging devices 210). The image data of the seal may be obtained by a vision system. In some embodiments, the image data of the seal is obtained while the packaged item is being transported by a transport system. It should be understood that the image data may be captured after the seal is formed in the formation of the packaged item or after the seal for sealing the product in the packaged item is formed.

[0271] At box 904, the image data of the seal is input into the trained image classification model. The trained image classification model can be operated on a computing device or a remote computing device away from the local computing device. The trained image classification model is configured to classify the state of the seal based on the image data. At box 906, the classification of the state of the seal is received from the trained image classification model. In some embodiments, the classified state includes an indication that the seal is defective, non-defective, or has a specific defect and / or an indication of the degree of certainty of whether the seal is defective, non-defective, or has a specific defect. In some embodiments, the classified state is received by displaying the classification on a user interface output device, transmitting the classification to one or more external devices via a communication interface, or storing the classification in a database. In some embodiments, the received classification of the seal includes one or more of the state of the classification of the seal or the degree of certainty of the state of the classification of the seal. In a specific example, the state is transmitted to a routing system (e.g., a sending device), which is configured to route the packaged items on the transport system based on the sealing state of the packaged items, such as routing defective packages to locations for testing, repackaging, recycling and / or waste disposal.

[0272] As noted above, method 700 is used to obtain a trained classification model at block 710, and the trained classification model may then be used in method 900 to classify seals. In some embodiments, the training image data acquired at block 702 is image data of a seal for a particular type of product packaged in a packaging article, and the image data acquired at block 902 is image data of the same type of seal for the particular type of product packaged in a packaging article. In one example, the training image data acquired at block 702 is image data of a seal, and the image data acquired at block 902 is image data of the same type of seal. In some embodiments, the training image data acquired at block 702 is image data of a particular type of seal, and the image data acquired at block 902 is image data of a different type of seal. Even though the seal imaged online may be of a different type than the seal used in the training set, the trained classification model using the training image data from the training seal image data may be able to classify the state of the seal with sufficient accuracy.

[0273] Fig.18 An embodiment of a method 1010 for developing a trained image classification model is depicted in FIG. At box 1012, training image data is collected for a plurality of seals. At box 1014, the training image data is manually labeled as defective or non-defective. Manual labeling of the training image data may be accomplished by a user inputting an indication of defective or non-defective for each of the seals represented in the training image data into a user interface input device of a computing device. In some embodiments, training packaging items having seals may be labeled with their respective states (e.g., defective or non-defective). The training packaging items are packaging items having known defective seals or known non-defective seals. The training packaging items may further identify specific defects. The training packaging items are used to train the model to improve the reliability of the model.

[0274] At box 1016, model information, training objectives, and constraints are initialized. In some examples, the model information includes the type of model to be used, such as a neural network, a plurality of input vectors, etc. In some examples, the training objectives may include the expected or anticipated performance of the artificial intelligence algorithm, such as an evaluation matrix having a confidence level greater than or equal to a predetermined ratio (e.g., greater than or equal to one or more of 90%, 95%, 96%, 97%, 98%, or 99%). In some examples, the constraints may include limitations of the artificial intelligence algorithm, such as a minimum number of layers of a neural network, a maximum number of layers of a neural network, a minimum weight of an input vector, a maximum weight of an input vector, or any other constraint of the artificial intelligence algorithm. At box 1018, the model may be trained using the model information and model constraints. In some embodiments, the training image data is divided into two subsets (a training subset and a validation subset), and the training of the model at box 1018 includes training the model using the training subset of the image data.

[0275] At box 1020, a determination is made as to whether the training objective is met. In some embodiments, the determination at box 1020 is made by comparing the results of the artificial intelligence algorithm to the training objective initialized at box 1016. In some embodiments in which the training image data is divided into a training subset and a validation subset, the determination at box 1020 includes testing the model trained at box 1018 using the validation subset of the image data. If it is determined at box 1020 that the training objective is not met, then the method 1010 proceeds to box 1022, where the training objective and / or constraints are updated. After the training objective and / or constraints are updated at box 1022, the method 1010 returns to box 1018, where the model is trained using the updated training objective and / or constraints. If it is determined at box 1020 that the training objective is met, then the method 1010 proceeds to box 1024, where the artificial intelligence algorithm is stored. Storing the artificial intelligence algorithm may include storing the artificial intelligence algorithm in one or more memories in a computing device (e.g., a local computing device, a remote computing device, etc.).

[0276] In some embodiments, the vision system can be used both to train a model to classify the state of a seal and to apply an artificial intelligence algorithm to classify the state of a seal. Fig.19 An embodiment of a method 1100 for enabling a visual inspection engine to both train a model to classify the state of a seal for a packaged item and to apply an artificial intelligence algorithm to classify the state of the seal is depicted in . In some embodiments, the vision system includes an image sensor system and a computing device that may define a visual inspection engine 504 and an acquisition system 502. In those embodiments, the model may be operated on the computing device while (multiple) imaging devices obtain image data of the seal for training or applying the model.

[0277] At box 1102, the vision system and the classification model are initialized. In some embodiments, the initialization of the vision system includes initializing the computing device and initializing (multiple) imaging devices, and the initialization of the classification model includes loading startup software including the classification model on the computing device. At box 1104, image data of the seal is collected (e.g., by (multiple) imaging devices and the acquisition system). In some embodiments, the (multiple) imaging devices collect image data of the seal and provide the image data to the computing device. At box 1106, it is determined whether the classification model is in training mode. This determination can be made by software operating on the computing system including the classification model.

[0278] If it is determined at box 1106 that the classification model is in training mode, the model is passed to box 1108, where it is determined whether the status is available for sealing. When a user manually enters the status for sealing into a computing device or scans the status of sealing (e.g., from a barcode or other mark on a training packaged item), the status may be available for sealing. If it is determined at box 1108 that the status is available, the method proceeds to box 1110. At box 1110, the classification model is updated based on the image data and the status for sealing. Updating the classification model may include any of the methods described herein for training and / or developing a classification model. The sealing status (e.g., the manually entered status) is available, as shown in box 1112. However, if it is determined at box 1106 that the classification model is not in training mode, or if it is determined at box 1108 that the status is not available, the method proceeds to box 1114.

[0279] At box 1114, the classification model classifies the state of the seal. In some embodiments, the state of the seal classified by the classification model includes a determination of whether the seal is defective (e.g., wrinkles or a weak seal), non-defective, or has a specific defect, and an indication of the degree of certainty as to whether the seal is defective, non-defective, or has a specific defect. At box 1116, it is determined whether the confidence level of the classified state is low or high. In some embodiments, the confidence level is a percentage of the degree of certainty indicating that the classified state of the seal is accurate, and if the degree of certainty is less than a predetermined percentage of an acceptable degree of certainty, the confidence level is low. For example, if the acceptable degree of certainty is 90%, then if the degree of certainty of the classified state is less than 90%, the classified state of the seal is considered to be low. If the confidence level is determined to be not low at box 1116, then the seal state has been determined, as shown at box 1118. However, if the handling confidence level is determined to be low at block 1116, the method proceeds to block 1120 where the sealed product is singled out for offline and / or manual sorting (e.g., sorting by a user after visual inspection or physical testing separate from the production line). In embodiments where redundant or multiple vision systems of different types (e.g., thermal and photoelastic) are utilized, the likelihood of increased confidence is enhanced.

[0280] If the status of the seal is available at block 1112 or at block 1118, the method proceeds to block 1122. At block 1122, the status of the seal is output. In some embodiments, outputting the status of the seal includes one or more of displaying the status of the seal on a user interface output device, transmitting the status of the seal to one or more external devices via a communication interface, or storing the status of the seal in a database. In some embodiments, the status of the seal includes one or more of an indication of whether the seal is defective, non-defective, or has a specific defect or a degree of certainty that the seal is defective, non-defective, or has a specific defect.

[0281] Whether it is the state of outputting the seal at frame 1122 or keeping the packaged items at frame 1120 for manual sorting, method 1100 then proceeds to frame 1124. At frame 1124, determine whether another packaged item is available. In some embodiments, the determination at frame 1124 can be based on whether another packaged item is detected on the conveyor (e.g., via one or more sensors). In some embodiments, the determination at frame 1124 can be based on whether the user inputs an indication of whether another packaged item is available. If it is determined at frame 1124 that another packaged item is unavailable, then the visual system and classification model at frame 1126 are closed. However, if it is determined at frame 1124 that another packaged item is available, then method 1100 loops back to frame 1104, at frame 1104, collect the image data of the seal of the next packaged item, and as described above, method 1100 proceeds from frame 1104, for the next packaged item.

[0282] As discussed above, the artificial intelligence algorithm for classifying the state of a seal for a packaged article based on image data may include a decision process, such as a decision tree or a neural network. In some embodiments, the artificial intelligence algorithm for classifying the state of a seal based on image data may include more than one decision process. Fig.21 1200 . In the depicted embodiment, the method 1200 is performed in part by an image sensor system 1202, a detection decision process 1204, a classification decision process 1206, and an output device 1208. At block 1210, the image sensor system acquires image data of the seal. In some embodiments, the image sensor system 1202 may acquire image data while the packaged item is being transported by a transport system. After the image data is acquired at block 1210, the image sensor system has image data 1212 that can be transmitted to the detection decision process 1204. In one embodiment, the detection decision process 1204 is a software-based decision process operating on one or more computing devices.

[0283] At box 1214, the detection decision process 1204 processes the image data received from the image sensor system 1202. In some embodiments, the processing of the image data at box 1214 is performed by an artificial intelligence algorithm that has been trained to detect an area of ​​interest associated with a seal of a packaged item in the image data. In some embodiments, the processing of the image data at box 1214 includes one or more of: cropping an image in the image data around one or more seals detected in the image, selecting a frame or a subset of frames from a video in the image data, identifying irrelevant pixels from an image in the image data, and replacing irrelevant pixels with the lowest significant value of the image data. In some embodiments, the processing of the image data produces a single image having a rectangular shape, wherein the identified seal is substantially centered in the image, and pixels that are considered irrelevant are replaced with the lowest significant value. In some embodiments, the processing of the image data may include masking portions of the image, wherein image areas outside the area of ​​interest (e.g., outside the seal) are replaced with low-value data (e.g., pixels are all black) to reduce the amount of processing used to classify the state of the seal and reduce the possibility of errors in classifying the state of the seal.

[0284] In one embodiment of processing image data, a custom boundary is constructed around a representation of a seal in the image data. A bounding box containing the seal is also constructed in the custom boundary. Processing also includes cropping the bounding box from the entire image data. One advantage of cropping image data based on a custom boundary is that the later classification of the state of the seal can be limited to the area of ​​interest without the need to examine the uninteresting area of ​​the image data. This in turn can increase the confidence level of the classification and thus improve the overall performance of the classification. In some embodiments in which the detection decision process 1204 is a multi-layer neural network, creating a bounding box around the custom boundary simplifies the compatibility requirements between the image data and the first layer of the neural network. In addition, cropping the image data causes a portion of the image data to be processed for classification, rather than processing the entire image data, which reduces the processing time for classification. In some embodiments, the custom boundary can help generate a numerical value for one or more of the sealed area, its centroid, or its orientation.

[0285] At block 1216, a determination is made as to whether the presence of a seal is detected in the image data. In some embodiments, the determination made at block 1216 is part of the processing of the image data at block 1216. In some embodiments, the determination as to whether a seal is detected at block 1216 is a separate process from the processing of the image data at block 1216. If it is determined at block 1216 that the presence of a seal is not detected, then the method 1200 proceeds to block 1218 where the image data is discarded (e.g., deleted) and the method 1200 ends. However, if it is determined at block 1216 that the presence of a seal is detected, then the processed image data represented at block 1220 may be transmitted to the classification decision process 1206. In some embodiments, the classification decision process 1206 is a software-based decision process operating on one or more computing devices, which may be the same or different than the one or more computing devices on which the detection decision process 1204 operates. In some embodiments, processing the image data at box 1214 to obtain processed image data (as shown at box 1220) prior to classifying the state of the seal represented in the data increases the evaluation matrix for classification later performed by the classification decision process 1206.

[0286] At block 1222, the classification decision process 1206 classifies the processed image data received from the detection decision process 1204. In some embodiments, the classification of the image data at block 1222 is performed by an artificial intelligence algorithm that has been trained to classify the state of the seal represented in the processed image data. In some embodiments, the classification of the state of the seal represented in the processed image data at block 1222 includes a determination of whether the seal is defective, non-defective, or has a specific defect. In some embodiments, the classification of the state of the seal represented in the processed image data at block 1222 includes a determination of whether the seal is defective, non-defective, or has a specific defect, and an indication of the degree of certainty as to whether the seal is defective, non-defective, or has a specific defect.

[0287] At box 1224, it is determined whether the confidence level of the classified state is low. In some embodiments, the confidence level is a percentage of certainty that the classified state of the seal is accurate, and if the certainty is below a predetermined percentage of an acceptable certainty, the confidence level is low. For example, if the acceptable certainty is 90%, then if the certainty of the classified state is below 90%, then the classified state of the seal is considered low. If the handling confidence level is determined to be not low at box 1224, then the seal state has been determined, as shown at box 1226. However, if the handling confidence level is determined to be low at box 1224, then the method proceeds to box 1228, where the seal and / or image data is marked for manual classification.

[0288] At box 1230, the state of the seal is manually classified outside the classification decision process. In some embodiments, the seal is manually classified by the user after visual inspection or physical testing of the packaged item. At box 1232, the user inputs the state of the manually classified seal into the classification decision process 1206. At box 1234, the classification decision process 1206 is updated. In embodiments where the classification decision process 1206 is an artificial intelligence algorithm, updating the classification decision process 1206 includes further training the artificial intelligence algorithm based on the manual classification. After updating the classification decision process 1206, the method 1100 returns to box 1226, where the classified state of the vacuum packaged product is the manually classified state of the seal.

[0289] After the status of the classification of the vacuum packaged product as represented at block 1226 is classified or obtained by the classification decision process 1206, the classification decision process 1206 sends the status of the classification of the seal to an output device 1208. In embodiments where the classification decision process 1206 is software operating on one or more computing devices, the output device 1208 may be a user interface output device. In some embodiments, outputting the status of the classification of the seal at block 1236 includes one or more of outputting the status of the classification of the seal to a user via a user interface (e.g., a monitor, a touch screen, etc.), transmitting the status of the classification of the seal to an external device via a communication circuit, or storing the status of the classification of the seal locally in a database.

[0290] In any of the embodiments disclosed herein, the image data received for any one seal may include image data in multiple forms about the same seal. For example, the image data about the seal may include two images within the visible light range of the same seal. In another embodiment, the image data about the seal may include three images, one is a thermal imaging image, one is an ultraviolet image, and one is a photoelastic image. These multiple different forms of image data for the same seal can be passed through an artificial intelligence algorithm alone. If the artificial intelligence algorithm uses multiple different forms of image data to return the same classified state for the seal, the confidence level of the classification for the seal can be significantly increased. In one example, if the artificial intelligence algorithm classifies one of the images as a packaged item with an imperfect seal at a confidence level of 98%, and classifies another image as a packaged item with an imperfect seal at a confidence level of 96%, the confidence level that the packaged item has an imperfect seal may be greater than 99%. In another example, if the artificial intelligence algorithm classifies one of the images as having a flawless seal at a confidence level of 60%, and classifies the other image as having a flawless seal at a confidence level of 70%, the confidence level that the vacuum-packed product is flawless may be 88%. Even though the confidence level using two images may be significantly higher than either image alone, the combined confidence level from the two images (e.g., 88%) may still be lower than a predetermined percentage (e.g., 95%) of acceptable certainty, which may cause the packaged item to be marked for manual classification. In some further embodiments, multiple camera angles may be used to image the seal on multiple surfaces and from multiple viewpoints, such as through the packaged item, at an angle to the packaged item, or directly above the packaged item. In some embodiments, two or more camera angles may be used to image the same seal. It will be apparent that the number of multiple forms of image data is not limited to two, but may be any number of forms of image data.

[0291] In some embodiments, not every area of ​​increased or altered image indicators or defects is necessary in detection and reporting, and a user or computer device may determine one or more thresholds to facilitate identification of defective seals. The thresholds may be based on any of the parameters detailed herein, and may be predetermined or applied as a result of an algorithm or model. The thresholds may be set so that only altered image indicators and defects exceeding a threshold size are marked for removal. For example, a threshold may be set at a fluorescent area having a size of at least 2 mm in at least one direction (i.e., altered fluorescence having a size of at least 2 mm in the machine direction and / or at least 2 mm in the cross direction). Alternatively, a threshold may be set at a thermal image anomaly having a size of at least 1 mm in at least one direction (i.e., altered fluorescence having a size of at least 1 mm in at least one direction). In some embodiments, the threshold may be set at a predetermined surface area (e.g., at least 1 mm 2 , 2mm 2 , 3 mm 2 or 4 mm 2 Such a threshold may be set even though the system has the ability to see defects as small as 10 microns in size in at least one direction. The setting of the threshold is distinct from the ability of the machine vision system to detect defects up to at least a certain size in at least one direction. Rather, the setting of the threshold is the setting of a minimum value for the size of the area in response to the generation of this trigger signal. The threshold may be set to any desired value and is distinct from the ability of the machine vision system to detect defects at least down to a specified size.

[0292] In some embodiments, the algorithm or model based detection means disclosed herein can be used to detect defects in packaged items. For example, in some embodiments, the algorithm or model based detection means described herein can detect areas with high fluorescence that is higher than a large portion of the seal, which can indicate wrinkles or other defects in the seal. In some embodiments, the algorithm or model based detection means can detect substantially uniform, constant fluorescence from the seal, indicating a strong seal. In some embodiments, areas with less or no fluorescence in the seal can indicate discontinuities or weak spots in the seal.

[0293] In some embodiments, the algorithm or model based detection means disclosed herein can be used to detect defects in packaged items. For example, in some embodiments, the algorithm or model based detection means described herein can detect areas with high concentrations of thermal radiation that are higher than the bulk of the seal, which can indicate wrinkles or other defects in the seal. In some embodiments, the algorithm or model based detection means can detect a substantially uniform, constant thermal image from the seal, indicating a strong seal. In some embodiments, areas with less or no thermal radiation in the seal can indicate a discontinuity or weakness in the seal.

[0294] In some embodiments, the algorithm or model-based detection means disclosed herein can be used to detect defects in packaged items. For example, in some embodiments, the algorithm or model-based detection means described herein can detect defects visible under a photoelastic image.

[0295] Example

[0296] In an example embodiment, the algorithm and image processing may include some or all of the following steps: (1) capturing an image; (2) applying denoising to the image; (3) converting the image to grayscale; (4) applying thresholding to reduce the area of ​​the image to be considered; (5) eliminating undesirable areas of the image; (6) identifying features in the remaining image that correspond to a defective seal in the packaged item. Examples of each of these processing steps are shown and described in the following figures.

[0297] Go to Fig. 22 , thermal image data showing a fully sealed seal. An impulse seal is formed in a packaged article. It is known that impulse sealing produces a non-linear or wavy seal area. This image was captured 5 seconds after the seal was created. Uniform active cooling was utilized after sealing and before the image was captured. Such a fully sealed image can be used as a good seal image for artificial intelligence algorithms. Fig.23 is an exemplary image of thermal image data of a defective seal. An impulse seal was formed in a packaged item. This image was captured 5 seconds after the seal was created. Uniform active cooling was utilized after the seal and before the image was captured. This image and similar defective seal images can be used to train an artificial intelligence algorithm to indicate defective or inadequate seals.

[0298] Fig.23 is an exemplary thermal image showing a double straight line seal with localized seal wrinkles. An impulse seal is formed in the packaged item. This image was captured 5 seconds after the seal was created. Uniform active cooling was utilized after the seal and before the image was captured. Discontinuities in the thermal image along the seal area indicate a cold and weak seal. Such images are useful in training artificial intelligence algorithms as described herein.

[0299] Fig.24 is an exemplary thermal image showing a double straight line seal with a weak seal area. An impulse seal is formed in the packaged item. This image was captured 5 seconds after the seal was created. Uniform active cooling was utilized after the seal and before the image was captured. Discontinuities in the thermal image along the seal area indicate a cold and weak seal. Such images are useful in training artificial intelligence algorithms as described herein.

[0300] Now go to Fig.25 and Fig.26, showing an image of thermal image data indicating an acceptable seal. Fig.25 and Fig.26 Here are images of the same seal taken at two different locations and at two different points in time. The seal area emits fairly consistent thermal radiation along the seal without any hot or cold spots or areas. The heat in the seal area is residual heat from the sealing process and dissipates over time. Fig.25 The image was acquired 15 s after the heat sealing process. Fig.26 The image is acquired 20 s after heat sealing. The time to capture the thermal image after heat sealing can be adjusted depending on the film, packaging items, thickness of the film, heat sealing temperature and method.

[0301] refer to Fig. 27 and Fig.28 , showing an image of thermal image data indicating a defective seal. Fig.25 and Fig.26 3101, 3102, 3201, and 3202. The seal area emits fairly consistent thermal radiation along the seal, except for hot spots 3101, 3102, 3201, and 3202. The heat in the seal area is residual heat from the sealing process and dissipates over time. Fig. 27 The image was acquired 15 s after the heat sealing process. Fig.28 The image is taken 20s after heat sealing. The time to capture the thermal image after heat sealing can be adjusted depending on the film, packaging items, film thickness, heat sealing temperature and method.

[0302] Now go to Fig.29 , showing a seal area captured by a photoelastic imaging system such as described herein. Photoelastic image capture mechanisms provide enhanced visual capabilities compared to standard image capture techniques. Fig.29 Indicates a proper seal without any defects.

[0303] Fig.30 and Fig.31 Indicates the seal area captured by the photoelastic imaging system. These images have improved clarity of wrinkle defects compared to standard imaging. The visual inspection engine analyzes the images for changes in color gradients of adjacent pixels, segment comparisons, etc. The visual inspection engine can compare multiple pixels or segments of the image with respect to one or more of the following: average, variance, skew, minimum, maximum, range, or variation in the seal area. Variations between pixels or segments of the image can indicate a seal defect.

[0304] Fig.32An acquired UV image is shown of a packaged article having an optical brightener disposed therein. The optical brightener fluoresces with greater intensity at a seal area containing a wrinkle defect. Additional material is pleated around the wrinkle defect, causing additional illumination intensity. The visual inspection engine analyzes the image for changes in color gradients of adjacent pixels, segment comparisons, and the like. The visual inspection engine may compare multiple pixels or segments of the image with respect to one or more of the following: average, variance, skew, minimum, maximum, range, or variation in the seal area. Variations between pixels or segments of the image may indicate a seal defect.

[0305] in conclusion

[0306] The technical personnel in the field to which the inventions set forth herein relate, who benefit from the teachings presented in the foregoing description and the associated drawings, will think of many modifications and other embodiments of these inventions. Therefore, it will be understood that the present invention is not limited to the specific embodiments disclosed, and modifications and other embodiments are intended to be included within the scope of the appended claims. In addition, although the foregoing description and the associated drawings describe example embodiments in the context of certain example combinations of elements and / or functions, it should be appreciated that alternative embodiments may provide different combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, for example, it is also envisioned that the combination of elements and / or functions different from those explicitly described above, as may be set forth in some of the appended claims. Although specific terms are used herein, they are used only in a general and descriptive sense, and not for the purpose of limitation.

Claims

1. A process for detecting defective seals of packaged articles, comprising: A) heat sealing a polymeric multilayer film having a total free shrink of less than 10% in the machine or transverse direction at 85°C to itself or another polymeric multilayer film to form a packaged article by forming at least one sealed area; B) acquiring a first set of image data of the at least one sealing area using a first vision system including an image capture device, the first vision system being a thermal imaging vision system including an infrared imaging device capable of capturing a temperature distribution based on an amount of infrared radiation emitted from the sealing area, the first set of image data being time-lapse thermal imaging data captured at a time after formation of the at least one sealing area; wherein the first set of image data is acquired between 2 and 30 seconds after the formation of the at least one sealing area; C) collecting a second set of image data of the at least one sealing area using a second visual system, the second visual system being different from the first visual system, the second visual system comprising: a. a first linear polarizer having a polarization direction oriented in a first direction; b. a second linear polarizer having a polarization direction oriented orthogonal to said first direction; c. light source; and d. an image capture device for capturing said second set of image data, said second set of image data being photoelastic image data; D) evaluating the first set of image data and the second set of image data of the seal area using a visual inspection engine to verify the continuity of the seal, the strength of the seal, or both the continuity and strength of the seal area.

2. The process of claim 1, wherein: The forming of at least one sealing area is formed by heat-generated sealing.

3. The process of claim 2, wherein: The heat generating seal is a non-linear seal.

4. A process according to claim 2 or 3, wherein: The heat generated seal is selected from at least one of the group consisting of: impulse sealing, ultrasonic sealing, laser sealing, and heat sealing.

5. The process according to any one of the preceding claims 1 to 3, wherein: The polymeric multilayer film includes at least one layer comprising a fluorescence-based indicator.

6. The process of claim 5, further comprising a third vision system, the third vision system being an ultraviolet vision system further comprising an ultraviolet light source.

7. The process of claim 6, wherein: The third visual system further includes a blue light bandpass filter.

8. The process of claim 6, wherein: The ultraviolet vision system further includes a white light source.

9. The process of claim 8, wherein: The ultraviolet vision system illuminates the white light source and the ultraviolet light source in a variable pattern.

10. The process of claim 5, wherein: The image capture device is capable of capturing visible images in the visible spectrum.

11. The process of claim 5, further comprising the steps of: C) exposing the packaged article to incident radiation to excite the fluorescence-based indicator such that the fluorescence-based indicator fluoresces; D) while the indicator emits fluorescence, collecting image data of the fluorescence emitted from the sealing area by the packaged article.

12. The process of claim 11, wherein: The fluorescence-based indicator includes at least one selected from the group consisting of: an ultraviolet indicator, an infrared indicator, a dye, a pigment, an optical brightener, a fluorescent whitening agent, 2,2'-(2,5-thiophenediyl)bis(5-tert-butylbenzoxazole), hydroxy-4-(p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes.

13. A process according to claim 11 or 12, wherein: The fluorescence-based indicator is present in at least one layer of the polymeric multilayer film, wherein the fluorescence-based indicator is present at a level of from 0.5 to 1000 ppm based on the weight of the layer.

14. A process according to claim 11 or 12, wherein: The fluorescence-based indicator is present in at least one layer of the polymeric multilayer film, wherein the fluorescence-based indicator is present at a level of from 5 to 10 ppm based on the weight of the layer.

15. The process of claim 11 or 12, wherein: The visual inspection engine includes a computing device including computer-executable instructions configured to determine whether fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator indicates a defective seal.

16. The process of claim 15, wherein: The computer executable instructions include at least one artificial intelligence algorithm selected from the group consisting of supervised, unsupervised, or semi-supervised methods.

17. The process of claim 16, further comprising the step of training the artificial intelligence algorithm using a set of training packaged item images including defective and acceptable seals.

18. The process of claim 11 or 12, wherein: Determining that the seal is defective includes determining that the polymeric multilayer film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to a second region of the seal.

19. The process of claim 11 or 12, wherein: Determining that the seal is defective includes determining that the polymeric multilayer film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to an expected level of fluorescence.

20. The process of claim 11 or 12, wherein: Determining that the seal is defective includes determining at least one of: (i) the polymeric multilayer film exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to a second area of ​​the seal, (ii) the polymeric multilayer film exhibits a higher or lower fluorescence intensity in the first area of ​​the seal relative to an expected level of fluorescence, or (iii) both (i) and (ii).

21. The process of claim 1 or 2, wherein: The first set of image data is acquired between 5 and 20 seconds after the forming of the at least one sealing area.

22. The process of claim 1 or 2, further comprising an active cooling system.

23. The process of claim 1 or 2, wherein: The visual inspection engine includes a computing device including computer-executable instructions configured to determine whether captured thermal image data of the seal indicates a defective seal.

24. The process of claim 23, wherein: The computer executable instructions include at least one artificial intelligence algorithm selected from the group consisting of supervised, unsupervised, or semi-supervised methods.

25. The process of claim 24, further comprising the step of training the artificial intelligence algorithm using a set of training packaged item images including defective and acceptable seals.

26. The process of claim 1 or 2, wherein: Determining that the seal is defective includes determining that the polymeric multilayer film exhibits a higher or lower thermal temperature in a first region of the seal relative to a second region of the seal.

27. The process of claim 1, wherein: The first linear polarizer is integrated into the light source.

28. The process of claim 1, wherein: The second linear polarizer is integrated into the image capture device.

29. The process of claim 1, wherein: The light source is a white light source having a wavelength spectrum from 400 nm to 700 nm.

30. The process of claim 29, wherein: The white light source is a diffuse white light source.

31. The process of claim 1, wherein: The second vision system includes a light diffuser positioned between the light source and the packaged article.

32. The process of claim 1, wherein: Acquiring photoelastic image data is performed synchronously with the forming of the packaging article.

33. The process of claim 1 or 2, wherein: Determining that the seal is defective indicates at least one of a gap in the seal of the packaged article, a wrinkle in the seal of the packaged article, a weak seal, or a cold seal.

34. The process of claim 1 or 2, wherein: The product placed in the packaged article is a food product.

35. The process of claim 34, wherein: The food product includes at least one selected from the group consisting of meat and cheese.

36. The process of claim 5, wherein: The polymeric multilayer film comprises a functional layer; and wherein the fluorescence-based indicator is present in the functional layer.

37. The process of claim 36, wherein: The functional layer is one selected from the group consisting of: an oxygen barrier layer, a sensory barrier layer, a moisture barrier layer, a harmful chemical barrier layer, a microbial barrier layer, an acid layer, an acid salt layer, a bacteriocin layer, a bacteriophage layer, a metal layer, a metal salt layer, a natural oil layer, a natural extract layer, a layer containing polyhexamethylene biguanide hydrochloride, a layer containing parabens, a layer containing grafted silane-quaternary ammonium, a layer containing triclosan, and a layer containing zeolite containing silver, copper and / or zinc.

38. The process of claim 36, wherein: The functional layer is an oxygen barrier layer including at least one selected from the group consisting of a vinylidene chloride copolymer, a saponified ethylene / vinyl acetate copolymer, a polyamide, a polyester, an oriented polypropylene, and an ethylene homopolymer.

39. The process of claim 1, wherein: The polymeric multilayer film comprises: A) a first layer, which is a first outer film layer configured to function as a heat seal layer; B) a second layer, which is a second outer layer configured to function as a durable layer; C) a third layer between the first layer and the second layer, wherein the third layer is configured to function as a functional layer; D) a fourth layer between the first layer and the third layer, wherein the fourth layer is configured to function as a first bonding layer; and E) a fifth layer between the second layer and the third layer, wherein the fifth layer is configured to function as a second bonding layer.

40. The process of claim 1 or 2, wherein: The packaging article is selected from at least one of the group consisting of an end seal bag, a side seal bag, a pouch, or a back seam packaging article.

41. The process of claim 1 or 2, wherein: In an instance where a defective seal is detected, the process further includes generating a signal comprising at least one selected from the group consisting of: an alarm, a package marking, an image showing the defective seal, generating a report, identifying the packaged item, and diverting the packaged item.

42. The process of claim 1 or 2, wherein: At least a portion of the first vision system and / or the second vision system is contained within a housing configured to block at least 50% of ambient light.

43. The process of claim 1 or 2, wherein: At least portions of the first vision system and / or the second vision system are contained within a housing configured to block at least 85% of ambient light.

44. The process of claim 1 or 2, wherein: The first set of image data and / or the second set of image data of the at least one sealing area are captured by the image capture device at a rate of at least 5 images per second.

45. The process of claim 1 or 2, wherein: The first set of image data and / or the second set of image data of the at least one sealing area are captured by the image capture device at a rate of at least 30 images per second.

46. ​​The process of claim 1 or 2, wherein: The first set of image data and / or the second set of image data of the at least one sealing area are captured by the image capture device at a rate of at least 100 images per second.

47. The process of claim 1 or 2, wherein: The first set of image data and / or the second set of image data of the at least one sealing area are captured by the image capture device at a rate of at least 250 images per second.

48. The process of claim 1 or 2, wherein: The first set of image data and / or the second set of image data of the at least one sealing area are captured by the first vision system and / or the second vision system before the packaged product is sent away.

49. The process of claim 1 or 2, wherein: The visual inspection engine assigns a seal score value to the first set of image data and / or the second set of image data for the seal area.

50. The process of claim 49, wherein: The visual inspection engine compares the seal score value of the first set of image data and / or the second set of image data of the seal area to a threshold value.

51. The process of claim 50, wherein: If the seal score value exceeds a threshold, the visual inspection engine sends instructions to a seal defect mechanism.

52. The process of claim 49, wherein: The visual inspection engine assigns a confidence level to the seal score value.

53. A system for detecting a defective seal of a packaged article, comprising: A) a heat sealing mechanism configured to seal a polymeric multilayer film having a total free shrink of less than 10% in the machine or cross direction at 85°C to itself or another polymeric multilayer film to form a packaged article by forming at least one sealed area; B) a first vision system comprising an image capture device, the image capture device being configured to collect a first set of image data of the at least one sealing area, the first vision system being a thermal imaging vision system comprising an infrared imaging device, the infrared imaging device being capable of capturing a temperature distribution based on an amount of infrared radiation emitted from the sealing area, the first set of image data being time-lapse thermal imaging data captured at a time after formation of the at least one sealing area; wherein the first set of image data is acquired between 2 and 30 seconds after the formation of the at least one sealing area; C) a second vision system comprising an image capture device configured to acquire a second set of image data of the at least one sealing area, the second vision system being different from the first vision system, the second vision system comprising: a. a first linear polarizer having a polarization direction oriented in a first direction; b. a second linear polarizer having a polarization direction oriented orthogonal to said first direction; c. light source; and d. an image capture device for capturing said second set of image data, said second set of image data being photoelastic image data; D) evaluating the first set of image data and the second set of image data of the seal area using a visual inspection engine to verify the continuity of the seal, the strength of the seal, or both the continuity and strength of the seal area.

54. The system of claim 53, wherein: The heat sealing mechanism forms a heat seal.

55. The system of claim 53, wherein: The polymeric multilayer film includes at least one layer comprising a fluorescence-based indicator.

56. The system of claim 53, further comprising a third visual system, the third visual system being an ultraviolet visual system further comprising an ultraviolet light source.

57. The system of claim 56, wherein: The third visual system includes a blue light bandpass filter.

58. The system of claim 55, further comprising: A) exposing the packaged article to incident radiation to excite the fluorescence-based indicator such that the fluorescence-based indicator fluoresces; B) while the indicator emits fluorescence, collecting image data of the fluorescence emitted from the sealing area by the packaged article.

59. The system of claim 58, wherein: The fluorescence-based indicator includes at least one selected from the group consisting of: an ultraviolet indicator, an infrared indicator, a dye, a pigment, an optical brightener, a fluorescent whitening agent, 2,2'-(2,5-thiophenediyl)bis(5-tert-butylbenzoxazole), hydroxy-4-(p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes.

60. The system of claim 58, wherein: The visual inspection engine includes a computing device including computer-executable instructions configured to determine whether fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator indicates a defective seal.

61. The system of claim 60, wherein: The computer executable instructions include at least one artificial intelligence algorithm selected from the group consisting of supervised, unsupervised, or semi-supervised methods.

62. The system of claim 60, wherein: Determining that the seal is defective includes determining that the polymeric multilayer film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to a second region of the seal.

63. The system of claim 60, wherein: Determining that the seal is defective includes determining that the polymeric multilayer film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to an expected level of fluorescence.

64. The system of claim 60, wherein: Determining that the seal is defective includes determining at least one of: (i) the polymeric multilayer film exhibits a higher or lower fluorescence intensity in a first area of ​​the seal relative to a second area of ​​the seal, (ii) the polymeric multilayer film exhibits a higher or lower fluorescence intensity in the first area of ​​the seal relative to an expected level of fluorescence, or (iii) both (i) and (ii).

65. The system of claim 53, wherein: The visual inspection engine includes a computing device including computer-executable instructions configured to determine whether captured thermal image data of the seal indicates a defective seal.

66. The system of claim 65, wherein: Determining that the seal is defective includes determining that the polymeric multilayer film exhibits a higher or lower thermal temperature in a first region of the seal relative to a second region of the seal.

67. The system of claim 53, wherein: The first set of image data and / or the second set of image data of the at least one sealing area are captured by the image capture device at a rate of at least 30 images per second.

68. The system of claim 53, wherein: The visual inspection engine assigns a seal score value to the first set of image data and / or the second set of image data for the seal area.

69. The system of claim 68, wherein: The visual inspection engine compares the seal score value of the first set of image data and / or the second set of image data of the seal area to a threshold value.

70. The system of claim 68, wherein: The visual inspection engine assigns a confidence level to the seal score value.

Images