A system for in-line inspection of seal integrity

Abstract

A process for monitoring seal quality of a packaged article. In one embodiment, the process for monitoring seal quality includes sealing a film to form a packaged article and forming a seal area. The seal area is analyzed by a vision system to obtain image data of the seal area. A vision inspection engine analyzes the image data to determine seal continuity, seal strength, or both.

Description

[Technical field]

[0001] The subject matter disclosed herein relates to systems and processes for inspecting packaged products for quality assurance to ensure the seal integrity of the packaged products. [Background technology]

[0002] Flexible films are frequently used in packaging because they can conform to the surface of the product. The quality of the packaging depends on the quality of the film material and the effective closure of the seal. Proper seal quality means complete fusion of the two opposing seal layers. In heat sealing, this fusion is achieved by applying a specific combination of temperature and pressure for a period of time, allowing the long chain molecules of the seal layers to bond. Seal defects can compromise the seal by an improper combination of sealing parameters or by the presence of water vapor, bubbles, wrinkles, or product contamination in the seal. Such seal defects can be burnt seals, cold seals, weak seals, pleats, holes, contamination of the seal area, seal burns, seal voids, and localized defects. Seal defects can allow air and microorganisms to enter the package and spoil the enclosed food. They can also adversely affect the appearance of the seal and cause a loss of consumer confidence in the integrity of the package.

[0003] Detecting faulty seals remains today a labor-intensive, offline, destructive process. Current seal test methods are described in ASTM F 1921 "Standard Test Methods for Hot Seal Strength (Hot Tack) of Thermoplastic Polymers and Blends Comprising the Sealing Surfaces of Flexible Webs", ASTM F 88 "Standard Test Method for Seal Strength of Flexible Barrier Materials", ASTM F 2054 "Standard Test Method for Burst Testing of Flexible Package Seals Using Internal Air Pressurization Within Restraining Plates" and ASTM F 2095 "Standard Test Methods for Pressure Decay Leak Test for Flexible Packages With and Without Restraining Plates". Traditional test methods are destructive and can only be performed offline. Furthermore, these processes only allow testing of a limited number (e.g., less than 1%) of packaged articles and result in the destruction of the packaged articles.

[0004] It would be desirable to be able to check packaged product seals in a non-destructive process, and further, to be able to check the seals in-line with the production of the packaged product or the packaging of articles within the packaged product prior to delivery of the packaged product.

[0005] The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] ASTM F 1921 “Standard Test Methods for Hot Seal Strength(Hot Tack) of Thermoplastic Polymers and Blends Comprising the Sealing Surfaces of Flexible Webs” [Non-Patent Document 2] ASTM F 88 “Standard Test Method for Seal Strength of Flexible Barrier Materials” [Non-Patent Document 3] ASTM F 2054 “Standard Test Method for Burst Testing of Flexible Package Seals Using Internal Air Pressurization Within Restraining Plates” [Non-Patent Document 4] ASTM F 2095 “Standard Test Methods for Pressure Decay Leak Test for Flexible Packages With and Without Restraining Plates” Summary of the Invention [Means for solving the problem]

[0007] A process for monitoring seal quality of a packaged article. In one embodiment, the process for monitoring seal quality includes sealing a film to form a packaged article and forming a seal area. The seal area is analyzed by a vision system to obtain image data of the seal area. A vision inspection engine analyzes the image data to determine seal continuity, seal strength, or both.

[0008] In one exemplary embodiment, a process for monitoring seal quality of a packaged article is disclosed. The process for monitoring seal quality of a packaged article includes: A) sealing the film to itself, another film, or a packaging support to form a packaged article by forming at least one seal area; B) acquiring image data of at least one seal area using a vision system comprising an image capture device; C) evaluating image data of the seal area with a vision inspection engine to verify seal continuity, seal strength, or both continuity and strength of the seal area. Equipped with.

[0009] In an exemplary embodiment, forming the at least one sealed area is formed by heat sealing.

[0010] In an exemplary embodiment, the heat seal is a non-linear seal.

[0011] In an exemplary embodiment, the exothermic seal is selected from the group consisting of an impulse seal, an ultrasonic seal, a laser seal and a heat seal.

[0012] In an exemplary embodiment, the film includes at least one layer that includes a fluorescent-based indicator.

[0013] In an exemplary embodiment, the vision system further comprises a blue bandpass filter.

[0014] In an exemplary embodiment, the vision system is an ultraviolet vision system further comprising an ultraviolet light source.

[0015] In an exemplary embodiment, the ultraviolet vision system further comprises a white light source.

[0016] In an exemplary embodiment, an ultraviolet vision system illuminates a white light source and an ultraviolet light source in a variable pattern.

[0017] In an exemplary embodiment, the image capture device is capable of capturing visual images in the visible spectrum.

[0018] In an exemplary embodiment, a process for monitoring seal quality of a packaged article includes: A) exposing the packaged article to incident radiation to excite the fluorescence-based indicator such that the fluorescence-based indicator fluoresces; B) acquiring image data of the fluorescence emitted from the seal area by the packaged article while the indicator is fluorescing. Further includes:

[0019] In an exemplary embodiment, the fluorescence-based indicator comprises at least one member selected from the group consisting of ultraviolet indicators, infrared indicators, dyes, pigments, optical brighteners, optical brighteners, 2,2'-(2,5-thiophenylenediyl)bis(5-tert-butylbenzoxazole), hydroxyl-4-)p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes.

[0020] In an exemplary embodiment, the fluorescent-based indicator is present in at least one layer of the film and is present at a level of from 0.5 to 1000 ppm based on layer weight.

[0021] In an exemplary embodiment, the fluorescent-based indicator is present in at least one layer of the film and is present at a level of from 5 to 10 ppm based on layer weight.

[0022] In an exemplary embodiment, the visual inspection engine comprises a computing device including computer-executable instructions configured to determine whether fluorescent electromagnetic energy emitted by an excited fluorescence-based indicator is indicative of a defective seal.

[0023] In an exemplary embodiment, the computer executable instructions include an artificial intelligence algorithm.

[0024] In an exemplary embodiment, the artificial intelligence algorithm has a training set of training packaging articles before analyzing the image data.

[0025] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescent intensity in a first region of the seal relative to a second region of the seal.

[0026] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to an expected level of fluorescence.

[0027] In an exemplary embodiment, determining that the seal is defective includes determining at least one of: (i) the film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to a second region of the seal; (ii) the film exhibits a higher or lower fluorescence intensity in the first region of the seal relative to an expected level of fluorescence; or (iii) both (i) and (ii).

[0028] In an exemplary embodiment, the image data is time-delayed thermographic data captured once after formation of the at least one seal area.

[0029] In an exemplary embodiment, the vision system is a thermographic vision system that includes an infrared imaging device capable of capturing a temperature distribution based on the amount of infrared radiation emitted from the seal area.

[0030] In an exemplary embodiment, image data is captured between 2 and 30 seconds after forming the at least one seal area.

[0031] In an exemplary embodiment, image data is captured between 5 and 20 seconds after forming the at least one seal area.

[0032] In an exemplary embodiment, the process further includes an active cooling system.

[0033] In an exemplary embodiment, the vision inspection engine comprises a computing device including computer-executable instructions configured to determine whether captured thermal imaging data of the seal indicates a defective seal.

[0034] In an exemplary embodiment, the computer executable instructions include an artificial intelligence algorithm.

[0035] In an exemplary embodiment, the process further includes generating an artificial intelligence algorithm using a training set of training packaging articles prior to analyzing the image data.

[0036] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower thermal temperature in a first region of the seal relative to a second region of the seal.

[0037] In an exemplary embodiment, the vision system: (i) a first linear polarizer having a polarization direction oriented in a first direction; (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction; (iii) a light source; (iv) an imaging device; The photoelastic vision system includes:

[0038] In an exemplary embodiment, a first linear polarizer is integrated into the light source.

[0039] In an exemplary embodiment, a second linear polarizer is integrated into the imaging device.

[0040] In an exemplary embodiment, the light source is a white light source having a wavelength spectrum from 400 nm to 700 nm.

[0041] In an exemplary embodiment, the white light source is a diffuse white light source.

[0042] In an exemplary embodiment, the process further comprises a light diffuser disposed between the light source and the packaged article.

[0043] In an exemplary embodiment, the acquisition of photoelastic image data occurs in-line with the formation of the packaging article.

[0044] In an exemplary embodiment, the vision system is a first vision system and further comprises a second vision system distinct from the first vision system.

[0045] In an exemplary embodiment, the first vision system is a thermographic vision system comprising an infrared imaging device; and the second vision system is: (i) a first linear polarizer having a polarization direction oriented in a first direction; (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction; (iii) a light source; (iv) an image capture device; The photoelastic vision system includes:

[0046] In an exemplary embodiment, the process further comprises a third vision system, the third vision system being an ultraviolet vision system comprising an ultraviolet light source.

[0047] In an exemplary embodiment, the first vision system is a thermographic vision system comprising an infrared imaging device; and the second vision system is an ultraviolet vision system comprising an ultraviolet light source.

[0048] In an exemplary embodiment, the process further comprises a third vision system, the third vision system: (i) a first linear polarizer having a polarization direction oriented in a first direction; (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction; (iii) a light source; (iv) an image capture device; The photoelastic vision system includes:

[0049] In an exemplary embodiment, the first vision system is an ultraviolet vision system comprising an ultraviolet light source; and the second vision system comprises: (i) a first linear polarizer having a polarization direction oriented in a first direction; (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction; (iii) a light source; (iv) an image capture device; The photoelastic vision system includes:

[0050] In an exemplary embodiment, the process further comprises a third vision system, the third vision system being a thermographic vision system comprising an infrared imaging device.

[0051] In an exemplary embodiment, the vision inspection engine comprises a computing device including computer-executable instructions configured to determine whether visual image data captured of a seal indicates a defective seal.

[0052] In an exemplary embodiment, the film has a total free shrink of less than 10% at 85° C. in either the machine or cross direction.

[0053] In an exemplary embodiment, the film has a total free shrink at 85°C of at least 10% in either the machine direction or the cross direction.

[0054] In an exemplary embodiment, determining that the seal is defective is indicative of at least one of a gap in the seal of the packaging article, a pleat in the seal of the packaging article, a weak seal, or a cold seal.

[0055] In an exemplary embodiment, the product is a food product.

[0056] In an exemplary embodiment, the food product comprises at least one member selected from the group consisting of meat and cheese.

[0057] In an exemplary embodiment, the film is a monolayer film.

[0058] In an exemplary embodiment, the film is a multi-layer film.

[0059] In an exemplary embodiment, the multilayer film includes a functional layer; the fluorescence-based indicator is present in the functional layer.

[0060] In an exemplary embodiment, the functional layer is a member selected from the group consisting of an oxygen barrier layer, an organoleptic barrier layer, a moisture barrier layer, a harmful chemical barrier layer, a microbial barrier layer, an acidic layer, an acid salt layer, a bacteriocin layer, a bacteriophage layer, a metal layer, a metal salt layer, a natural oil layer, a natural extract layer, a layer comprising polyhexamethylene biguanide hydrochloride, a layer comprising parabens, a layer comprising grafted silane-quaternary amine, a layer comprising triclosan, a layer comprising silver, copper and / or zinc zeolite.

[0061] In an exemplary embodiment, the functional layer is an oxygen barrier layer comprising at least one member selected from the group consisting of vinylidene chloride copolymers, saponified ethylene / vinyl acetate copolymers, polyamides, polyesters, oriented polypropylene, and ethylene homopolymers.

[0062] In an exemplary embodiment, the multilayer film comprises: A) a first layer that is a first outer film layer configured to function as a heat seal layer; B) a second layer, the second outer layer configured to function as an abuse layer; C) a third layer between the first layer and the second layer, the third layer configured to function as a functional layer; D) a fourth layer between the first layer and the third layer, the fourth layer configured to function as a first tie layer; E) a fifth layer between the second layer and the third layer, the fifth layer configured to function as a second tie layer; Equipped with.

[0063] In an exemplary embodiment, the packaging article is selected from the group consisting of an end-seal bag, a side-seal bag, a pouch, or a back-seamed packaging article.

[0064] In an exemplary embodiment, if a defective seal is detected, the process further includes generating a signal comprising at least one member selected from the group consisting of an alarm, flagging the package, displaying an image of the defective seal, generating a report, marking the packaged article, and redirecting the packaged article.

[0065] In an exemplary embodiment, at least a portion of the vision system is contained within an enclosure configured to block at least 50% of the ambient light.

[0066] In an exemplary embodiment, at least a portion of the vision system is contained within an enclosure configured to block at least 85% of ambient light.

[0067] In an exemplary embodiment, the film is a monolayer film.

[0068] In an exemplary embodiment, the image data is selected from the group consisting of thermal image data, photoelastic image data, and ultraviolet fluorescent emitted image data.

[0069] In an exemplary embodiment, if a defective seal is detected, the process further includes generating a signal comprising at least one member selected from the group consisting of an alarm, flagging the package, displaying an image of the defective seal, generating a report, marking the packaged article, and redirecting the packaged article.

[0070] In an exemplary embodiment, image data of at least one seal area is captured by an image capture device at a rate of at least 5 images per second.

[0071] In an exemplary embodiment, image data of the at least one seal area is captured by an image capture device at a rate of at least 30 images per second.

[0072] In an exemplary embodiment, image data of the at least one seal area is captured by an image capture device at a rate of at least 100 images per second.

[0073] In an exemplary embodiment, image data of the at least one seal area is captured by an image capture device at a rate of at least 250 images per second.

[0074] In an exemplary embodiment, image data of at least one seal area is captured by a vision system prior to dispensing the packaged product.

[0075] In an exemplary embodiment, a vision inspection engine assigns a seal score value to the image data of the seal area.

[0076] In an exemplary embodiment, a vision inspection engine compares the seal score value of the image data of the seal area to a threshold value.

[0077] In an exemplary embodiment, the vision inspection engine sends instructions to a seal defect mechanism if the seal score value exceeds a threshold value.

[0078] In an exemplary embodiment, a visual inspection engine assigns a confidence level to the seal score value.

[0079] In one exemplary embodiment, a system for detecting a defective seal on a packaged article is disclosed. The system for detecting a defective seal on a packaged article includes: A) a sealing mechanism configured to seal the film to itself, another film, or a packaging support to form a packaged article by forming at least one seal area; B) a vision system comprising an image capture device configured to acquire image data of the at least one seal area using the vision system; C) evaluating image data of the seal area with a vision inspection engine to verify seal continuity, seal strength, or both continuity and strength of the seal area. Equipped with.

[0080] In an exemplary embodiment, the sealing mechanism forms a heat seal.

[0081] In an exemplary embodiment, the film includes at least one layer that includes a fluorescent-based indicator.

[0082] In an exemplary embodiment, the system further comprises a blue bandpass filter.

[0083] In an exemplary embodiment, the vision system is an ultraviolet vision system further comprising an ultraviolet light source.

[0084] In an exemplary embodiment, the system: A) exposing the packaged article to incident radiation to excite the fluorescence-based indicator such that the fluorescence-based indicator fluoresces; B) acquiring image data of the fluorescence emitted from the seal area by the packaged article while the indicator is fluorescing; Further includes:

[0085] In an exemplary embodiment, the fluorescence-based indicator comprises at least one member selected from the group consisting of ultraviolet indicators, infrared indicators, dyes, pigments, optical brighteners, optical brighteners, 2,2'-(2,5-thiophenylenediyl)bis(5-tert-butylbenzoxazole), hydroxyl-4-)p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes.

[0086] In an exemplary embodiment, the visual inspection engine comprises a computing device including computer-executable instructions configured to determine whether fluorescent electromagnetic energy emitted by an excited fluorescence-based indicator is indicative of a defective seal.

[0087] In an exemplary embodiment, the computer executable instructions include an artificial intelligence algorithm.

[0088] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescent intensity in a first region of the seal relative to a second region of the seal.

[0089] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to an expected level of fluorescence.

[0090] In an exemplary embodiment, determining that the seal is defective includes determining at least one of: (i) the film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to a second region of the seal; (ii) the film exhibits a higher or lower fluorescence intensity in the first region of the seal relative to an expected level of fluorescence; or (iii) both (i) and (ii).

[0091] In an exemplary embodiment, the image data is time-delayed thermographic data captured once after formation of the at least one seal area.

[0092] In an exemplary embodiment, the vision system is a thermographic vision system that includes an infrared imaging device capable of capturing a temperature distribution based on the amount of infrared radiation emitted from the seal area.

[0093] In an exemplary embodiment, image data is captured between 2 and 30 seconds after forming the at least one seal area.

[0094] In an exemplary embodiment, the vision inspection engine comprises a computing device including computer-executable instructions configured to determine whether captured thermal imaging data of the seal indicates a defective seal.

[0095] In an exemplary embodiment, determining that the seal is defective includes determining that the film exhibits a higher or lower thermal temperature in a first region of the seal relative to a second region of the seal.

[0096] In an exemplary embodiment, the vision system: (i) a first linear polarizer having a polarization direction oriented in a first direction; (ii) a second linear polarizer having a polarization direction oriented orthogonal to the first direction; (iii) a light source; (iv) an imaging device; The photoelastic vision system includes:

[0097] In an exemplary embodiment, the image data is selected from the group consisting of thermal image data, photoelastic image data, and ultraviolet fluorescent emitted image data.

[0098] In an exemplary embodiment, image data of the at least one seal area is captured by an image capture device at a rate of at least 30 images per second.

[0099] In an exemplary embodiment, a vision inspection engine assigns a seal score value to the image data of the seal area.

[0100] In an exemplary embodiment, a vision inspection engine compares the seal score value of the image data of the seal area to a threshold value.

[0101] In an exemplary embodiment, a visual inspection engine assigns a confidence level to the seal score value.

[0102] This brief description of the invention is intended only to provide a brief overview of the subject matter disclosed herein according to one or more exemplary embodiments, and does not serve as a guide for interpreting the claims or for defining or limiting the scope of the invention, which is defined solely by the appended claims. This brief description is provided to introduce in a simplified form an exemplary selection of concepts that are further described in the detailed description below. This brief description is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all of the shortcomings described in the background.

[0103] So that the features of the present invention can be understood, a detailed description of the present invention can be obtained by reference to several embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the drawings illustrate only certain embodiments of the present invention and therefore should not be considered as limiting the scope of the present invention, since it encompasses other equally effective embodiments. The drawings are not necessarily to scale, and emphasis is generally placed on illustrating the features of certain embodiments of the present invention. In the drawings, like numerals are used to denote like parts throughout the various views. For a further understanding of the present invention, therefore, reference can be made to the following detailed description read in conjunction with the drawings. [Brief description of the drawings]

[0104] [Figure 1] FIG. 2 is a schematic diagram of a web manufacturing process for extruding a circular web to create a circular tape, according to some embodiments herein. [Diagram 2] FIG. 2 is a schematic diagram of a further web manufacturing process for converting the annular tape produced in FIG. 1 into an annular film tubing, according to some embodiments herein. [Diagram 3] FIG. 1 is a block diagram of an in-line seal defect detection system including a thermal vision system according to one embodiment. [Figure 4] FIG. 1 is a block diagram of an in-line seal defect detection system including an embodiment of a photoelastic vision system. [Diagram 5] FIG. 1 is a block diagram of an in-line seal defect detection system including an ultraviolet vision system according to one embodiment. [Figure 6] FIG. 1 is a block diagram of an in-line seal defect detection system including a thermal vision system, a photoelastic vision system, and an ultraviolet vision system according to one embodiment. [Figure 7] FIG. 1 is a block diagram of a seal defect detection system including a thermal vision system, a photoelastic vision system, and an ultraviolet vision system according to one embodiment. [Figure 8] FIG. 1 is a schematic diagram of a thermographic vision system of one embodiment. [Figure 9] FIG. 1 is a schematic diagram of a photoelastic vision system according to one embodiment. [Figure 10] 1 is a schematic diagram of the operating principle of a photoelastic vision system according to one embodiment; [Figure 11] FIG. 1 is a schematic diagram of an ultraviolet vision system according to one embodiment. [Figure 12] FIG. 1 is a block diagram of a vision system according to one embodiment. [Figure 13] FIG. 1 is a block diagram of a system for assessing seal integrity according to one embodiment. [Figure 14] FIG. 1 is a block diagram of a system for assessing seal integrity according to one embodiment. [Figure 15] 1 is a flow sheet of an algorithm according to one embodiment. [Figure 16] 1 is a flowsheet of a method for developing a trained image classification model according to one embodiment. [Figure 17] 1 is a flowsheet of a method for developing a trained image classification model based on several parameters, according to one embodiment. [Figure 18] 1 is a flowsheet of a method for developing a trained image classification model based on several parameters, according to one embodiment. [Figure 19] 1 is a flow sheet of a method for an image classification system for both training a model to classify seals and applying artificial intelligence algorithms to classify the condition of the seal, according to one embodiment. [Figure 20] FIG. 1 is a schematic diagram of a neural network, which is a multi-layer neural network, according to one embodiment. [Figure 21] 1 is a flow diagram of a method for classifying the condition of a seal according to one embodiment. [Figure 22] 1 is a thermographic image showing an acceptable seal. [Figure 23] FIG. 1 is a pictorial representation of a thermographic image showing a faulty seal. [Figure 24] FIG. 1 is a pictorial representation of a thermographic image showing a faulty seal. [Diagram 25] FIG. 13 is a pictorial representation of a thermographic image showing an acceptable seal. [Figure 26] FIG. 13 is a pictorial representation of a thermographic image showing an acceptable seal. [Figure 27] FIG. 13 is a pictorial representation of a thermographic image showing a poorly sealed area. [Figure 28] FIG. 13 is a pictorial representation of a thermographic image showing a poorly sealed area. [Figure 29] FIG. 13 is a pictorial representation of a photoelastic image showing an acceptable seal. [Diagram 30] FIG. 13 is a photoelastic image showing a faulty seal. [Diagram 31]FIG. 13 is a photoelastic image showing a faulty seal. [Diagram 32] FIG. 13 is an image of an ultraviolet image showing a faulty seal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0105] Exemplary embodiments will now be described more fully below with reference to the accompanying drawings, in which some, but not all, embodiments are shown. Indeed, the present disclosure may take many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. The terms "data," "content," "information," and similar terms may be used interchangeably to refer to data that may be transmitted, received, operated on, and / or stored, according to some exemplary embodiments. Furthermore, the term "exemplary" as may be used herein is not provided to convey a qualitative assessment, but merely to convey a description of an example. Thus, use of such terms should not be taken to limit the spirit and scope of the embodiments of the present invention.

[0106] 1-2, an exemplary embodiment of forming a packaging article is shown. In some embodiments, the packaging article can be fabricated using any known method, so long as the indicator described herein is incorporated into the packaging article. In some embodiments, solid polymer beads (not shown) are fed into a plurality of extruders 28 (for simplicity, only one extruder is shown). Inside the extruders 28, the polymer beads are fed, melted, and degassed, and then the resulting bubble-free melt is fed into a die head 30 and extruded through an annular die to obtain an annular tape 32.

[0107] After cooling and quenching by water spray from cooling ring 34, the annular tape 32 is folded into a lay-flat configuration by nip rollers 36. The annular tape 32 in its lay-flat configuration then passes through an irradiation vault 38 surrounded by shielding 40 where the annular tape 32 is irradiated with high energy electrons (i.e., ionizing radiation) from an iron-core transformer accelerator 42. The annular tape 32 is guided through the irradiation vault 38 on rolls 44.

[0108] After irradiation, the irradiated annular tape 46 is directed through the pre-coating nip rollers 48, after which the irradiated annular tape 46 expands slightly, resulting in trapped air bubbles 50. At the trapped air bubbles 50, the irradiated annular tape 46 is not significantly stretched in the longitudinal direction because the surface speed of the post-coating nip rollers 52 is approximately the same as the surface speed of the pre-coating nip rollers 48. Moreover, the irradiated tape 46 expands sufficiently to place the annular tape in a substantially circular configuration without significant lateral orientation, i.e., without lateral stretching.

[0109] The irradiated annular tape 46, slightly expanded by the gas bubbles 50, passes through a vacuum chamber 54 and then through a coating die 56. An annular coating stream 58 is melt extruded from the coating die 56 and coated onto the expanded irradiated annular tape 46 to form a coated annular tape 60. The coating stream 58 included an O2 barrier layer made from PVDC along with additional layers, not all of which are transparent to ionizing radiation. Further details of the above-mentioned coating steps are generally as described in U.S. Patent No. 4,278,738 to BRAX et al., which is incorporated herein by reference in its entirety. After irradiation and coating, the coated annular tape 60 is wound onto a take-up roll 62.

[0110] 2, in the second step of the process of producing the desired heat shrinkable film tubing, take-up roll 62 is installed as unwind roll 64. After the coated annular tape 60 is unwound from unwind roll 64 and passed over guide roll 66, the coated annular tape 60 is passed through a hot water bath 68 containing hot water 70. The coated tubular film 60, still in a lay-flat configuration, is immersed in the hot water 70 long enough to bring the annular tape 60 to its softening point, i.e., the desired temperature for biaxial orientation, while the coated annular tape is in a solid state.

[0111] The coated annular tape 60 is then guided through nip rolls 72, which blow in an air bubble 74 to solid-state stretch the coated annular tape 60 in the transverse direction. Also, while being blown, i.e., stretched in the transverse direction, the nip rolls 76 pull the annular tape 60 in the longitudinal direction because the nip rolls 76 have a surface speed that is faster than the surface speed of the nip rollers 72. As a result of the transverse stretching and longitudinal tension, the annular tape 60 is biaxially oriented in the solid state to form a biaxially oriented heat-shrinkable film tubing 78. The heat-shrinkable film tubing 78 is transversely stretched and longitudinally stretched. While the air bubble 74 is maintained between the nip roller pairs 72 and 76, the resulting blown film tubing 78 is folded by roller 80 into a lay-flat configuration. The film tubing 78 in the lay-flat configuration is then conveyed through the nip rollers 76 across a guide roll 82 and then rolled onto a take-up roll 84. Idler rolls 86 ensure good take-up.

[0112] As used herein, the term "film" includes plastic webs, whether they are films, bags, or sheets. The multilayer films disclosed herein can have a total thickness of up to 0.15 mm. In one embodiment, the multilayer film comprises: A) a first layer, which is a first outer film layer and functions as a heat seal layer; B) a second layer, which is a second outer layer and is configured to function as an abuse layer; C) a third film layer between the first layer and the second layer, which functions as a functional layer; D) a fourth film layer between the first layer and the third layer, which functions as a first tie layer; and E) a fifth film layer between the second layer and the third layer, which functions as a second tie layer. In some embodiments, the film can include the first layer, second layer, and third layer described above without the first and second tie layers. The functional layers include, but are not limited to, the group consisting of oxygen barrier layers, functional barrier layers, moisture barrier layers, harmful chemical barrier layers, microbial barrier layers, acidic layers, acid salt layers, bacteriocin layers, bacteriophage layers, metal layers, metal salt layers, natural oil layers, natural extract layers, layers comprising polyhexamethylene biguanide hydrochloride, layers comprising parabens, layers comprising grafted silane-quaternary amines, layers comprising triclosan, layers comprising silver, copper and / or zinc zeolites.

[0113] In one embodiment, the film is adhered to a packaging support, such as a tray or another film, to form a packaged article. In one embodiment, the packaged article is a member selected from the group consisting of an end-sealed bag, a side-sealed bag, a pouch, or a back-seamed packaged article. In some embodiments, the packaged article can be a film container, such as a bag (e.g., an end-sealed bag, a side-sealed bag, etc.) having a film surrounding the product. The film can include a substantially uniformly distributed fluorescent indicator therein such that the packaged product can indicate gas within the packaged article on all sides of the product. In some embodiments, the packaged article can include a rigid or semi-rigid shell with a film covering or enclosing seal over the shell. In one embodiment, the product placed within the packaged article is a food product. In one embodiment, the food product includes at least one member selected from the group consisting of meat and cheese.

[0114] Referring now to FIG. 3, there is shown a block diagram of an exemplary process flow for detecting defects in a sealed area of ​​a packaging article. The film or packaging article is sealed by a sealing mechanism 31, which creates a sealed area by sealing the film to itself or to another packaging support. In some embodiments, the packaging article may include a film, pouch, bag, tubing, rigid or semi-rigid shell or tray, with a film covering or enclosing a seal. In some embodiments, the packaging article may include a skin pack. The sealing mechanism 31 creates a sealed area in the film. The sealing mechanism 31 is an exothermic sealing mechanism, such as a seal bar, impulse seal, ultrasonic seal, high speed heat seal (100 seals per minute or more), heat from a laser seal, etc. The sealing mechanism seals the film to itself. In the case of film tubing as described above, the sealing mechanism seals one end of the tubing to create a packaging article with one opening. It is understood that the sealing mechanism can also seal the sides of the packaging article and can also seal the final opening after the product is placed in the packaging article.

[0115] After sealing of the packaged article, the thermographic vision system 33 captures a thermographic image of the sealed area. Residual heat from the sealing process performed by the sealing mechanism 31 is retained within the sealed area. This allows for the capture of a thermographic image of the sealed area. The thermal image is captured between 2 and 40 seconds after sealing. In one embodiment, the thermal image is captured between 5 and 30 seconds after sealing. In one embodiment, the thermal image is captured between 10 and 20 seconds after sealing. In one embodiment, the thermal image is captured approximately 15 seconds after sealing. In one embodiment, the thermal image is captured approximately 20 seconds after sealing. It is understood that active cooling systems such as blown air, chilled air, cooling chambers, etc. can be used to reduce the image capture time frame.

[0116] Thermography is a non-contact, non-destructive testing method in which a surface temperature distribution (thermogram) is recorded based on the amount of infrared radiation emitted by the inspected scene. The inspected object is thermally excited and the response to this excitation is recorded in time. Active thermography data are three-dimensional, since the temperature of each spatial pixel is evaluated in time. Active thermography is used to detect (sub)surface defects based on differences in thermo-physical properties.

[0117] The thermographic vision system 33 includes at least one thermographic image capture device and a visual inspection engine. In some embodiments, the thermographic vision system 33 includes a light source, several light sources (of the same or different types), filters, and multiple image capture devices or reflectors. In one embodiment, the thermographic image capture device is a thermal camera. The light source may be a continuous light source or a flashing light source. The light source may be white light, filtered light, diffuse light, infrared light, ultraviolet light, within the visible spectrum or outside the visible spectrum. In some embodiments, the thermographic vision system 33 is housed in an enclosure to provide an isolated environment for the image capture device to capture the image. The enclosure can isolate or reduce the impact on the imaging area from external sources of light and contained objects such as dust.

[0118] In one embodiment, the thermographic image capture device captures an image of at least one seal of each packaged article. In another embodiment, the thermographic image capture device captures at least two images of at least one seal of each packaged article. In another embodiment, the thermographic image capture device captures at least four images of at least one seal of each packaged article. In one embodiment, the thermographic image capture device captures image data at a frame rate of at least 20 frames per second. In one embodiment, the thermographic image capture device captures image data at a frame rate of at least 40 frames per second. In one embodiment, the thermographic image capture device captures image data at a frame rate of at least 60 frames per second. Having additional image data of the seals can provide additional data and result in a more reliable determination of seal quality. In one embodiment, the thermographic image capture device transmits image data in a video format, including but not limited to Motion JPEG, MPEG, SEQ and H.264 formats. In one embodiment, the thermographic image capture device captures thermal image data. In another embodiment, the thermographic image capture device captures multi-spectral dynamic image data.

[0119] The image data acquired by the thermal imaging device is sent to a vision inspection engine. As described in more detail herein, the vision inspection engine can analyze the image and determine whether the seal is acceptable or defective. The vision inspection engine creates a seal score related to the quality of the seal area. The vision inspection engine analyzes at least one of color gradient change, light intensity, color intensity, change in data from adjacent pixels, change in data from adjacent pixel sections, and combinations thereof to create the seal score. In one embodiment, the vision inspection engine further analyzes at least one of color gradient change, light intensity, color intensity, change in data from adjacent pixels, change in data from adjacent pixel sections, and combinations thereof to create a confidence score. The confidence score provides an assessment of the confidence of the assigned seal score. The vision inspection engine may be local to the thermographic vision system 33, integrated into another part of the system, or remote.

[0120] Still referring to FIG. 3 , the defect mechanism 35 receives instructions to identify packaging articles that have a defective seal or have a confidence level below a certain threshold. In one embodiment, the defect mechanism pulls the defective packaging articles from the line for offline testing, verification, recycling, repackaging or disposal, thereby allowing only packaging articles with proper seals to continue past the defect mechanism 35. In one embodiment, the defect mechanism tags, flags or prints at least one indicia on the packaging article to identify the packaging article as possibly having a defective seal. In one embodiment, detection of a defective seal can result in the defect mechanism activating an alarm, flagging the defective packaging article, displaying an image of the defective seal or packaging article, displaying data regarding the defective seal, and generating a report of the defective seal or packaging article. In one embodiment, the defect mechanism removes the packaging article with the defective seal. The defect mechanism may pause the production line while the packaging article is unloaded, or may remove the packaging article without pausing the production line. In one embodiment, the defect mechanism includes a vacuum suction cup utilized in combination with a robotic arm to automatically remove the packaging article. In another embodiment, the defect mechanism moves packages to a defect line away from line packaged articles that do not have defective seals.

[0121] In one embodiment, signals responsive to defective seals are generated to activate alarms, flagging, discontinuity image displays, discontinuity data, discontinuity data reports, etc. while the web is moving, i.e., instantaneously and online. Alternatively, signals responsive to defective seals are generated after production is completed, i.e., offline. Signals responsive to discontinuities can include electronic messaging, emails, data logs, and reports.

[0122] It is understood that the vision system and defect mechanism may be implemented after the formation of the packaged article and before the product is packaged. In another embodiment, the vision system and defect mechanism may be implemented after packaging and sealing of the product within the packaged article.

[0123] Referring now to FIG. 4, there is shown a block diagram of an exemplary process flow for detecting defects in a seal area of ​​a packaging article. The film or packaging article is sealed by a sealing mechanism 31 as described above. The sealing mechanism 31 forms a seal area in the film. The sealing mechanism 31 may be a seal bar heat seal, an impulse seal, an ultrasonic seal, a high speed heat seal or a laser seal. In one embodiment, the sealing mechanism creates at least 100 seals per minute. In another embodiment, the sealing mechanism creates at least 300 seals per minute. The sealing mechanism seals the film to itself. In the case of film tubing as described above, the sealing mechanism seals one end of the tubing to create a packaging article with one opening. It is understood that the sealing mechanism can also seal the sides of the packaging article and can also seal the final opening after the product is placed in the packaging article.

[0124] The film is placed between two cross-polarizing filters set orthogonally to each other. The first filter passes light oriented in a first direction. The second filter is oriented orthogonal to the first filter to block all light coming through the first filter. When a piece of transparent material is placed between the two filters, the light is rotated. The amount of rotation varies depending on the type of material and the amount of internal strain within the material. When the light is rotated out of the polarization plane of the second filter, some of the light passes through the second filter.

[0125] After sealing the packaged article, the photoelastic vision system 43 captures an image of the sealed area. The photoelastic vision system 43 includes at least one photoelastic image capture device, a light source, a first polarizing filter, a second polarizing filter, and a vision inspection engine. In some embodiments, the photoelastic vision system 43 includes several (same or different types) light sources, filters, multiple image capture devices or reflectors. In one embodiment, the photoelastic image capture device is a high-speed camera. The light source may be a continuous light source or a flash light source. The light source may be white light, filtered light, diffuse light, infrared light, ultraviolet light, within or outside the visible spectrum. In some embodiments, the photoelastic vision system 43 is stored in an enclosure to provide an isolated environment for the image capture device to capture the image. The enclosure can isolate or reduce the impact on the imaging area from external light sources and contained objects such as dust.

[0126] In one embodiment, the photoelastic image capture device captures an image of at least one seal of each packaged article. In another embodiment, the photoelastic image capture device captures at least two images of at least one seal of each packaged article. In another embodiment, the photoelastic image capture device captures at least four images of at least one seal of each packaged article. In one embodiment, the photoelastic image capture device captures image data at a frame rate of at least 20 frames per second. In one embodiment, the photoelastic image capture device captures image data at a frame rate of at least 40 frames per second. In one embodiment, the photoelastic image capture device captures image data at a frame rate of at least 60 frames per second. Having additional image data of the seals can provide additional data and provide a more reliable determination of seal quality. In one embodiment, the ultraviolet image capture device is an RGB camera capable of capturing images in the visible spectrum.

[0127] The image data acquired by the photoelastic imaging device is sent to a vision inspection engine. As described in more detail herein, the vision inspection engine can analyze the image and determine whether the seal is acceptable or defective. In one embodiment, the vision inspection engine determines a confidence level along with the seal classification. The vision inspection engine may be local to the photoelastic vision system 43, integrated into another part of the system, or remote. The vision inspection engine analyzes the image for variations in color gradients of adjacent pixels, comparison of sections, etc. The vision inspection engine can compare one or more of the average, variability, skew, minimum, maximum, range, or variation of the seal area, or number of pixels or sections of the image. Variations between pixels or sections of the image can indicate a seal defect.

[0128] 4, the defect mechanism 35 receives instructions to identify packaging articles that have a defective seal or have a confidence level below a certain threshold. In one embodiment, the defect mechanism pulls the defective packaging articles off the line for offline testing, verification, recycling, repackaging or disposal, thereby allowing only packaging articles with proper seals to continue past the defect mechanism 35. In one embodiment, the defect mechanism tags, flags or prints at least one indicia on the packaging article to identify the packaging article as possibly having a defective seal. In one embodiment, detection of a defective seal can result in the defect mechanism activating an alarm, flagging the defective packaging article, displaying an image of the defective seal or packaging article, displaying data regarding the defective seal, and generating a report of the defective seal or packaging article.

[0129] In one embodiment, signals responsive to defective seals are generated to activate alarms, flagging, discontinuity image displays, discontinuity data, discontinuity data reports, etc. while the web is moving, i.e., instantaneously and online. Alternatively, signals responsive to defective seals are generated after production is completed, i.e., offline. Signals responsive to discontinuities can include electronic messaging, emails, data logs, and reports.

[0130] It is understood that the vision system and defect mechanism may be implemented after the formation of the packaged article and before the product is packaged. In another embodiment, the vision system and defect mechanism may be implemented after packaging and sealing of the product within the packaged article.

[0131] Referring now to FIG. 5, there is shown a block diagram of an exemplary process flow for detecting defects in a seal area of ​​a packaging article. A film or packaging article is sealed by a sealing mechanism 31. In some embodiments, the packaging article may include a film, pouch, bag, tubing, rigid or semi-rigid shell, with a film covering or enclosing a seal. In some embodiments, the film may include a detectable moiety that allows for seal defect detection. In some embodiments, the packaging article may include a skin pack. In some embodiments, a film with an ultraviolet ("UV") fluorescence based indicator may only partially surround the product.

[0132] The film includes a detectable moiety such as a fluorescent-based indicator. In one embodiment, the film includes a layer including a blend of a polymer and a fluorescent-based indicator in an optional functional layer, where the concentration of the fluorescent-based indicator is present in the blend in an amount of about 5 to 50 ppm based on the total layer weight, and the fluorescent-based indicator is uniformly blended with the polymer. The uniformity of the indicator in the indicator / polymer blend refers to the concentration of the indicator in the blend when 10 random samples are taken, each sample having a size of 10 grams, and subject to variation of less than 20% and more than 20% based on the target concentration of the indicator in the blend. In one embodiment, the polymer includes PVDC, i.e., polyvinylidene chloride, ethylene vinyl alcohol, or nylon.

[0133] The UV fluorescent indicator can be present in the functional layer at any level detectable by a detector while allowing the functional layer to maintain its intended function. Too much fluorescent indicator can interfere with the layer function. Too little fluorescent indicator can go undetectable by the detector. In one embodiment, the fluorescent indicator can be present at a level of at least 0.5 parts per million (ppm). As used herein, the phrase "parts per million" and the equivalent expression "ppm" refer to the weight of the fluorescent indicator relative to the total weight of the layer (weight fluorescent indicator + weight of the remaining components in the layer). Of course, the majority of the components of the layer are one or more thermoplastic polymers that are solid at room temperature. Both the fluorescent indicator and the thermoplastic polymer of the layer can be solid at room temperature. In one embodiment, the fluorescent indicator may be present at a level of at least 1 ppm, or at least 1.5 ppm, or at least 2 ppm, or at least 3 ppm, or at least 5 ppm, or at least 10 ppm, or at least 20 ppm, or at least 40 ppm, or at least 80 ppm, or at least 120 ppm, or at least 160 ppm, or at least 200 ppm, or at least 300 ppm, or at least 500 ppm. In one embodiment, the fluorescent indicator may be present in the layer at a level of 0.5 to 40 ppm, or 1 to 20 ppm, or 1.5 to 10 ppm, or 2 to 5 ppm. For the film to be suitable for food contact end uses, the fluorescent indicator is present in the layer in an amount of 150 ppm or less.

[0134] UV-based fluorescent indicators are UV-absorbing compounds that have unique absorption and / or fluorescence properties. Preferred UV-absorbing fluorescent indicator components have unique optical signatures that are not naturally occurring and are not easily confused with signals from natural sources. Preferred UV fluorescent indicators have multiple unique absorption or fluorescence features in the UV spectrum. For example, as used herein, electromagnetic radiation of 375 nanometers is used as the incident radiation to excite the fluorescent indicator known as 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole), which has been assigned the CAS Registry Number 7128-64-5, which is also known as: 2,2'-(2,5-thiophenediyl)bis[5-tert-butylbenzoxazole]; 2,5-bis-2(5-tert-butyl-benzoxalyl)thiophene; 2,5-bis(5-t-butyl-2-benzoxazolyl)thiophene; 2,5-bis-(5-t-butylbenzoxazolyl-[2-yl])-thiophene; 2,5-bis-(5-tert-butyl-2-benzoxazol-2-yl)thiophene; 2,5-bis(5'-tert-butylbenzoxazolyl-[2-yl])-thiophene; Also known as t-butyl-2-benzoxazol-2-yl)thiophene; 2,5-bis(5'-tert-butyl-2'-benzoxazolyl)thiophene; 2,5-bis(5-tert-butyl-2-benzoxazolyl)thiophene; 2,5-bis(5-tert-butyl-benzoxazol-2-yl)thiophene; 2,5-bis(5-tert-butylbenzoxazolyl)-2-thiophene; 2,5-di(5-tert-butylbenzoxazol-2-yl)thiophene; 2,2'-(2,5-thiophenediyl)bis[5-(1,1-dimethylethyl)-benzoxazole; 2,5-bis(5'-tert-butyl-2-benzoxazolyl)thiophene; and 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole). Absorption of incident radiation at 375 nanometers caused the detectable component of the excited 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) optical brightener to emit radiation at 435 nanometers.A fluorescent indicator was homogeneously blended into the PVDC resin and used to create the oxygen barrier layer of the multilayer film. Exposing the resulting annular tape and / or heat shrinkable film tubing to incident radiation at 375 nm excited the 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) optical brightener to emit radiation at 435 nanometers. The emitted 435 nm radiation was detected by a machine vision system, which revealed the presence, continuity, and thickness of the PVDC barrier layer of the tape and multilayer film tubing. In one embodiment, the UV-based fluorescent indicator comprises at least one member selected from the group consisting of ultraviolet indicators, infrared indicators, dyes, pigments, optical brighteners, optical brighteners, 2,2'-(2,5-thiophenylenediyl)bis(5-tert-butylbenzoxazole), hydroxyl-4-)p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes. The indicator is of a type that upon exposure to radiation at a first peak wavelength, emits radiation at a second peak wavelength.

[0135] In one embodiment, the detectable moiety comprises at least one member selected from the group consisting of ultraviolet indicators, infrared indicators, dyes, pigments, optical brighteners, optical brighteners, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole), 2,2'-(2,5-thiophenylenediyl)bis(5-tert-butylbenzoxazole), hydroxyl-4-)p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes. 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) is commercially available as an optical brightener from several suppliers, including BASF Corporation (TINOPAL OP® 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) optical brightener) and Mayzo, Inc. (BENETEX OB PLUS® 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) optical brightener). The indicator is of a type that, when exposed to radiation at a first peak wavelength, emits radiation at a second peak wavelength.

[0136] In one embodiment, the UV-based fluorescent indicator is present in at least one layer of the film, and the indicator is present at a level of from 0.5 to 150 ppm by layer weight. In another embodiment, the indicator is present in at least one layer of the film, and the indicator is present at a level of from 1 to 20 ppm, or from 2 to 10 ppm by layer weight. In one embodiment, the detectable moiety is present in the film layer at a level of at least 1 part per million.

[0137] In one embodiment, the UV-based fluorescent indicator is of a type that, when exposed to radiation at a first peak wavelength, emits radiation at a second peak wavelength.

[0138] Still referring to FIG. 5, the sealing mechanism 31 creates a seal area in the film. The sealing mechanism 31 seals the film to itself or to another packaging article. In one embodiment, the sealing mechanism is a heat sealing mechanism, such as heat from a seal bar, impulse seal, ultrasonic seal or laser seal. The sealing mechanism seals the film to itself. In the case of film tubing as described above, the sealing mechanism seals one end of the tubing to create a packaging article with one opening. It is understood that the sealing mechanism can also seal the sides of the packaging article and can also seal the final opening after the product is placed in the packaging article.

[0139] After sealing of the packaged article, the ultraviolet system 53 captures an ultraviolet image of the sealed area. The ultraviolet vision system 53 includes at least one ultraviolet image capture device, an ultraviolet light source, and a vision inspection engine. In one embodiment, the ultraviolet image capture device is an RGB camera capable of capturing images in the visible spectrum. In some embodiments, the ultraviolet vision system 53 includes several (same or different types) light sources, filters, multiple image capture devices or reflectors. The light source may be a continuous light source or a flashing light source. The additional light sources may be white light, filtered light, diffuse light, infrared light, ultraviolet light, within or outside the visible spectrum. In some embodiments, the ultraviolet vision system 53 is housed in an enclosure to provide an isolated environment for the image capture device to capture the image. The enclosure can isolate or reduce the impact on the imaging area from external sources of light and contained items such as dust. The light source of the ultraviolet vision system 53 excites detectable components in the sealed area of ​​the packaged article. The detectable components enable the image capture device to identify the detectable components.

[0140] In one embodiment, the detectable component is a UV fluorescent indicator and at least one of: (i) the fluorescent electromagnetic energy intensity is higher in the area where the seal defect is located compared to a proper seal, or (ii) a fluorescent electromagnetic energy color shift (downward in energy, i.e., longer wavelength, lower frequency) occurs in the area where the seal defect is located compared to a proper seal.

[0141] In one embodiment, the fluorescent electromagnetic energy intensity is at least 10% higher in the area where the seal defect is located compared to the intensity of the fluorescent electromagnetic energy in the area where the seal is proper. In another embodiment, the fluorescent electromagnetic energy intensity is 15% or more higher, or 20% or more higher, or 30% or more higher, or 40% or more higher, or 50% or more higher, or 60% or more higher in the area where the seal defect is located relative to the intensity of the fluorescent electromagnetic energy in the area where the seal is proper. Factors that can affect the fluorescent image emitted by the package include, among others: (i) the angle of view, (ii) the focal length between the imaging device and the area of ​​the package under consideration, (iii) the exposure time, (iv) the amount of excitation, and (v) variations in the thickness of the film, including the increase in thickness upon heat sealing of the film to itself or to another film.

[0142] In one embodiment, the ultraviolet image capture device captures an image of the at least one seal of each packaged item. In another embodiment, the ultraviolet image capture device captures at least two images of the at least one seal of each packaged item. In another embodiment, the ultraviolet image capture device captures at least four images of the at least one seal of each packaged item. In one embodiment, the ultraviolet image capture device captures image data at a frame rate of at least 20 frames per second. In one embodiment, the ultraviolet image capture device captures image data at a frame rate of at least 40 frames per second. In one embodiment, the ultraviolet image capture device captures image data at a frame rate of at least 60 frames per second. Having additional image data of the seals can provide additional data and result in a more reliable determination of seal quality.

[0143] The image data acquired by the ultraviolet imaging device is sent to a vision inspection engine. As described in more detail herein, the vision inspection engine can analyze the image and determine whether the seal is acceptable or defective. In one embodiment, the acquisition of the image data is performed using an ultraviolet imaging device that generates image data of the fluorescent electromagnetic energy emitted by the excited indicator, and the evaluation of the image data is performed using a vision inspection engine that is programmed with an algorithm that can evaluate the intensity of the fluorescent electromagnetic energy emitted by the excited indicator. In one embodiment, the vision inspection engine determines a confidence level along with the seal classification. The vision inspection engine may be local to the ultraviolet vision system 33, integrated into another part of the system, or remote.

[0144] 5, the defect mechanism 35 receives instructions to identify packaging articles that have a defective seal or have a confidence level below a certain threshold. In one embodiment, the defect mechanism pulls the defective packaging articles off the line for offline testing, verification, recycling, repackaging or disposal, thereby allowing only packaging articles with proper seals to continue past the defect mechanism 35. In one embodiment, the defect mechanism tags, flags or prints at least one indicia on the packaging article to identify the packaging article as possibly having a defective seal. In one embodiment, detection of a defective seal can result in the defect mechanism activating an alarm, flagging the defective packaging article, displaying an image of the defective seal or packaging article, displaying data regarding the defective seal, and generating a report of the defective seal or packaging article.

[0145] In one embodiment, signals responsive to defective seals are generated to activate alarms, flagging, discontinuity image displays, discontinuity data, discontinuity data reports, etc. while the web is moving, i.e., instantaneously and online. Alternatively, signals responsive to defective seals are generated after production is completed, i.e., offline. Signals responsive to discontinuities can include electronic messaging, emails, data logs, and reports.

[0146] It is understood that the vision system and defect mechanism may be implemented after the formation of the packaged article and before the product is packaged. In another embodiment, the vision system and defect mechanism may be implemented after packaging and sealing of the product within the packaged article.

[0147] In some embodiments, multiple vision systems are contemplated. The vision systems may be redundant as separate, similar types of vision systems utilizing light sources, image capture devices, illumination angles, illumination characteristics, etc. Referring now to FIG. 6, there is shown a block diagram of an exemplary process flow for detecting defects in a seal area of ​​a packaged article. The process is similar to the embodiment described above with reference to FIGS. 3-5, with additional vision systems included. The thermographic vision system 33, the photoelastic vision system 43, and the ultraviolet vision system 53 each send image data to a vision inspection engine. It is understood that a common shared vision inspection engine or multiple vision inspection engines are contemplated. The vision inspection engine is programmed with one or more algorithms capable of evaluating the thermographic image, the photoelastic image, and the intensity of the fluorescent electromagnetic energy emitted by the excited indicator. The combined image data from the three vision systems results in a higher confidence level that is sent to the defect mechanism 35. Although the thermographic vision system 33, the photoelastic vision system 43, and the ultraviolet vision system 53 are shown, a system utilizing only two vision systems is contemplated.

[0148] Referring now to FIG. 7, a block diagram of an exemplary process flow for detecting defects in a seal area of ​​a packaging article after the product has been sealed therein. A product loading mechanism loads the product into the packaging article. In one embodiment, the product is a food product. In another embodiment, the product is meat or cheese. After the product is loaded, a sealing mechanism 31 seals the product into the packaging article. In some embodiments, the sealing mechanism includes a vacuum chamber for vacuum sealing the product within the packaging article. In one embodiment, the packaging article is shrunk around the product. In one embodiment, the packaging article is a non-shrunk packaging article. The seal or seals of the packaging article are then analyzed by a thermographic vision system 33, a photoelastic vision system 43, and an ultraviolet vision system 53, as well as a vision inspection engine, as described more fully herein. The combined image data from the three vision systems results in a higher confidence level that is sent to a defect mechanism 35. Although a thermographic vision system 33, a photoelastic vision system 43, and an ultraviolet vision system 53 are shown, systems utilizing only two vision systems are contemplated.

[0149] 8, a thermographic vision system 81 is shown according to one embodiment. The film 100 is conveyed to a sealing mechanism 85 for sealing the film 100 to itself or another article. In one embodiment, the sealing mechanism makes at least 10 seals per minute. In one embodiment, the sealing mechanism makes at least 50 seals per minute. In one embodiment, the sealing mechanism makes at least 100 seals per minute. In one embodiment, the sealing mechanism makes at least 250 seals per minute. In one embodiment, the sealing mechanism 85 is a heat sealing mechanism such as a seal bar sealer, impulse seal, ultrasonic seal, or laser seal.

[0150] After sealing, the film 100 continues past the sealing mechanism 85 where the thermographic image capture device 87 captures at least one image of the seal created by the sealing mechanism 85. The thermographic image capture device is an infrared scanner or camera. In one embodiment, the infrared camera can capture at least two images of each seal as the film is moving. In one embodiment, the infrared camera can capture at least four images of each seal as the film is moving. In one embodiment, the infrared camera can capture images of each seal at a rate of at least 30 frames per second. In one embodiment, the infrared camera can capture images of each seal at a rate of at least 60 frames per second. In one embodiment, the thermographic image capture device 87 is positioned to capture images of the seal between 2-40 seconds after the seal is created by the sealing mechanism 85 on the film 100. In one embodiment, the thermographic image capture device 87 is positioned to capture images of the seal between 5-30 seconds after the seal is created by the sealing mechanism 85 on the film 100. In one embodiment, the thermographic image capture device 87 is positioned to capture an image of the seal 10-20 seconds after the seal is created by the sealing mechanism 85 on the film 100. In one embodiment, the thermographic image capture device 87 is positioned to capture an image of the seal approximately 2 seconds after the seal is created by the sealing mechanism 85 on the film 100. In one embodiment, the thermographic image capture device 87 is positioned to capture an image of the seal approximately 5 seconds after the seal is created by the sealing mechanism 85 on the film 100. In one embodiment, the thermographic image capture device 87 is positioned to capture an image of the seal approximately 10 seconds after the seal is created by the sealing mechanism 85 on the film 100. In one embodiment, the thermographic image capture device 87 is positioned to capture an image of the seal approximately 20 seconds after the seal is created by the sealing mechanism 85 on the film 100.Residual heat from the sealing mechanism 85 allows the thermographic image capture device 87 to capture a thermal image of the seal along with variations in temperature along the seal area. Such variations may indicate a defective seal. The image data captured by the thermal image capture device 87 is sent to a vision inspection engine 101, as described in more detail herein.

[0151] Referring now to FIG. 9, a photoelastic vision system 91 is shown according to one embodiment. The film 100 is transported to the location of a photoelastic image capture device 96, which captures light that has passed through the seal area of ​​the film 100 and the linear polarizers 94, 95. In one embodiment, the photoelastic image capture device is an RGB camera capable of capturing images in the visible spectrum. Light from the light source 91 passes through a diffuser plate 93 to diffuse the light, and a first linear polarizer 94 has a polarization direction oriented in a first direction and polarizes the light from the light source 91. It is understood that instead of or in addition to the use of a diffuser plate, a diffuse light source can be utilized. The polarized light then passes through the film 100, which rotates the light. The amount of rotation varies based on variables such as the type of film, the seal strength and the seal integrity. Once a portion of the light has been rotated, the light then passes through a second linear polarizer 94, which has a polarization direction oriented orthogonal to the first direction of the first linear polarizer 95. In the absence of film 100, no light is transmitted from light source 91 to photoelastic image capture device 96. However, because film 100 rotates the light, photoelastic image capture device 96 captures an image of film 100.

[0152] In one embodiment, light source 91 is a white light source with a wavelength spectrum from 400 nm to 700 nm. In one embodiment, the light source is a background light below the film. In another embodiment, the light source is a polarized white light source which eliminates the need for the first linear polarizer.

[0153] Although two linear polarizers are described above, it is contemplated that at least one of the linear polarizers is replaced by utilizing a polarizing filter film on the light source or lens of the photoelastic image capture device. In another embodiment, the photoelastic image capture device includes a polarizing lens to climate the need for a second linear polarizer. In another embodiment, the light source is a polarized light source. In one embodiment, the photoelastic camera can capture at least two images of each seal as the film moves. In one embodiment, the photoelastic camera can capture at least four images of each seal as the film moves. In one embodiment, the photoelastic camera can capture images of each seal at a rate of at least 30 frames per second. In one embodiment, the photoelastic camera can capture images of each seal at a rate of at least 60 frames per second.

[0154] The image data captured by the photoelastic image capture device 97 is sent to a vision inspection engine 101, as described in more detail herein.

[0155] Referring now to FIG. 10, there is a schematic diagram of photoelastic operation. A first polarizing filter 1002 has a polarization direction that is orthogonal to that of a second polarizing filter 1004. Unpolarized light 1001 passes through the first polarizing filter 1002 (i.e., a linear polarizer) which causes the unpolarized light 1001 to become polarized light 1003. In the absence of the film 1005, the polarized light 1003 is filtered by the second polarizing filter 1004, and no or very little light passes through the second polarizing filter 1004. In contrast, when the polarized light 1003 passes through the film 1005, the polarized light 1003 is rotated causing a shifted polarized light 1006. This rotation allows some light to be transmitted through the second polarizing filter 1004. Variables that change the amount of rotation or filtering of the light include the thickness of the film, the stress in the film, and the strain in the film. Varying the amount of rotation allows for the capture of an image as described above.

[0156] Referring now to FIG. 11, an ultraviolet vision system 1100 according to one embodiment is shown. The film 100 is transported within a darkened enclosure 1102. An ultraviolet light source 1101 illuminates the film 100, causing detectable components in the film to excite and emit fluorescent electromagnetic energy. An ultraviolet image capture device 1106 captures at least one image of the seal area of ​​the film 100. In one embodiment, a blue pass filter is used to filter out wavelengths of light below 450 nm and about 525 nm. The blue pass filter improves use of higher than normal ambient lighting conditions. In one embodiment, either the light source or the blue pass filter transmits light between 425-495 nm.

[0157] In one embodiment, the ultraviolet camera can capture at least two images of each seal as the film moves. In one embodiment, the ultraviolet camera can capture at least four images of each seal as the film moves. In one embodiment, the ultraviolet camera can capture images of each seal at a rate of at least 30 frames per second. In one embodiment, the ultraviolet camera can capture images of each seal at a rate of at least 60 frames per second. The seal characteristics allow the ultraviolet image capture device 1106 to capture ultraviolet images of the seal along with variations in radiant energy along the seal area. Such variations may indicate a defective seal. The image data captured by the ultraviolet image capture device 101 is sent to the vision inspection engine 101 as described in more detail herein.

[0158] Exemplary System Architecture 12, in some embodiments, one or more computing systems, such as a computing device, can be used to control the imaging device 1204, the light source 1205, and the vision system 1201. In some embodiments, the one or more computing systems can control or direct other computing systems to control other functions within the packaging environment, such as in-line conveyors and material handling devices.

[0159] The vision inspection engine 101 (e.g., as shown in FIG. 8, FIG. 9, and FIG. 11) may be a computing device 1203 that may include a processor 1206, a memory 1207, input / output circuitry 1208, communication circuitry 1211, a vision inspection circuitry 1210, and an acquisition circuitry 1209, and may be configured to perform the functions described herein. In some embodiments, the processor 1206 (and / or a coprocessor or any other processing circuitry that assists the processor or is otherwise associated with the processor) may communicate with the memory 1207 via a bus for passing information between components of the device. In some embodiments, the computing device 1203 may be a distributed system of computing components and / or a remotely located computing device. In some embodiments, the computing device 1203 may be local or remote. The memory 1207 may be non-transitory, e.g., including one or more volatile and / or non-volatile memories. That is, for example, the memory may be an electronic storage device (e.g., a computer-readable storage medium). The memory may be configured to store information, data, content, applications, instructions, etc. to enable the device to perform various functions according to any exemplary embodiment of the present invention.

[0160] The processor 1206 may be embodied in a number of different ways, and may include, for example, one or more processing devices configured to perform independently. Additionally or alternatively, the processor may include one or more processors configured in tandem via a bus to enable independent execution, pipelining, and / or multithreading of instructions.

[0161] In an exemplary embodiment, the processor 1206 may be configured to execute instructions stored in memory 1207 or otherwise accessible to the processor. Additionally or alternatively, the processor may be configured to execute hard-coded functions. Thus, whether configured by hardware or software methods, or a combination thereof, a processor may represent an entity (e.g., physically embodied in a circuit) that is configured according to an embodiment of the present invention and capable of performing operations according to an embodiment of the present invention. Alternatively, as another example, when a processor is embodied as an execution unit of software instructions, the instructions may specifically configure the processor to perform the algorithms and / or operations described herein when the instructions are executed.

[0162] In some embodiments, the vision system 1201 may in turn include input / output circuitry 1208 that may communicate with the processor 1206 to provide output to a user and, in some embodiments, to receive indications of user input. The input / output circuitry 1208 may comprise a user interface and may include a display, may comprise a web user interface, a mobile application, a client device, a kiosk, and the like. In some embodiments, the input / output circuitry 1208 may also include a keyboard, a mouse, a joystick, a touch screen, a touch area, soft keys, a microphone, a speaker, or other input / output mechanisms. The processor and / or user interface circuitry comprising the processor may be configured to control one or more functions of one or more user interface elements through computer program instructions (e.g., software and / or firmware) stored in a memory accessible to the processor (e.g., memory 1207, etc.).

[0163] Meanwhile, the communication circuitry 1211 may be any means, such as a device or circuitry embodied in either hardware or a combination of hardware and software, configured to receive and / or transmit data to and from any other device or module in communication with the network and / or computing device 1203. In this regard, the communication circuitry may include, for example, one or more cables (e.g., USB cables) connecting the imaging device 1204 and the light source 1205 to the vision system 1201 for use with the software and hardware configurations described herein. In some embodiments, the communication circuitry 1211 may include, for example, an antenna (or multiple antennas) and supporting hardware and / or software for enabling communication with a wireless communication network or one or more wireless devices. Additionally or alternatively, the communication interface may include circuitry for interacting with the cable and / or antenna to cause transmission of signals via the cable and / or antenna or to handle reception of signals received via the cable and / or antenna. In some environments, the communication interface may additionally or alternatively support wired communication with a network (e.g., Ethernet). Thus, for example, the communications interface may include a communications modem and / or other hardware / software to support communications via cable, digital subscriber line (DSL), universal serial bus (USB), or other mechanism.

[0164] The acquisition circuitry 1212 can be used to buffer a series of images and data captured by the imaging device 1204 (e.g., a camera). In some embodiments, the imaging device 1204 can capture raw data incident on one or more surfaces (e.g., on one or more substrates of an image sensor) within the imaging device. The imaging device 1204 can convert the raw data into computer-readable image data via one or more circuits and can transmit the image data to the acquisition circuitry 1212. In some embodiments, the image data can include any sensor data corresponding to the wavelength and / or intensity of electromagnetic energy used to detect defects in the packaged product. The image data may include individual images; a sequence of images; a video; and the like. The acquisition circuitry 1212 can further control the imaging device 1204 and the light source 1205 to trigger and time the illumination and capture of the raw data of each of the packaged products according to any embodiment of the present invention. In some embodiments, image data may be captured through any of the means for generating image data disclosed herein, including, but not limited to, any of the imaging devices (e.g., cameras, sensors, etc.) disclosed herein, in both manual, autonomous, and partially manual and partially autonomous modes of operation.

[0165] The vision inspection circuitry 1210 can be used to facilitate processing and analysis of image data received from the acquisition circuitry 1212. The vision inspection circuitry 1210 can further trigger alerts or other downstream notifications or actions based on the results of the processing and analysis. The vision inspection circuitry can further connect to one or more remote servers for data mining and storage (e.g., to access models and data for training new models). In some embodiments, the vision system 1201 and the vision inspection circuitry 1210 may comprise a single computing device or multiple devices connected locally or interacting over a wired and / or wireless network.

[0166] In some embodiments, the vision system 1201 may include an operator user interface (e.g., as part of the input / output circuitry 1208). Defect data (e.g., from the vision inspection circuitry 1210) may be displayed on the interface and archived in a database either locally or remotely (e.g., via a local or network connection). Defect data and images may be displayed in real-time on the interface. In some embodiments, instantaneous, historical, and statistical data may also be viewed on the interface on demand. The computing device 1203 may be set up to selectively detect and accurately classify defects in packaged products, including detection of excited fluorescence-based indicators indicative of gas trapped within the packaged article.

[0167] Images of each defect may be classified, stored, displayed, and compared with previous and future images of other defects. The computing device 1203 may capture high resolution images of each defect in real time. Individual defect information, such as individual defect geometric information and group defect statistics, may be provided for instantaneous decision-making and action regarding process improvement and monitoring, such as defect alarming. In some embodiments, each potential defect may be presented to an operator for manual decision on how to handle the defective packaged product. In some embodiments, screening, flagging, and / or segregation of defective packaged products may be done partially or wholly automatically. A human operator may, in some cases, verify the work of an otherwise automated system.

[0168] Various outputs for marking / flag, alarming, and autonomy can be set for different defect severity levels. Data can be exported, for example, to MS Excel and / or SQL databases located anywhere on the network, and data mining software allows various reports to be easily generated automatically and / or on demand. Defect data can be processed on a processing unit such as a digital processing board. Flagging can be used in conjunction with separating and / or discarding packaged products with damaged film or film with damage above a predetermined threshold. Flagging can be performed by applying a label to the film at (or corresponding to) the location of the film defect for manual or automatic separation (e.g., with a robotic package separator). In some embodiments, defective packages (e.g., packaged products exhibiting leaks or low vacuum conditions) can be opened and repackaged with new packaged articles.

[0169] In one embodiment, the input / output circuitry 1208 can allow for external signal input, such as a new roll or new production run indication and an inspection pause indication. The output of alarms for user-defined defect alarm criteria is also handled through the input / output circuitry 1208 (e.g., a user interface). Outputs can also be initiated to control downstream flagging or marking devices. Alarms can be activated for defects of different predefined severity or criteria. Alarm and defect information from the computing device 1203 can be sent via OPC (i.e., a software interface standard) to a plant network, a programmable logic controller (PLC), or a supervisory control and data acquisition / human machine interface (SCADA / HMI).

[0170] In one embodiment, an encoder (not shown) may be used to measure the conveyor speed so that the location of a detected defective packaged product may be ascertained. A series of pulses from the encoder are received and counted by the system. The count is sent to the processor 1206 to determine the distance down the conveyor where the detected defective packaged product is located, and may be used to time the operation of a defect separator to remove the defective packaged product from the line prior to delivery (as the packaged product moves off the production line).

[0171] Although a local system is shown, it is understood that a distributed system can be utilized by connecting multiple computing devices via a network. In some embodiments, the network can include any wired or wireless communication network, including, for example, a wired or wireless local area network (LAN), a personal area network (PAN), a metropolitan area network (MAN), a wide area network (WAN), and the like, as well as any hardware, software, and / or firmware necessary to implement the same (e.g., network routers, etc.). For example, the network can include cellular, 802.11, 802.16, 802.20, and / or WiMax networks. Furthermore, the network can include a public network, such as the Internet, a private network, such as an intranet, or a combination thereof, and can utilize a variety of currently available or later developed networking protocols, including, but not limited to, TCP / IP-based networking protocols.

[0172] In some embodiments, the multiple computing devices may collectively include the features of computing device 1203 described above with respect to Figure 4. In some embodiments, any number of computing devices may constitute a computing device.

[0173] Exemplary Detection Models and Algorithms In some embodiments, various algorithms, models, and processes can be implemented to detect defects in product packaging and seals of packaged products. Each of the algorithms, models, and processes can be configured to find and identify seal defects. In some embodiments, artificial intelligence algorithms can be implemented (e.g., as shown in FIG. 13), and in other embodiments, algorithmic approaches can be used based on received sensor information (e.g., as shown in FIG. 14). The processes disclosed herein (e.g., algorithmic solutions, model-based solutions, etc.) can receive image data in one or more states of processing and pre-processing; the image data can be further processed, sorted, classified, and analyzed; and seal defects present in the packaged article can be identified based thereon.

[0174] FIG. 13 illustrates a flow diagram of a model-based process 500 (illustrated as a functional block diagram) for evaluating whether a seal of a packaged article is defective. The process 500 may be embodied and performed, for example, by the vision system illustrated in FIG. 8, FIG. 9, and FIG. 11 herein. As illustrated, the system may include one or more imaging devices 210, a data acquisition system 502 (e.g., utilizing the computing device 1203 and acquisition circuitry 1212 illustrated in FIG. 12), a vision inspection engine 504 (e.g., utilizing the computing device 1203 and data acquisition circuitry 1213 illustrated in FIG. 12), an in-line action system 506, a database management system 508, a data collection system 510 (e.g., utilizing the computing device 1203 and data collection circuitry 1213 illustrated in FIG. 12), and a model generator 520 (e.g., utilizing the computing device 1203 and process generation circuitry 1209 illustrated in FIG. 12). FIG. 13 illustrates an example flow diagram for performing the process 500 described according to some embodiments.

[0175] In the illustrated embodiment, using an appropriate combination of imaging devices (e.g., including a selection of lenses and / or sensors) and illumination (e.g., via light source 215), a series of images are acquired and fed to acquisition system 502, where the data is acquired, buffered, and forwarded to either the vision inspection engine 504 or data collection system 510. The illustrated embodiment of FIG. 13 includes two exemplary use cases: a model generation workflow and a model application workflow.

[0176] In the model generation workflow, the process 500 can generate a model from multiple images. The images can be captured by the imaging device 210 and received by the acquisition system 502. The acquisition system can include an image capture engine 540, an acquisition routine 542, a memory buffer 544, and a workspace 546. The acquisition system 502 can buffer the images and send the images to a data collection system 510, which can label the images 512 (e.g., good / sufficient seal or bad / weak seal) and store the images 514. In some embodiments, the model generation workflow can retrieve images directly from the storage device 514, or the images (e.g., a training set) can be loaded into the system separately (e.g., via the communication circuitry 1211 shown in FIG. 12).

[0177] The images can be labeled 512 using any process described herein. For example, a user can enter a label into the system in association with each image. In some embodiments, the packaged article can include a tag or label that identifies a characteristic of the packaged article to a user or a vision system for association with the respective image for the packaged article.

[0178] From the data collection system 510, the labeled images may be input to a model generator 520 to generate one or more models, which may receive images in a first step 521 by receiving the labeled images in an image database 522 and initializing images 523 for use in the model generator. The initialized images may then be passed to image analysis 524 where the images may be pre-processed 525 and analyzed 526. Model generation 527 may then take place to create a number of models based on the analyzed images using the training process described herein 528, and the models may be tested, and in some embodiments a preferred model may be selected 529 based on a model evaluation matrix of its predictions in the test data (which includes parameters such as accuracy, false positives, true positives, recall, precision, etc.).

[0179] Once a model is created and selected by the model generator 520, the model can be deployed to the visual inspection engine 504 in a model application workflow. In the model application workflow, an image is captured by the imaging device 210 and fed to the acquisition system 502 where the data is acquired, buffered, and forwarded to the visual inspection engine 504. In the visual inspection engine 504, the model can be applied to an unknown image to classify the image (e.g., good / sufficient seal or bad / weak seal) based on an artificial intelligence algorithm developed by the model generator 520 in the model generation workflow. The process can include initializing the model 532, and the image can be input to a decision function 534 to receive a decision at a detection model output 536.

[0180] The detection results 538 can be provided to the in-line action system 506 to set up a predefined alarm, flag the film, display (e.g., via a user interface) an image of the faulty seal, display data related to one or more defects including displaying data related to the geometric characteristics of the faulty seal area, the location of the defect, the extent of the occurrence of the defect; the severity of the defect, generate a report of the defect data, and / or any other desired output. The data related to the defects can be displayed online instantly or accessible in the offline database management system 508 after the production is completed (i.e., offline or not on the fly). Using data mining, the data can be manipulated, visualized, and organized into any desired on-demand report form.

[0181] The detection results can be further transferred as labeled images from the visual inspection engine 504 to the data collection system 510, stored in the image storage device 514, and used in the subsequent model generation process to recursively reduce the loss in the objective function and improve the performance of the model based on the model evaluation matrix.

[0182] Data processing software and hardware can be set up to accommodate different seal types, minimizing the need for on-the-fly adjustment of parameters such as exposure time and light intensity. In some embodiments, parts of the process 500 may be performed offline or remotely. For example, the data collection system 510 and / or the model generator 520 may be located remotely from the rest of the process (e.g., performed on a server). Additionally, some or all of the visual inspection process may be performed remotely.

[0183] Turning to FIG. 14, an exemplary flow diagram of an algorithmic process 550 for evaluating whether a packaged article has a faulty seal is shown (illustrated as a functional block diagram). The process 550 may be embodied and performed, for example, in the vision systems shown in FIG. 8, FIG. 9, and FIG. 11 herein. As shown, the system may include one or more imaging devices 210, a data acquisition system 502 (e.g., utilizing the computing device 1203 and acquisition circuitry 1212 shown in FIG. 12), a vision inspection engine 504 (e.g., utilizing the computing device 1203 and data acquisition circuitry 1213 shown in FIG. 12), an inline action system 506, a database management system 508, a data collection system 510 (e.g., utilizing the computing device 1203 and data collection circuitry 1213 shown in FIG. 12), and an inference engine 560 (e.g., utilizing the computing device 1203 and process generation circuitry 1209 shown in FIG. 12). FIG. 14 illustrates an exemplary flow diagram for performing the process 550 described according to some embodiments.

[0184] In the illustrated embodiment, using an appropriate combination of imaging device 210 (e.g., including a selection of lenses and / or sensors) and light source 215 (e.g., via an illuminator, UV illumination, polarized light), a series of images are acquired and fed to acquisition system 502 where the data is acquired, buffered, and forwarded to either the vision inspection engine 504 or data collection system 510. The illustrated embodiment of FIG. 14 includes two exemplary use cases, an algorithm generation workflow and an algorithm application workflow.

[0185] In the algorithm generation workflow, the process 550 can generate an algorithm from multiple images. The images can be captured by the imaging device 210 and received by the acquisition system 502. The acquisition system can include an image capture engine 540, an acquisition routine 542, a memory buffer 544, and a workspace 546. The acquisition system 502 can buffer the images and send the images to a data collection system 510, which can label 512 the images (e.g., good / sufficient seal or bad / weak seal) and store the images 514. In some embodiments, the algorithm generation workflow can retrieve images directly from the storage device 514, or the images (e.g., a test set of images) can be loaded into the system separately (e.g., via the communication circuitry 1211 shown in FIG. 12).

[0186] The images can be labeled 512 using any process described herein. For example, a user can enter a label into the system in association with each image. In some embodiments, the packaged products can include tags or labels that identify characteristics of the packaged product to a user or a vision system for association with the product's respective image. In some embodiments, the training set can be preloaded with labels.

[0187] From the data collection system 510, the labeled images can be input to an inference engine 560 to generate and identify one or more algorithms, which can receive the images in a first step 561 by receiving the labeled images in an image database 562 and initializing the images 563 for use in the inference engine. The initialized images can then be passed to image analysis 564 where the images can be pre-processed 565 and analyzed 566. Algorithm determination 567 can then be made to create one or more algorithms 568 based on the images analyzed using the processes described herein, and in some embodiments, the algorithms can be tested and a preferred algorithm selected 569 based on a model evaluation matrix of its predictions in the test data. In some embodiments, 100 test images are used. In some embodiments, 1000 test images are used. In some embodiments, a user can manually identify desired features in the images and / or image processing parameters to identify features during or prior to algorithm generation. In some embodiments, the system may use the techniques described herein to partially or fully autonomously process images and / or detect desired features within images.

[0188] Once an algorithm is created and selected in the inference engine 560, the algorithm may be deployed to the vision inspection engine 504 in an algorithm application workflow. In the algorithm application workflow, an image is captured by the imaging device 210 and fed to the acquisition system 502 where the data is acquired, buffered, and forwarded to the vision inspection engine 504. In the vision inspection engine 504, an algorithm may be applied to an unknown (e.g., unlabeled) image to classify the image (e.g., good / sufficient seal or bad / weak seal) based on the algorithm selected in the inference engine 560 in the algorithm generation workflow. The process may include initializing an algorithm 572, and the image may be input to a decision function 574 to receive a decision at a detection algorithm output 576.

[0189] The detection results 578 can be provided to the in-line action system 506 to set up a predefined alarm, flag the film, display an image of the seal area (e.g., via a user interface), display data related to one or more defects including displaying data related to the seal geometric characteristics, the location of the defect, the extent of the defect occurrence; the severity of the defect, generate a report of the defect data, and / or any other desired output. The data related to the defects can be displayed online instantly or accessible in the offline database management system 508 after the production is completed (i.e., offline or not on the fly). Using data mining, the data can be manipulated, visualized, and organized into any desired on-demand report form.

[0190] The detection results may be further transferred from the vision inspection engine 504 to the data collection system 510 as labeled images stored in the image storage 514 and used in the subsequent algorithm generation process to improve the evaluation matrix of the algorithm. In some embodiments, the image analysis and algorithm generation in the inference engine 560 may be performed autonomously. In some embodiments, the image analysis and algorithm generation in the inference engine 560 may be performed partially manually. In some embodiments, the image analysis and algorithm generation in the inference engine 560 may be performed manually.

[0191] Data processing software and hardware can be set up to accommodate different concentration levels, minimizing the need for on-the-fly adjustment of parameters such as exposure time and light intensity. In some embodiments, parts of the process 550 may be performed offline or remotely. For example, the data collection system 510 and / or the inference engine 560 may be located remotely from the rest of the process (e.g., performed on a server). Additionally, some or all of the visual inspection process may be performed remotely.

[0192] In some exemplary embodiments, an algorithmic process can be used to detect the seal quality of the packaging article. For example, in some embodiments, a feature extraction based algorithm can be used to detect portions of the seal in the packaging article that exceed a predetermined threshold intensity or exhibit excited fluorescence of a predetermined wavelength or exhibit a change in wavelength from the incident light. In some embodiments, a feature extraction based algorithm can be used to detect portions of the seal in the packaging article that exhibit a temperature gradient that differs from a baseline temperature gradient. In some embodiments, dark feature detection can be used to identify dark and bright spots in the captured image data, which can separate areas of low fluorescence from areas of high fluorescence, which indicate a possible faulty seal. In some embodiments, bright feature detection can be used in a similar manner to dark feature detection. In some embodiments, dark feature detection and bright feature detection can be calculated based on a grayscale value threshold of the image data. In some embodiments, feature extraction can include extracting one or more textures of the image. The image texture can be compared to a reference image to identify faulty seals. In some embodiments, feature extraction can be further used to filter out undesirable areas such as the background or areas around the seal. For example, in some embodiments, seals on packaged articles may have inconsistent textures or thicknesses, which may create false positives in the captured images, hi some embodiments, bright spots caused by seals may be excluded from the algorithm.

[0193] In some embodiments, the algorithmic process may include streak detection, which may be based on extracted geometric features in the captured image.

[0194] 15, in an exemplary embodiment, algorithm 600 (e.g., an algorithm generated by inference engine 560) and associated image processing may include the following steps: (1) capture an image 602; (2) apply de-noising to the image 604; (3) convert the image to grayscale 606; (4) apply thresholding to reduce sections of the image to be considered 608; (5) in some embodiments, eliminate undesirable areas of the image 610; (6) identify features in the remaining image that correspond to faulty seals 612. In some embodiments, algorithm 600 may further include classifying the image based on the detected features 614. In some embodiments, the steps of the aforementioned algorithm may be eliminated as long as faulty seals can be identified.

[0195] As discussed above, in some embodiments, an artificial intelligence algorithm (e.g., the detection model shown in FIG. 13) can be developed to classify image data of a seal (e.g., trained using the model generator 520 shown in FIG. 13). FIG. 16 illustrates one embodiment of a method 700 for developing a trained image classification model. At block 702, training image data of a seal area is obtained. In some embodiments, the training image data includes images and / or videos (i.e., a sequence of images) of a seal having a known state. In some embodiments, the vision system used to obtain the training image data is the same as the vision system used to obtain image data of the seal area of ​​a packaging article in an unknown state after the trained image classification model is created (e.g., a vision system of an end-of-line production line). In some embodiments, a test bed or other experimental configuration can be used to obtain the training image data. At block 704, the training image data is manually labeled with the state of the seal area of ​​the packaging article in the training image data. For example, a user can manually input the state (e.g., the packaging article has a defective seal, has a good seal, or has a particular defect) for each image and / or video of a seal in the image data. Manually labeling the image data may include physically testing the seals to determine whether each seal is suitable or defective, and then applying a label to the image data based on the results of the physical testing. The training sample may include a plurality of seals and a plurality of defective seals. In some embodiments, the number of seals represented in the training image data ranges from tens of seals, hundreds of seals, thousands of seals, or more. At these numbers, the manual labeling process of the training image data may be a labor-intensive and time-intensive process. At block 706, the labeled training image data is input into a training module.

[0196] In some embodiments, the training model is a machine learning module, such as a "deep learning" module. Deep learning is a subset of machine learning that generates a model based on a training dataset provided to it. In some embodiments, the training model can use supervised learning techniques, semi-supervised learning techniques including clustering, anomaly detection, Hebbian learning, unsupervised learning techniques, and learning latent variable models such as expectation maximization algorithms, method of moments (mean, covariance), and blind signal separation techniques, including principal component analysis, independent component analysis, non-negative matrix factorization, and singular value decomposition. In one example, unsupervised learning techniques were utilized in combination with several image pre / post processing techniques, including image texture analysis (e.g., calculation of Leaker Detection index values) to achieve detection and classification. In another embodiment, the unsupervised learning technique Anomaly Detection was utilized in combination with several image pre / post processing techniques, including image texture analysis to achieve detection and classification.

[0197] In an embodiment, the unsupervised learning technique includes the following steps: 1) Data collection, where images are collected for all types of seals without category definition. Best practices are implemented to ensure that only baseline seals are manufactured. 2) Algorithm development, where autoencoder components such as encoders, decoders, and latent space vectors are designed based on input data specifications and expected output. 3) Algorithm training, where an autoencoder neural network is trained with objects against the baseline seal data set to reproduce samples with the finest details. 4) Defect isolation, where the autoencoder post training is deployed to production. Input seals that produce different errors from the baseline examples are isolated. 5) Algorithm retraining, where the isolated defects are verified by humans. A set of false positive samples is prepared and used to retrain the autoencoder to further improve accuracy. This process may be repeated to improve performance.

[0198] In an embodiment, the semi-supervised learning technique includes the following steps: 1) Separating defective seals from the baseline using an autoencoder; 2) Clustering defective seals by grouping them based on similarity scores using a clustering algorithm; 3) Manual categorization and category assignment by a human who evaluates the defective seal clusters; and 4) Automatic categorization, which requires developing a classification algorithm based on the manually categorized defects.

[0199] At block 708, an artificial intelligence algorithm is developed to classify the seal. In some embodiments, when the artificial intelligence algorithm is developed, one or more learning algorithms are used to create the artificial intelligence algorithm based on the labeled states of the seal in the training image data. In some embodiments, the artificial intelligence algorithm is created based on one or more input vectors that indicate characteristics of the seal. In one example, the input vector may be pleats in the seal. In another example, the input vector may be characteristics of fluorescent electromagnetic energy emitted by an excited fluorescent-based indicator in the film according to the particular indicator and illumination used, as described above. For example, the input vector may include one or more of: a wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescent-based indicator; an intensity of the fluorescent electromagnetic energy emitted by the excited fluorescent-based indicator; and / or a change in wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescent-based indicator compared to the incident electromagnetic energy from the illuminator. In some embodiments, the input vector can correspond to the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescent-based indicator that is one or more colors in the visible spectrum, detection of additives in the film of the packaging article using non-visible electromagnetic energy (e.g., ultraviolet, infrared), thermal imaging scans, visible light scans, photoelastic scans, the presence and number of film folds, or any other number of possible input vectors. In one embodiment, the light source 215 may be an ultraviolet backlight with software to control the shutter speed and light intensity. In other embodiments, the light source 215 is a white light. In some embodiments, the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescent-based indicator can be at least in the ultraviolet range. In some embodiments, the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescent-based indicator can be at least in the visible spectrum. In some embodiments, the wavelength of the fluorescent electromagnetic energy emitted by the excited fluorescent-based indicator can be at least in the blue or violet range.In some embodiments, the wavelength of the incident light from the illuminator may be at least in the ultraviolet range. In some embodiments, the wavelength of the incident light from the illuminator may define a peak in the ultraviolet range. In some embodiments, the wavelength of the incident light may be in the ultraviolet range and the fluorescent electromagnetic energy emitted by the excited fluorescence-based indicator may be in the visible range. In embodiments where the process is designed to simultaneously inspect multiple film layers at the same time for the same film, multiple light sources 215 and / or multiple imaging devices 210 may be used along with one or more controls for shutter speed, light intensity, light source, backlighting or overlighting.

[0200] The use of input vectors for training can help the artificial intelligence algorithm to identify defective seals without identifying the root cause. For example, a seal may have small pleats that are difficult to detect using image data captured while the packaged article is moving on a conveying system. The use of input vectors, fluorescent-based indicators, thermal imaging data, photoelastic imaging data, as detailed herein, allows the artificial intelligence algorithm to detect that a seal is defective without having to identify the defect itself. After the input vectors are modeled, the artificial intelligence algorithm can be developed as a decision-making process based on several input vectors. Examples of decision-making processes include decision trees, neural networks, and the like. In some embodiments, the decision-making process of the artificial intelligence algorithm is based on determining an acceptable arrangement of the input vectors in the decision-making process.

[0201] The result of the development of the artificial intelligence algorithm in block 708 is an artificial intelligence algorithm shown in block 710. The artificial intelligence algorithm can be used during normal operation (e.g., operation not used to train the artificial intelligence algorithm) to identify the condition of the seal. In some embodiments, the artificial intelligence algorithm includes a neural network having several layers. An example of a neural network 1800 that is a multi-layer neural network is shown in FIG. 20. In the illustrated embodiment, the neural network 1800 includes a first layer 1802 with three input nodes, a second layer 1804 with five hidden nodes, a third layer 1806 with four hidden nodes, a fourth layer 1808 with four hidden nodes, and a fifth layer 1810 with one output node.

[0202] The neural network 1800 also includes a first set 1812 of connections between each pair of the three input nodes of the first layer 1802 and the five input nodes of the second layer 1804, a second set 1814 of connections between each pair of the five input nodes of the second layer 1804 and the four hidden nodes of the third layer 1806, a third set 1816 of connections between each pair of the four hidden nodes of the third layer 1806 and the four hidden nodes of the fourth layer 1808, and a fourth set 1818 of connections between each pair of the four hidden nodes of the fourth layer 1808 and the output nodes of the fifth layer 1810. In some embodiments, the input nodes represent inputs to the artificial intelligence algorithm (e.g., image data, metadata associated with the image data, etc.), one or more hidden nodes (e.g., one of the layers of hidden nodes) can represent one of the input vectors determined during development of the model, and the output node represents the determined state of the seal.

[0203] 17, one embodiment of a method 900 for classifying a seal condition (e.g., pleated, weak seal, cold seal, channel pleated) using a trained image classification model. At block 902, image data of the seal is acquired (e.g., by the imaging device 210). The image data of the seal may be obtained by a vision system. In some embodiments, the image data of the seal is obtained while the packaged article is being transported by the transport system. It is understood that the image data may be captured after forming a seal in the formation of the packaged article or after forming a seal to seal a product within the packaged article.

[0204] At block 904, image data of the seal is input to a trained image classification model. The trained image classification model can operate on a computing device or a remote computing device from the local computing device. The trained image classification model is configured to classify a state of the seal based on the image data. At block 906, a classification of the state of the seal is received from the trained image classification model. In some embodiments, the classified state includes an indication that the seal is defective, not defective, or has a particular defect, and / or an indication of a certainty as to whether the seal is defective, not defective, or has a particular defect. In some embodiments, the classified state is received by one or more of displaying the classification on a user interface output device, communicating the classification to one or more external devices via a communication interface, or storing the classification in a database. In some embodiments, the received classification of the seal includes one or more of a classified state or seal, or a certainty of the classified state of the classified state of the seal. In one particular example, the status is communicated to a routing system (e.g., a sending device) configured to route the packaged articles on a conveying system based on their seal status, such as routing defective packages to a location for testing, repackaging, recycling and / or waste disposal.

[0205] As described above, the method 700 may be used in block 710 to obtain a trained classification model, which may then be used in method 900 to classify seals. In some embodiments, the training image data acquired in block 702 is image data of seals of a particular type of product packaged in a packaging article, and the image data acquired in block 902 is image data of the same type of seals of a particular type of product packaged in a packaging article. In one example, the training image data acquired in block 702 is image data of seals, and the image data acquired in block 902 is image data of the same type of seals. In some embodiments, the training image data acquired in block 702 is image data of a particular type of seal, and the image data acquired in block 902 is image data of a different type of seal. Even if the seals imaged in-line are a different type than the seals used in the training set, the trained classification model using the training image data from the training seal image data may be able to classify the condition of the seal with sufficient accuracy.

[0206] FIG. 18 illustrates one embodiment of a method 1010 for developing a trained image classification model. In block 1012, training image data is obtained for a number of seals. In block 1014, the training image data is manually labeled as defective or non-defective. Manual labeling of the training image data may be performed by a user inputting an indication of defect or non-defect for each of the seals represented in the training image data into a user interface input device of the computing device. In some embodiments, training packaging articles having seals may be labeled with their respective status (e.g., defective or non-defective). The training packaging articles are packaging articles having either known defective seals or known non-defective seals. The training packaging articles may further identify certain defects. The training packaging articles are used to train the model to improve the reliability of the model.

[0207] At block 1016, model information, training goals, and constraints are initialized. In some examples, the model information includes the type of model to be used, such as neural network, number of input vectors, etc. In some examples, the training goals can include a desired or expected performance of the artificial intelligence algorithm, such as an evaluation matrix having a confidence level equal to or greater than a predetermined rate (e.g., one or more of 90%, 95%, 96%, 97%, 98%, or 99%). In some examples, the constraints can include limitations of the artificial intelligence algorithm, such as a minimum number of layers of the neural network, a maximum number of layers of the neural network, a minimum weighting of the input vectors, a maximum weighting of the input vectors, or some other constraint of the artificial intelligence algorithm. At block 1018, a model can be trained using the model information and the model constraints. In some embodiments, the training image data is divided into two subsets - a training subset and a validation subset - and training the model at block 1018 includes training the model using the training subset of the image data.

[0208] At block 1020, a determination is made whether the training goals have been met. In some embodiments, the determination at block 1020 is made by comparing the results of the artificial intelligence algorithm to the training goals initialized at block 1016. In some embodiments, if the training image data is split into a training subset and a validation subset, the determination at block 1020 includes testing the model trained at block 1018 using the validation subset of the image data. If a determination is made at block 1020 that the training goals have not been met, the method 1010 proceeds to block 1022 where the training goals and / or constraints are updated. After the training goals and / or constraints are updated at block 1022, the method 1010 returns to block 1018 where the model is trained using the updated training goals and / or constraints. If a determination is made at block 1020 that the training goals have been met, the method 1010 proceeds to block 1024 where the artificial intelligence algorithm is stored. Storing the artificial intelligence algorithm may include storing the artificial intelligence algorithm in one or more memories within a computing device (eg, a local computing device, a remote computing device, etc.).

[0209] In some embodiments, the vision system may be used both to train a model to classify the seal condition and to apply an artificial intelligence algorithm to classify the seal condition. In FIG. 19, an embodiment of a method 1100 for a vision inspection engine for both training a model to classify the seal condition of a packaging article and applying an artificial intelligence algorithm to classify the seal condition is shown. In some embodiments, the vision system includes an image sensor system and a computing device that can define the vision inspection engine 504 and the acquisition system 502. In these embodiments, the model can run on the computing device while the imaging device obtains image data of the seal to either train or apply the model.

[0210] At block 1102, the vision system and the classification model are initialized. In some embodiments, the initialization of the vision system includes initializing the computing device and initializing the imaging device, and the initialization of the classification model includes loading startup software including the classification model onto the computing device. At block 1104, image data of the seal is acquired (e.g., by the imaging device and acquisition system). In some embodiments, the imaging device acquires image data of the seal and provides the image data to the computing device. At block 1106, a determination is made whether the classification model is in training mode. The determination may be by software running on the computing system that includes the classification model.

[0211] If a determination is made in block 1106 that the classification model is in training mode, the model proceeds to block 1108 where it is determined whether a state is available for the seal. The state can be made available when a user manually inputs the seal state into a computing device or scans the seal state (e.g., from a bar code or other indicia on a training package article). If a determination is made in block 1108 that the state is available, the method proceeds to block 1110. In block 1110, the classification model is updated based on the image data and the seal state. Updating the classification model may include any of the methods described herein for training and / or developing a classification model. As shown in block 1112, the seal state (e.g., a manually entered state) is available. However, if a determination is made in block 1106 that the classification model is not in training mode or if a determination is made in block 1108 that the state is not available, the method proceeds to block 1114.

[0212] At block 1114, the classification model classifies the seal condition. In some embodiments, the seal condition classified by the classification model includes a determination of whether the seal is defective (e.g., pleated or weak seal), non-defective, or has a particular defect, and an indication of a confidence level as to whether the seal is defective, non-defective, or has a particular defect. At block 1116, a determination is made of whether the confidence level of the classified condition is low or high. In some embodiments, the confidence level is a percentage representing the confidence that the classified seal condition is accurate, and the confidence level is low if the confidence level is below a predetermined percentage of an acceptable confidence level. For example, if an acceptable confidence level is 90%, then if the confidence level of the classified condition is less than 90%, then the classified condition of the seal is considered low. If it is determined at block 1116 that the confidence level is not low, then the seal condition has been determined, as shown at block 1118. However, if at block 1116 the confidence level is determined to be low, the method proceeds to block 1120 where the sealed product is reserved for offline and / or manual sorting (e.g., sorting by a user after visual inspection or physical testing off the production line). In embodiments where multiple vision systems are utilized, either redundant or of different types (e.g., thermal and photoelastic), the potential for increased confidence is improved.

[0213] If the seal status is available at either block 1112 or block 1118, the method proceeds to block 1122. At block 1122, the seal status is output. In some embodiments, outputting the seal status includes one or more of displaying the seal status on a user interface output device, communicating the seal status to one or more external devices via a communication interface, or storing the seal status in a database. In some embodiments, the seal status includes one or more of an indication of whether the seal is defective, non-defective, or has a particular defect, or a certainty of whether the seal is defective, non-defective, or has a particular defect.

[0214] Whether the seal status is output at block 1122 or the packaged article is retained for manual sorting at block 1120, the method 1100 then proceeds to block 1124. At block 1124, a determination is made whether another packaged article is available. In some embodiments, the determination at block 1124 can be based on whether another packaged article is detected on the conveyor (e.g., via one or more sensors). In some embodiments, the determination at block 1124 can be based on whether a user inputs an indication of whether another packaged article is available. If a determination is made at block 1124 that another packaged article is not available, then at block 1126, the vision system and the classification model are shut down. However, if a determination is made at block 1124 that another packaged article is available, the method 1100 loops back to block 1104, image data of the seal of the next packaged article is obtained, and the method 1100 proceeds from block 1104 as described above for the next packaged article.

[0215] As discussed above, an artificial intelligence algorithm that classifies the seal condition of a packaged article from image data can include one decision-making process, such as a decision tree or a neural network. In some embodiments, an artificial intelligence algorithm that classifies the seal condition from image data can include two or more decision-making processes. FIG. 21 illustrates an embodiment of a method 1200 for classifying a seal condition. In the illustrated embodiment, the method 1200 is performed in part by an image sensor system 1202, a detection decision-making process 1204, a classification decision-making process 1206, and an output device 1208. In block 1210, the image sensor system acquires image data of the seal. In some embodiments, the image sensor system 1202 can acquire image data as the packaged article is being transported by the transport system. After the image data is acquired in block 1210, the image sensor system has image data 1212 that can be communicated to the detection decision-making process 1204. In one embodiment, the detection decision-making process 1204 is a software-based decision-making process running on one or more computing devices.

[0216] In block 1214, the detection decision process 1204 processes the image data received from the image sensor system 1202. In some embodiments, the processing of the image data in block 1214 is performed by an artificial intelligence algorithm trained to detect regions of interest associated with seals of packaged articles in the image data. In some embodiments, the processing of the image data in block 1214 includes one or more of cropping an image in the image data around a detected seal in the image, selecting a frame or a subset of frames from a video in the image data, identifying irrelevant pixels from the image in the image data, and replacing the irrelevant pixels with a least significant value in the image data. In some embodiments, the processing of the image data produces a single image having a rectangular shape, the identified seal is substantially in the center of the image, and the pixels deemed irrelevant are replaced with the least significant value. In some embodiments, processing of the image data may include masking portions of the image, where areas of the image outside the region of interest (e.g., outside the seal) are replaced with low value data (e.g., all pixels are changed to black) to reduce the amount of processing to classify the seal condition and to reduce the chance of error when classifying the seal condition.

[0217] In one embodiment of processing the image data, a custom boundary is constructed around a representation of the seal in the image data. A bounding box encompassing the seal is also constructed within the custom boundary. This processing also includes cropping the bounding box from the entire image data. One advantage of cropping the image data based on the custom boundary is that subsequent classification of the seal's condition may be limited to areas of interest without the need to inspect areas of the image data that are not of interest. This, in turn, may increase the confidence level of the classification and therefore the overall performance of the classification. In some embodiments where the detection decision process 1204 is a multi-layer neural network, creating a bounding box around the custom boundary simplifies the compatibility requirements between the image data and the first layer of the neural network. Additionally, cropping the image data results in a portion of the image data being processed for classification rather than the entire image data, which reduces processing time for classification. In some embodiments, the custom boundary may help generate a numerical value for one or more of the seal's area, its center of gravity, or its orientation.

[0218] At block 1216, a determination is made whether the presence of a seal is detected in the image data. In some embodiments, the determination made at block 1216 is part of the processing of the image data at block 1216. In some embodiments, the determination of whether a seal is detected at block 1216 is a separate process from the processing of the image data at block 1216. If a determination is made at block 1216 that the presence of a seal is not detected, the method 1200 proceeds to block 1218 where the image data is discarded (e.g., deleted) and the method 1200 ends. However, if a determination is made at block 1216 that the presence of a seal is detected, the processed image data, represented by block 1220, can be communicated to a classification decision-making process 1206. In some embodiments, the classification decision-making process 1206 is a software-based decision-making process operating on one or more computing devices that may be the same or different from the one or more computing devices on which the detection decision-making process 1204 operates. In some embodiments, as shown in block 1220, processing the image data in block 1214 to obtain processed image data prior to classifying the state of the seal represented in the data increases the evaluation matrix for subsequent classification by the classification decision-making process 1206.

[0219] At block 1222, the classification decision process 1206 classifies the processed image data received from the detection decision process 1204. In some embodiments, the classification of the image data at block 1222 is performed by an artificial intelligence algorithm trained to classify the condition of the seal represented in the processed image data. In some embodiments, the classification of the condition of the seal represented in the processed image data at block 1222 includes a determination of whether the seal is defective, non-defective, or has a particular defect. In some embodiments, the classification of the condition of the seal represented in the processed image data at block 1222 includes a determination of whether the seal is defective, non-defective, or has a particular defect and an indication of the certainty as to whether the seal is defective, non-defective, or has a particular defect.

[0220] At block 1224, a determination is made whether the confidence level of the classified state is low. In some embodiments, the confidence level is a percentage representing the confidence that the classified seal state is accurate, and the confidence level is low if the confidence level is below a predetermined percentage of an acceptable confidence level. For example, if an acceptable confidence level is 90%, then the classified state of the seal is considered low if the confidence level of the classified state is less than 90%. If at block 1224, it is determined that the confidence level is not low, then the seal state has been determined, as shown at block 1226. However, if at block 1224, a determination is made that the confidence level is low, the method proceeds to block 1228, where the seal and / or image data is flagged for manual classification.

[0221] At block 1230, the seal condition is manually classified outside of the classification decision-making process. In some embodiments, the seal is manually classified by a user after visual inspection or physical testing of the packaged article. At block 1232, the user inputs the manually classified seal condition into the classification decision-making process 1206. At block 1234, the classification decision-making process 1206 is updated. In embodiments where the classification decision-making process 1206 is an artificial intelligence algorithm, updating the classification decision-making process 1206 includes further training the artificial intelligence algorithm based on the manual classification. After updating the classification decision-making process 1206, the method 1100 returns to block 1226, where the classified condition of the vacuum packaged product is the manually classified status of the seal.

[0222] As represented in block 1226, after the classified status of the vacuum packaged product has been classified or obtained by the classification decision-making process 1206, the classification decision-making process 1206 sends the classified seal status to an output device 1208. In embodiments where the classification decision-making process 1206 is software running on one or more computing devices, the output device 1208 can be a user interface output device. In some embodiments, outputting the classified status of the seal at block 1236 includes one or more of outputting the classified status of the seal to a user via a user interface (e.g., a monitor, a touch screen, etc.), communicating the classified status of the seal to an external device via a communication circuitry, or storing the classified status of the seal locally in a database.

[0223] In any of the embodiments disclosed herein, the image data received for any one seal can include multiple forms of image data for the same seal. For example, the image data for the seal can include two images in the visible light range of the same seal. In another embodiment, the image data for the seal can include three images, one thermographic image, one ultraviolet image, and one photoelastic image. These multiple different forms of image data for the same seal can be passed through an artificial intelligence algorithm separately. If the artificial intelligence algorithm returns the same classification state of the seal using multiple different forms of image data, the confidence level of the classification of the seal can be significantly increased. In one example, if the artificial intelligence algorithm classifies one image as having a packaged article with a defective seal with a confidence level of 98% and the other image as having a packaged article with a defective seal with a confidence level of 96%, the confidence level that the packaged article has a defective seal can be greater than 99%. In another example, if an artificial intelligence algorithm classifies one image as having a non-defective seal with a confidence level of 60% and the other image as having a non-defective seal with a confidence level of 70%, the confidence level that the vacuum packaged product is free of defects may be 88%. Although the confidence level using two images may be significantly higher than either image alone, the combined confidence level from the two images (e.g., 88%) may still be below a predetermined percentage of acceptable accuracy (e.g., 95%) and the packaged article may be flagged for manual classification. In some further embodiments, multiple camera angles may be used to image the seal from multiple perspectives, such as on multiple surfaces, through the packaged article, at an angle from the packaged article, or directly above the packaged article. In some embodiments, more than one camera angle for imaging the same seal may be used. It will be apparent that the number of multiple forms of image data is not limited to two and may be any number of forms of image data.

[0224] In some embodiments, not all areas of increased or altered image indicators or defects are required for detection and reporting, and a user or computer device may determine one or more thresholds to facilitate identification of defective seals. The thresholds may be based on any of the parameters detailed herein, may be pre-determined, or may be applied as a result of an algorithm or model. The thresholds may be set such that only altered image indicators and defects exceeding a threshold size are flagged for removal. For example, the thresholds may be set for fluorescent areas having a size of at least 2 millimeters in at least one direction (i.e., altered fluorescence having a size of at least 2 millimeters in the machine direction and / or at least 2 mm in the cross direction). Alternatively, the thresholds may be set for thermal image anomalies of at least 1 millimeter in size in at least one direction (i.e., altered fluorescence of at least 1 millimeter in at least one direction). In some embodiments, the thresholds may be set for a predetermined surface area (e.g., an area of ​​at least 1 mm2, 2 mm2, 3 mm2, or 4 mm2). Such thresholds may be set even if the system has the ability to see defects down to a size of as little as 10 microns in at least one direction. Setting a threshold is distinct from the ability of a machine vision system to detect defects of at least a particular size in at least one direction. Rather, setting a threshold is a setting of a minimum value for the size of an area that will trigger the generation of a signal in response thereto. The threshold can be set to any desired value and is distinct from the ability of a machine vision system to detect defects at least up to a specified size.

[0225] In some embodiments, algorithms or model-based detection means disclosed herein can be used to detect defects in a packaging article. For example, in some embodiments, the algorithms or model-based detection means described herein can detect areas of high fluorescence higher than the majority of the seal, which can indicate pleats or other defects in the seal. In some embodiments, the algorithms or model-based detection means can detect a substantially uniform constant fluorescence from the seal, indicating a strong seal. In some embodiments, areas of low or no fluorescence within an area of ​​the seal can indicate a discontinuity or weakness in the seal.

[0226] In some embodiments, algorithms or model-based detection means disclosed herein can be used to detect defects in the packaged article. For example, in some embodiments, the algorithms or model-based detection means described herein can detect areas of high thermal radiation concentration greater than the majority of the seal, which can indicate pleats or other defects in the seal. In some embodiments, the algorithms or model-based detection means can detect a substantially uniform and consistent thermal image from the seal, indicating a strong seal. In some embodiments, areas of low or no thermal radiation within the area of ​​the seal can indicate discontinuities or weaknesses in the seal.

[0227] In some embodiments, the algorithms or model-based detection means disclosed herein can be used to detect defects in packaged articles. For example, in some embodiments, the algorithms or model-based detection means described herein can detect defects visible under a photoelastic image.

[0228] example In an exemplary embodiment, the algorithms and image processing may include some or all of the following steps: (1) capture an image; (2) apply noise removal to the image; (3) convert the image to grayscale; (4) apply thresholding to reduce the sections of the image that are considered; (5) eliminate undesirable areas of the image; and (6) identify features in the remaining image that correspond to faulty seals in the packaging article. Examples of each of these processing steps are shown and described in the following figures.

[0229] Turning now to FIG. 22, thermal imaging data of a satisfactory seal is shown. An impulse seal was created on a packaged article. Impulse seals are known to create non-linear or wavy seal areas. The image was captured 5 seconds after seal creation. Uniform active cooling was utilized after sealing and prior to image capture. Such a satisfactory seal image can be used as a good seal image for artificial intelligence algorithms. FIG. 23 is an exemplary image of thermal imaging data of a poor seal. An impulse seal was created on a packaged article. The image was captured 5 seconds after seal creation. Uniform active cooling was utilized after sealing and prior to image capture. The image and similar poor seal images can be used to train artificial intelligence algorithms to demonstrate defective or insufficient seals.

[0230] FIG. 23 is an exemplary thermal image showing a double linear seal with localized sealing pleats. An impulse seal was formed on the packaged article. The image was captured 5 seconds after the seal was created. Uniform active cooling was utilized after sealing and prior to image capture. Discontinuities in the thermal image along the seal area demonstrate a cool, weak seal. Such images are useful for training the artificial intelligence algorithms described herein.

[0231] FIG. 24 is an exemplary thermal image showing a double linear seal with a weak seal area. An impulse seal was formed on the packaged article. The image was captured 5 seconds after the seal was created. Uniform active cooling was utilized after sealing and prior to image capture. The discontinuities in the thermal image along the seal area demonstrate the cold and weak seal. Such images are useful for training the artificial intelligence algorithms described herein.

[0232] Turning now to Figures 25 and 26, thermal imaging data images are shown that demonstrate acceptable seals. Figures 25 and 26 are taken of the same seal at two separate locations and two separate times. The sealed area radiates fairly consistent heat radiation along the seal without hot or cold spots or areas. The heat in the sealed area is residual heat from the sealing process and dissipates over time. Figure 25 is an image taken 15 seconds into the heat sealing process. Figure 26 is an image taken 20 seconds after heat sealing. The time of taking the thermal image after heat sealing can be adjusted depending on the film, the packaged article, the thickness of the film, the heat sealing temperature, and the method.

[0233] Referring to Figures 27 and 28, thermal imaging data images are shown that demonstrate defective seals. Figures 25 and 26 are taken of the same seal at two separate locations and two separate times. The seal area radiates fairly consistent heat radiation along the seal, except for hot spots 3101, 3102, 3201, and 3202. The heat in the seal area is residual heat from the sealing process and dissipates over time. Figure 27 is an image taken 15 seconds into the heat sealing process. Figure 28 is an image taken 20 seconds after heat sealing. The time to capture the thermal image after heat sealing can be adjusted depending on the film, packaging article, film thickness, heat sealing temperature, and method.

[0234] Turning now to Figure 29, there is shown a seal area captured by a photoelastic imaging system as described herein. The photoelastic image capture mechanism provides enhanced visual capabilities compared to standard image capture techniques. Figure 29 demonstrates a proper seal without defects.

[0235] 30 and 31 demonstrate the seal area captured by the photoelastic imaging system. The images improve the visibility of pleat defects compared to standard images. The visual inspection engine analyzes the images for color gradient variations of adjacent pixels, section comparisons, etc. The visual inspection engine may compare one or more of the average, variability, skew, minimum, maximum, range or variation of the seal area, number of pixels or number of sections of the images. Variations between pixels or sections of the images may indicate seal defects.

[0236] FIG. 32 demonstrates an ultraviolet image being taken of a packaged article having an optical brightener disposed therein. The optical brightener fluoresces with a higher intensity in the sealed area including the pleat defect. Additional material is bunched around the pleat defect causing additional illumination. The visual inspection engine analyzes the image for variations in color gradients of adjacent pixels, comparison of sections, etc. The visual inspection engine may compare one or more of an average, variable, skew, minimum, maximum, range or variation of the sealed area, number of pixels or sections of the image. Variations between pixels or sections of the image may indicate a seal defect.

[0237] conclusion Many modifications and other embodiments of the disclosures set forth herein will occur to those skilled in the art to which these inventions pertain having the benefit of the teachings set forth in the foregoing description and the associated drawings. It is therefore to be understood that the disclosure is not limited to the specific embodiments disclosed herein, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Furthermore, while the foregoing description and the associated drawings describe example embodiments in the context of certain example combinations of elements and / or functions, it should be understood that different combinations of elements and / or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, different combinations of elements and / or functions other than those expressly set forth above are also contemplated, for example as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. 1. A process for monitoring seal quality of packaged articles, comprising: A. heat sealing a polymeric film having a total free shrink of at least 10 percent in either the machine direction or the cross direction at 85° C. to itself, or to another polymeric multilayer film, to form at least one sealed area, thereby forming a packaged article; B. acquiring a first set of image data of the at least one seal area with a first vision system comprising an image capture device, wherein the first vision system is a thermographic vision system comprising an infrared imaging device capable of capturing a temperature distribution based on an amount of infrared radiation emitted from the seal area, wherein the first set of image data is time delayed thermographic data captured once after formation of the at least one seal area, wherein the first set of image data is acquired between 2 and 30 seconds after formation of the at least one seal area; C. acquiring a second set of image data of the at least one seal area with a second vision system, the second vision system being different from the first vision system, and the second vision system: a. a first linear polarizer having a polarization direction oriented in a first direction; b. a second linear polarizer having a polarization direction oriented orthogonal to the first direction; c. a light source; d. an image capture device for capturing a second set of image data, the second set of image data being photoelastic image data; Equipped with acquiring a second set of image data; D. Evaluating the first and second sets of image data of the seal area with a vision inspection engine to verify both the continuity and strength of the seal area. A process comprising:

2. 10. The process of claim 1 further comprising a third vision system that is an ultraviolet vision system comprising an ultraviolet light source, the film comprising at least one layer comprising a blend of a polymer and a fluorescent-based indicator, the fluorescent-based indicator being exposed to radiation at a first peak wavelength and the fluorescent-based indicator emitting radiation at a second peak wavelength.

3. The process of claim 2 , wherein the ultraviolet vision system further comprises a white light source.

4. E. exposing the packaged article to incident radiation to excite the fluorescence-based indicator such that the fluorescence-based indicator fluoresces; F. acquiring image data of the fluorescence emitted from the seal area by the packaged article while the indicator is fluorescing; The process of claim 3 further comprising:

5. 5. The process of claim 4, wherein the fluorescence based indicator comprises at least one member selected from the group consisting of ultraviolet indicators, infrared indicators, dyes, pigments, optical brighteners, optical brighteners, 2,2'-(2,5-thiophenylenediyl)bis(5-tert-butylbenzoxazole), hydroxyl-4-)p-tolylamino)anthracene-9,10-dione, 2,5-thiophenediylbis(5-tert-butyl-1,3-benzoxazole) and anthraquinone dyes.

6. 6. The process of claim 2, wherein the visual inspection engine comprises a computing device including computer-executable instructions configured to determine whether fluorescent electromagnetic energy emitted by an excited fluorescence-based indicator indicates a defective seal.

7. The process of claim 6 , wherein the computer executable instructions include at least one artificial intelligence algorithm selected from the group of supervised, unsupervised, or semi-supervised methodologies.

8. 8. The process of any one of claims 2 to 7, wherein determining that the seal is defective comprises determining at least one of: (i) the film exhibits a higher or lower fluorescence intensity in a first region of the seal relative to a second region of the seal; (ii) the film exhibits a higher or lower fluorescence intensity in the first region of the seal relative to an expected level of fluorescence; or (iii) both (i) and (ii).

9. 9. The process of claim 1, wherein the visual inspection engine comprises a computing device including computer-executable instructions configured to determine whether captured thermal imaging data of the seal indicates a defective seal.

10. 10. The process of any one of claims 1 to 9, wherein determining that the seal is defective comprises determining that the film exhibits a higher or lower thermal temperature in a first region of the seal relative to a second region of the seal.

11. 11. The process of claim 1, wherein image data of at least one seal area is captured by an image capture device at a rate of at least any of 5 images per second, 30 images per second, 100 images per second, or 250 images per second.

12. The process of claim 1 , wherein the visual inspection engine assigns a seal score value to the image data of the seal area.

13. 1. A system for detecting defective seals on a packaged article, comprising: A. a sealing mechanism configured to heat seal a polymeric film having a total free shrink of at least 10 percent in either the machine direction or the cross direction at 85° C. to itself or to another polymeric multilayer film to form at least one seal area, thereby forming a packaged article; B. A first vision system comprising an image capture device configured to capture a first set of image data of the at least one seal area with the vision system, the first vision system being a thermographic vision system comprising an infrared imaging device capable of capturing a temperature distribution based on an amount of infrared radiation emitted from the seal area, the first set of image data being time delayed thermographic data captured once after formation of the at least one seal area, the first set of image data being captured between 2 and 30 seconds after formation of the at least one seal; C. a second vision system configured to acquire a second set of image data of the at least one seal area, the second vision system being different from the first vision system, the second vision system being: a. a first linear polarizer having a polarization direction oriented in a first direction; b. a second linear polarizer having a polarization direction oriented orthogonal to the first direction; c. a light source; d. an image capture device for capturing a second set of image data, the second set of image data being photoelastic image data; a second vision system comprising: D. a vision inspection engine configured to evaluate the first and second sets of image data of the seal area to verify both the continuity and strength of the seal area; A system comprising:

Images