Quality control method for packages produced by drum type packaging machine

Abstract

The invention relates to a method (100) for quality control of packages produced by a roll packaging machine, the roll packaging machine comprising an image data acquisition device arranged at a location on a web of packaging material. The method (100) comprises: acquiring (S102) image data depicting a section of the web from an image data acquisition device; identifying (S104) anomalies in a section of the web of packaging material using a machine learning model trained to identify anomalies in the image data of the web of packaging material; and transmitting (S112) a signal indicative of the operation to be performed to a control system of the packaging machine in response to the identified anomaly.

Description

Technical Field

[0001] The present invention relates to the field of packaging technology. More specifically, the present invention relates to a method and apparatus for quality control of packages produced by a reel-type packaging machine, and a corresponding food production system. Background Art

[0002] In today's food industry, food packaging technology plays an important role. Food packaging has multiple important functions. In addition to product branding and providing information to consumers, food packaging also plays an important role in ensuring food safety. The packaging materials used in food packaging can be designed to have strength and stability to avoid damage during transportation. In addition, the packaging materials can also form a protective environment for the food to protect it from bacteria, germs, oxygen, sunlight, etc., thereby extending the shelf life. However, the packaging materials are not the only important factor in packaging. The packaging also needs to be properly sealed. Any damage to the packaging materials or sealing errors may lead to the destruction of the protective environment. For example, in a reel-type packaging system, the packaging is formed by longitudinally sealing along the overlapping edges at both ends of the packaging material to form a tubular structure. Then, the packaging can be formed by transversely sealing the top and bottom of the tubular structure.

[0003] Therefore, many aspects of food packaging are crucial. Ensuring the quality of food packages produced by packaging machines is not an easy task. Currently, a common method is to manually evaluate the finished product batches. In addition to visually inspecting the outside of the package, it is also necessary to open the package to check the internal sealing. However, this is a cumbersome task and also results in product waste. In addition, in order to ensure quality, a large number of packages must be evaluated. Therefore, an improved method for quality control of packages produced by packaging machines is needed. Summary of the Invention

[0004] The technology disclosed herein aims to at least partially alleviate, mitigate, or eliminate one or more of the above-mentioned deficiencies and drawbacks in the prior art. Specifically, the object of the present invention is to provide a method and apparatus for controlling the quality of packages produced by a reel-type packaging machine.

[0005] The inventors have achieved a new and improved method for evaluating packaging quality using machine learning. The present invention further provides continuous monitoring of the packages, enabling a higher confidence in packaging quality compared to the current method of manually evaluating samples. It also allows for action to be taken at an early stage, thereby reducing product waste. The use of the proposed machine learning model helps to increase the throughput of image data while being able to detect even minor anomalies.

[0006] The various aspects and embodiments disclosed in the present invention are defined below and in the appended independent and dependent claims.

[0007] According to a first aspect, there is provided a method for controlling the quality of packages produced by a web-fed packaging machine, wherein the web-fed packaging machine includes an image data acquisition device disposed at a certain position on a web of packaging material. The method includes: obtaining image data depicting a section of the web from the image data acquisition device; using a machine learning model trained to identify anomalies in the image data of the web of packaging material to identify anomalies in the section of the web of packaging material; and in response to the identified anomalies, transmitting a signal indicating an operation to be performed to a control system of the packaging machine.

[0008] The term "quality control" as used throughout this disclosure refers to the assessment of one or more quality aspects of a package, particularly a web of packaging material. Quality aspects may be related to food safety, such as finding any damage or rupture of the web of packaging material (e.g., due to handling or transportation of the packaging material), or defects in the packaging material (e.g., due to incorrect positioning of pre-punched holes). Errors in the packaging material manufacturing process can lead to quality problems. Another quality aspect may be related to the aesthetics of the package, such as checking for deformation of the package, the presence of joints (i.e., the area where the two ends of the web of packaging material are joined), or printing errors / misalignments in the package decoration. Another example of a quality aspect is detecting whether the web of packaging material is misaligned in the packaging machine. Any visible signs in the image data related to the quality control of the web of packaging material are referred to herein as anomalies. Anomalies may be related to one or more different quality aspects. For example, an anomaly such as deformation of the web of packaging material may be related to both food safety aspects and aesthetic aspects.

[0009] The phrases "the package" or "a package" as used throughout this disclosure, unless otherwise specified or understood from the context, refer to a package formed from a portion of the web of packaging material that contains the section of the web where an anomaly may be depicted. For example, if an anomaly is found in a portion of the web of packaging material and not immediately addressed, the package containing the anomaly may subsequently be discarded.

[0010] The term "produced in a web-fed packaging machine" in phrases such as "produced in a web-fed packaging machine" means that the package has been produced in, or is being produced in, a web-fed packaging machine.

[0011] The image data acquisition device may, for example, be a camera. The image data may include a single frame acquired by the image data acquisition device. Alternatively, the image data may include a plurality of consecutive frames acquired by the image data acquisition device.

[0012] The depicted section of the web shall be understood as a finite area of the web in the feed direction (or longitudinal direction). In the width direction (or transverse direction), the depicted section of the web may span sub - portions of the web. The sub - portions may be, for example, a first edge section, an internal section, or a second edge section. The depicted section of the web may span the entire width of the web.

[0013] The operation to be performed may depend on the identified anomaly. Signaling the operation to be performed may include determining the operation to be performed based on the identified anomaly.

[0014] The term ″identifying an anomaly″ can be understood as detecting any possible signs of an anomaly in the image data and determining whether these signs are anomalies. It should also be noted that even if an anomaly is claimed to have been identified, multiple anomalies may be identified in the image data. In other words, the step of identifying an anomaly may include identifying one or more anomalies.

[0015] The method can be repeatedly executed (i.e., by iteratively repeating the steps of the method) in order to perform continuous quality control on the web of packaging material. Therefore, it should be noted that the image data obtained in each iteration may depict different sections of the web of packaging material. Additionally, each iteration may correspond to a subsequently formed package. In other words, the image data in each iteration may depict sections of the web corresponding to different packages.

[0016] The advantage of the proposed method is that it can detect any potential anomalies at an early stage of the packaging machine by looking at the web of packaging material. This, in turn, can quickly execute error or mitigation operations to reduce downtime or product waste and remove any non - compliant packages. In summary, the inventive concept can provide a more robust and reliable method for anomaly detection, such as web misalignment, packaging material rupture and damage, which anomalies may endanger food safety or the integrity of the production process. This can improve packaging quality, reduce sterility rates, and reduce waste.

[0017] In another example, the proposed method can identify an anomaly, but instead of stopping the machine to process the anomaly, it can let the machine continue to run and process the anomaly later. For example, if it is determined that a certain package should be discarded later, the packaging machine can be instructed not to fill that package with food. This can reduce food waste.

[0018] The image data may depict an edge section of the web. In other words, the edge section may be an outer part of the web of packaging material. Identifying an anomaly may include identifying an anomaly in the edge section of the web of packaging material.

[0019] Damage to the packaging material web most often occurs in the edge sections of the web. Therefore, by obtaining image data depicting the edge sections of the web, the amount of data that the machine learning model needs to process can be reduced while still maintaining a high recognition rate for abnormalities in the packaging material web. Additionally, any abnormalities in the edge sections can have a negative impact on the longitudinal sealing of the packaging. Therefore, identifying such abnormalities early in the process is crucial for ensuring the food safety of the packaging.

[0020] Furthermore, looking at the edge sections of the web may have an additional advantage as it can indicate whether the packaging material web is misaligned in the packaging machine.

[0021] Identifying an abnormality can include: determining data in the image data that indicates an abnormality and classifying the data indicating an abnormality as representing an abnormality or not representing an abnormality.

[0022] Determining the data indicating an abnormality can be performed by image or object segmentation. Classifying the data indicating an abnormality can be performed by a discriminator applied to the image segmentation.

[0023] Classifying the data indicating an abnormality as representing an abnormality can also include determining the type and / or severity of the abnormality. The method can also include determining an action to be performed based on the type and / or severity of the abnormality.

[0024] Depending on the severity of the abnormality or the type of the abnormality, it may be advantageous to take different actions. For example, a severe rupture in the packaging material web may endanger the food safety of the subsequent formed packaging. In such a case, it may be necessary to discard the packaging. If the packaging material web has only a minor dent, the packaging may be discarded for aesthetic reasons or retained even with the dent.

[0025] The action can be any of the following: (i) rejecting or accepting the packaging containing the abnormality; (ii) sending a notification signal to the packaging machine operator; (iii) stopping the packaging machine; and (iv) adjusting the settings of the packaging machine.

[0026] Through reject or accept operations, defective packages (i.e., packages formed by parts of the web of packaging material containing anomalies) can be automatically discarded based on the identified anomalies. For example, if a certain type of anomaly is detected, or the number of identified anomalies increases, a notification signal can be sent to the operator. For example, this may indicate a problem with the handling of the packaging material, and the operator can make corrections. As another example, the identified anomaly may be that the web of packaging material is not properly aligned. Sending a notification to the operator enables them to inspect the packaging machine, identify any potential problems, and correct them as early as possible. If one (or more) packages are detected to have more serious defects, which may result in a large number of defective packages, or if the packaging machine malfunctions, causing the packaging process to stop, the packaging machine can be stopped. If the defects are less serious, the settings of the packaging machine can be adjusted to correct the errors.

[0027] The image data of the section of the web can be associated with the package to be formed. The signal transmitting the operation to be performed can also include transmitting the package identifier of the package associated with the image data. In other words, the package identifier can associate the image data of the edge section of the web with the package to be formed later.

[0028] One possible related advantage is that if the operation is performed at a subsequent stage of the packaging machine, the package identifier can be used to associate the package with the operation to be performed. For example, if the operation is to reject a package, the package identifier can be used to know which package corresponds to the section of the web containing the anomaly.

[0029] The method can also include: collecting the image data as training data and retraining the machine learning model based on the training data. This can facilitate the continuous improvement of the machine learning model, thereby improving the quality control disclosed in the present invention.

[0030] The image data acquisition device can be a linear camera. In other words, the image data acquisition device can be a line scan camera. The image data can include multiple consecutive frames collected by the linear camera, so that a two-dimensional image of the section of the web can be formed. The advantage of using a linear camera is that it provides an efficient method for continuously acquiring the image data of the web.

[0031] The linear camera can be arranged to acquire the image data of the web of packaging material at the unwind position of the roll-fed packaging machine. The advantage of acquiring the image data of the web at the unwind position is that the movement of the web is less, so that higher-quality image data can be obtained. By using a linear camera, the distortion of the image data caused by the shape of the roll can be reduced.

[0032] Image data can depict the non-printed side of the packaging material. For example, the non-printed side can be the side that will form the inside of the package. By looking at the non-printed side of the packaging material, any abnormalities present in the packaging material can be seen more clearly than by using the printed side of the packaging material.

[0033] According to a second aspect, there is provided a quality control device for quality control of packages produced by a web-fed packaging machine, wherein the web-fed packaging machine includes image data acquisition equipment provided at a certain position along a web of packaging material. The control device includes circuitry configured to perform the following functions: an acquisition function configured to acquire image data depicting a section of the web from the image data acquisition equipment; an identification function configured to identify an abnormality in a section of the web of packaging material using a machine learning model trained to identify abnormalities in image data of the web of packaging material; and a communication function configured to transmit a signal indicating an operation to be performed to a control system of the packaging machine in response to the identified abnormality.

[0034] Identifying an abnormality by the identification function can include: determining data indicating an abnormality in the image data and classifying the data indicating an abnormality as representing an abnormality or not representing an abnormality.

[0035] The circuitry can also be configured to perform: a collection function for collecting the image data as training data; and a retraining function for retraining the machine learning model based on the training data.

[0036] The above features of the first aspect are also applicable to the second aspect when applicable. To avoid unnecessary repetition, please refer to the above.

[0037] According to a third aspect, there is provided a food production system. The food production system includes: a web-fed packaging machine configured to produce packages; image data acquisition equipment provided at a certain position along a web of packaging material in the web-fed packaging machine and configured to acquire image data depicting a section of the web of packaging material; and a quality control device for quality control of packages produced in the web-fed packaging machine according to the second aspect.

[0038] The features of the above first and second aspects are also applicable to this third aspect when applicable. To avoid excessive repetition, please refer to the above.

[0039] According to a fourth aspect, there is provided an upgrade kit that is applied to a web-fed packaging machine for performing quality control. The upgrade kit includes: image data acquisition equipment; and a quality control device according to the second aspect.

[0040] The features of the above first, second, and third aspects are also applicable to this fourth aspect when applicable. To avoid excessive repetition, please refer to the above.

[0041] According to a fifth aspect, a non-transitory computer-readable storage medium is provided. It is used to store one or more programs, which are configured to be executed by one or more processors of a processing system, and the one or more programs include instructions for performing the method according to the first aspect.

[0042] As used herein, the term "non-transitory" is intended to describe a computer-readable storage medium (or "memory") that does not include propagated electromagnetic signals, but is not intended to limit the type of physical computer-readable storage devices covered by the term "computer-readable medium" or "memory". For example, a "non-transitory computer-readable medium" or "tangible memory" is intended to cover storage device types that do not necessarily store information permanently, such as random access memory (RAM). Program instructions and data stored in a tangible computer-accessible storage medium in a non-transitory form can also be transmitted via a transmission medium or signal (such as an electrical signal, an electromagnetic signal, or a digital signal), and these signals can be transmitted via a communication medium such as a network and / or a wireless link. Therefore, the term "non-transitory" as used herein is a limitation on the medium itself (i.e., a tangible medium, rather than a signal), rather than a limitation on data storage persistence (such as RAM vs. ROM).

[0043] The above features of the first aspect are also applicable to this fifth aspect when applicable. To avoid excessive repetition, please refer to the above content.

[0044] According to a sixth aspect, a computer program product is provided. The computer program product contains instructions that, when executed by a computer, cause the computer to perform the steps of the method according to the first aspect.

[0045] The above features of the first aspect are also applicable to this sixth aspect when applicable. To avoid excessive repetition, please refer to the above content.

[0046] The machine learning model can be a deep learning model. A deep learning model should be understood as a model based on deep learning. Deep learning should be understood as a type of machine learning in which the hypothesis takes the form of a complex algebraic circuit with adjustable connection strengths. The term "deep" refers to the fact that the circuit is typically organized into multiple layers, which means that the computational path from input to output contains many steps.

[0047] The further scope of application of the present disclosure will be apparent from the following detailed description. However, it should be understood that although these detailed descriptions and specific examples show some variations of the inventive concept, they are given by way of illustration only, since various changes and modifications will be apparent to those skilled in the art within the scope of the inventive concept from these detailed descriptions.

[0048] Accordingly, it should be understood that the inventive concept is not limited to the specific steps of the method or the components of the system, as such methods and systems may vary. It should also be understood that the terms used herein are only for describing specific embodiments and are not intended to limit the present invention. It must be noted that, unless the context clearly dictates otherwise, in the specification and the appended claims, articles such as "a", "an", "the", and "said" are intended to denote the presence of one or more elements. Thus, for example, "a device" or "the device" may include multiple devices, and so on. In addition, the use of words such as "comprising", "including", "containing", etc. does not exclude other elements or steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other aspects of the inventive concept will now be described in more detail with reference to the drawings, which show variations of the inventive concept. These drawings should not be construed as limiting the invention to specific variations; rather, they are used to explain and understand the inventive concept.

[0050] As shown, for ease of illustration, the dimensions of the layers and regions are exaggerated, and thus they are only used to illustrate the general structure of the variations of the inventive concept. The same reference numerals always denote the same elements.

[0051] Figure 1 is a flowchart showing the steps of a method for quality control of packages produced by a reel-fed packaging machine.

[0052] Figure 2 schematically shows a quality control device.

[0053] Figure 3 exemplarily shows a food production system including a quality control device.

[0054] Figures 4A to 4D exemplarily shows different positioning manners of an image data acquisition device relative to a web of packaging material.

[0055] Figure 5 shows an upgrade kit for quality control.

[0056] Figure 6 shows a non-transitory computer-readable storage medium. DETAILED DESCRIPTION

[0057] The inventive concept will be described more fully hereinafter with reference to the drawings, in which some variations of the inventive concept are shown. However, the inventive concept may be embodied in many different forms and should not be construed as limited to the variations described herein; rather, these variations are provided for thoroughness and completeness and to fully convey the scope of the inventive concept to those skilled in the art.

[0058] It should also be understood that when the present disclosure is described in the form of a method, it can also be embodied as an apparatus or device including one or more processors and one or more memories coupled to the one or more processors, with computer code loaded therein to implement the method. For example, the one or more memories may store one or more computer programs, which, when executed by the one or more processors in some embodiments, can execute the steps, services, and functions described herein.

[0059] It should also be understood that the terms used herein are only for describing specific embodiments and are not intended to limit the present invention. It should be noted that, unless the context clearly dictates otherwise, the articles "a", "an", "the", and "said" used in this specification and the appended claims are all intended to mean the presence of one or more elements. Thus, for example, "a unit" or "the unit" may, in some cases, refer to multiple units, and so on. In addition, words such as "comprising" and "including" do not exclude other elements or steps. It should be emphasized that the term "comprising / including" used in this specification is used to specify the presence of the stated features, integers, steps, or components. It does not exclude the presence or addition of one or more other features, integers, steps, components, or combinations thereof. The term "and / or" should be interpreted to mean "both" and "each" respectively. The term "obtain" should be interpreted broadly herein to cover receiving, retrieving, collecting, acquiring, etc.

[0060] It should also be understood that although terms such as "first" and "second" may be used herein to describe various elements or features, these elements should not be limited by these terms. These terms are only used to distinguish the elements. For example, without departing from the scope of the embodiment, the first function may be referred to as the second function, and similarly, the second function may also be referred to as the first function. The first function and the second function are both functions, but they are not the same function.

[0061] Now reference will be made to Figures 1 to 6 Describe a method for quality control of packages produced by a reel-fed wrapping machine, as well as a quality control device, a food production system, an upgrade kit, and a non-transitory computer-readable storage medium.

[0062] Figure 1 is a flowchart showing the steps of a method 100 for quality control of packages produced by a reel-fed wrapping machine. In a reel-fed wrapping machine, such as Tetra Brik sold by Tetra Pak TM ) TMSystem, the packaging material web is usually fed from a roll of packaging material. Then, the packaging material web can form a tube and the tube can be filled with food. From the lower end of the tube, packages can be formed continuously. According to the present invention, the roll-fed packaging machine includes an image data acquisition device. The image data acquisition device is arranged at a certain position along the packaging material web in the packaging machine. Thus, before the tube is formed, the image data acquisition device can be positioned at any position along the packaging material web. In combination with Figures 4A to 4D The positioning of the image data acquisition device is further discussed. The packages formed by the roll-fed packaging machine can be cardboard packages containing food. More specifically, the package can include a cardboard layer and at least one plastic layer.

[0063] Figure 1 An example of method 100 is shown. In addition, multiple alternative steps (forming multiple alternative variants) of method 100 are shown in dashed lines. It should be noted that method 100 can be iteratively executed at multiple consecutive time points. In other words, method 100 can be repeatedly executed for continuous quality control of the packages produced by the roll-fed packaging machine. In each iteration, the acquired image data (discussed further below) depicts different sections of the packaging material web.

[0064] The following will refer to Figure 1 Each step will be described in more detail. Although the steps of method 100 are shown in a specific order in the figure, the steps of method 100 can be executed in any suitable order, can be executed in parallel, or can be executed multiple times. For example, steps S116 and S118 (discussed further below) can be executed at any time point after step S102 and are executed independently of other steps.

[0065] Obtain S102 image data depicting a section of the web from the image data acquisition device. Optionally, obtaining S102 image data includes instructing the image data acquisition device to acquire the image data. It should be noted that the image data does not have to be directly obtained from the image data acquisition device. For example, the image data acquisition device can acquire the image data and then transmit it to an intermediate memory, and any device executing method 100 can obtain the image data from the intermediate memory.

[0066] The image data can depict a section of the packaging material web relevant to the quality control to be performed. In one example, the image data depicts an edge section of the packaging material web. As Figure 4CAs shown, the edge segment can be defined as the outer part extending a distance d from the edge of the web into the web. Without doubt, the image data can depict the edge segment on either side of the web. The image data can depict the edge segments on both sides of the web. In another example, the image data depicts the inner segment of the web. The inner segment can be any part of the web located between the two outer parts. In another example, the image data depicts the entire width of the web of packaging material.

[0067] The image data can comprise a single frame of the web of packaging material. Alternatively, the image data can comprise a series of frames of the web. In other words, the image data can comprise multiple consecutive frames. Multiple subsequent frames can be combined together to form a single image, which can then be used in subsequent steps of method 100. The image data acquisition device can be a line camera. A line camera typically acquires a line of pixels in each time frame. Thus, in this example, the image data comprises multiple frames that are combined together to form a single two-dimensional image. Acquiring a line of pixels in each frame using a line camera can allow different frames of the image data to be acquired at intervals independent of the web speed. In other words, regardless of the web speed, consecutive frames of the image data can be acquired at a predetermined frame rate. This can be beneficial in making the method more robust to changes in web speed. The line camera can be arranged at the unwind position in a web-fed packaging machine to acquire image data of the web of packaging material. The positioning of the camera will be discussed further below in conjunction with Figure 4A further discussion.

[0068] The image data can depict the non-printed side of the packaging material. In other words, the image acquisition device can be arranged at a position visible from within the packaging material.

[0069] The image data acquisition device can be a camera configured to acquire image data of the web. The camera can be, for example, a line camera as described above, or a high-speed camera. The image acquisition device can include multiple cameras. Each camera can be configured to acquire images of different parts of the web of packaging material. In this case, the acquired image data can include multiple images of the web acquired by the multiple cameras. The multiple images can be combined to form a single image, which can then be used in subsequent steps of method 100.

[0070] Next, using a machine learning model trained to identify anomalies in the image data of the packaging material web, anomalies in the section of the S104 packaging material web are identified. In other words, anomalies can be identified in the image data depicting the section of the web. In other words, the image data can be input into the machine learning model. Before inputting the image data into the machine learning model, the image data can be preprocessed. The preprocessing of the image data can include, but is not limited to, shaping, normalizing, and / or cropping of the image data. Then, the machine learning model can output information indicating whether any anomalies exist in the packaging material web depicted in the image data. The identified anomalies of S104 can include one or more of the following: anomalies in the edge section of the packaging material web, joints present in the web, and / or anomalies related to the joints. In particular, a joint is defined as the connecting part between two different webs, for example, joined together by sealing.

[0071] The machine learning model can include one or more sub - networks. The machine learning model can include a convolutional neural network. The machine learning model can output a numerical score or a label indicating whether an anomaly exists. The machine learning model can output the type of the identified anomaly. In another example, the machine learning model can output an operation to be performed in response to the identified anomaly. For example, the machine learning model can output a label indicating whether the subsequent formed packaging should be discarded. The output of the machine learning model can also indicate the number of anomalies, the location of the anomalies, and the size and shape of the anomalies. The output of the machine learning model may depend on the type of the evaluated packaging or packaging material. For example, the evaluation method of sterile packaging may be different from that of non - sterile packaging. It goes without saying that anomalies are only identified when there are any anomalies in the image data. If method 100 is executed iteratively, anomalies may only be identified in some instances of the image data.

[0072] The machine learning model can be trained using supervised learning, that is, trained using image data and associated label pairs.

[0073] If the image data depicts the edge section of the web, identifying the S104 anomaly can include identifying the anomaly in the edge section of the packaging material web.

[0074] Identifying the S104 anomaly can include determining S106 the data indicating the anomaly in the image data. Identifying the S104 anomaly can also include classifying S108 the data indicating the anomaly as representing an anomaly or not representing an anomaly.

[0075] It is determined that the data indicating an anomaly in S106 can be performed through an image segmentation task. Thus, the data indicating an anomaly can be a segmented image. The image segmentation task can be semantic segmentation or instance segmentation. The image segmentation task can be performed using an image segmentation network. The image segmentation network can be part of a machine learning model. For example, the image segmentation network can be an autoencoder, such as a variational autoencoder, a denoising autoencoder, or a convolutional autoencoder. As another example, the image segmentation network can be a so-called U-Net. In this case, the U-Net can first encode the image data into a low-dimensional latent representation. Then, the U-Net can decode the latent representation back into a mask of the image data. The mask can be a binary mask, i.e., containing black or white pixels. For example, white pixels can represent a faulty part of the coil (i.e., indicating an anomaly), while black pixels can represent a healthy part of the coil. The image segmentation network can include multiple sub-segmentation networks trained for different sub-tasks.

[0076] Classifying the data indicating an anomaly in S108 as representative or non-representative of an anomaly can be performed by a discriminator network. In other words, the discriminator network can be a classification network. The discriminator network can be part of a machine learning model. The discriminator network can be configured to determine whether the data indicating an anomaly actually represents an anomaly. For example, the discriminator can look at the size and shape of a potential anomaly to determine whether it should be classified as an anomaly. In a more specific example, the discriminator can look at the number of pixels forming a potential anomaly along the coil feed direction. As mentioned above, the discriminator can look at the white and black pixels in the image data mask output by the U-Net. For example, the discriminator can look at the size or shape of the white pixel area in the segmented image.

[0077] Next, in response to the identified anomaly, a signal indicating the operation to be performed will be transmitted in S112 to the control system of the packaging machine.

[0078] The operation to be performed can be determined in S110 based on the identified anomaly. This can be done as a separate step (as shown herein), or as part of the step of identifying the anomaly in S104, or as part of the step of transmitting the signal indicating the operation to be performed in S112 in response to the identified anomaly. Classifying the data indicating an anomaly in S108 as representing or not representing an anomaly can also include determining the type and / or severity of the anomaly. The operation to be performed can also be determined in S110 based on the type and / or severity of the anomaly. The operation can be any of the following: (i) reject or accept the package containing the anomaly, (ii) send a notification signal to the packaging machine operator, (iii) stop the packaging machine and / or (iv) adjust the settings of the packaging machine. It should be noted that more than one operation to be performed can be determined. For example, a certain package can be rejected and the settings of the packaging machine can be adjusted.

[0079] Image data depicting a section of a web can be associated with a package to be formed. The image data can be associated with the package using a package identifier. A signal S112 indicating an operation to be performed can also include transmitting S114 the package identifier of the package associated with the image data. A control system or any other device, or an operator or user of the packaging machine, can use the package identifier to know which package may contain an identified anomaly and / or on which package an operation is to be performed. The package identifier can be a unique identifier of the package. The package identifier can be a number or a printed code (such as a two-dimensional bar code or a QR code) printed on the packaging material. The package identifier can be displayed in and extracted from the image data. Alternatively, the package identifier can be read by a scanner. Alternatively, the package identifier can be set in a label (such as an RFID or NFC label) and acquired by an RFID or NFC label reader. The package identifier can also be used to associate data related to the machine settings used when producing the package. The package identifier can also be used to associate the operating conditions during the period when an operator loads the web of packaging material into the packaging machine. The data associated with the package described above can be used to find deficiencies in the operating characteristics or machine settings, and these deficiencies can be used as feedback to further improve the quality of the produced package.

[0080] Method 100 may further include collecting S116 the image data as training data. The step of collecting S116 the image data may include transmitting the image data to a group of users and receiving, in response, user input indicating whether the image data shows an anomaly. The user input can be included in the training data as a label for the image data. The group of users can include packaging machine operators or experts who evaluate the image data. Method 100 may further include retraining S118 the machine learning model based on the training data.

[0081] Figure 2 is a schematic diagram of a quality control device 200 according to the inventive concept. The quality control device 200 is configured to perform quality control on packages produced by a web-fed packaging machine. In particular, the quality control device 200 is configured to execute method 100 described above in connection with Figure 1 described. Any features, aspects, or advantages described above in connection with method 100 also apply to the quality control device 200 described below, and vice versa. For the sake of avoiding repetition, reference is made to the above.

[0082] The quality control device 200 can be a component of the web-fed packaging machine. More specifically, the quality control device 200 can be part of the control system of the web-fed packaging machine. Alternatively, the quality control device 200 can be provided outside the web-fed packaging machine. In this case, the quality control device 200 can be communicatively connected to the web-fed packaging machine.

[0083] The quality control device 200 includes a circuit 202. The circuit 202 may physically consist of a single circuit device. Alternatively, the circuit 202 may be distributed across multiple circuit devices. As Figure 2 shown in the example of, the quality control device 200 may further include a transceiver 206 and a memory 208. The circuit 202 is communicatively connected to the transceiver 206 and the memory 208. The circuit 202 may include a data bus, and the circuit 202 may communicate with the transceiver 206 and / or the memory 208 via the data bus.

[0084] The circuit 202 may be configured to comprehensively control the functions and operations of the quality control device 200. The circuit 202 may include a processor 204, such as a central processing unit (CPU), a microcontroller, or a microprocessor. The processor 204 may be configured to execute program code stored in the memory 208 to perform the functions and operations of the quality control device 200.

[0085] The transceiver 206 may be configured to enable the quality control device 200 to communicate with other devices or apparatuses. For example, the quality control device 200 may communicate with the control system of a reel packaging machine via the transceiver 206. In another example, the quality control device 200 may be communicatively connected to an image data acquisition device (e.g., via the transceiver 206). The transceiver 206 can both transmit data to and receive data from the quality control device 200. The transceiver 206 may communicate via a wired or wireless communication protocol (such as Bluetooth, Wi-Fi, cellular communication, etc.).

[0086] The memory 208 may be a non-transitory computer-readable storage medium. The memory 208 may be one or more of a buffer, a flash memory, a hard disk, a removable medium, a volatile memory, a non-volatile memory, a random access memory (RAM), or other suitable devices. In a typical configuration, the memory 208 may include non-volatile memory for long-term data storage and volatile memory serving as the system memory of the quality control device 200. The memory 208 may exchange data with the circuit 202 via the data bus. There may also be accompanying control lines and address buses between the memory 208 and the circuit 202.

[0087] Although Figure 2 not explicitly stated in, the quality control device 200 may include input devices, such as one or more of a keyboard, a mouse, and a touch screen. For example, user input may be used to collect image data as training data, as will be further described below. The user may provide user input as a label for the image data. Thus, the user input may serve as feedback to the method or the quality control device to improve the performance of the machine learning model. The quality control device 200 may also include a display for providing output to the user, such as image data.

[0088] The functions and operations of the quality control device 200 can be implemented in the form of executable logic routines (e.g., lines of code, software programs, etc.), which are stored on a non-transitory computer-readable recording medium (e.g., memory 208) of the quality control device 200 and executed by the circuit 202 (e.g., using the processor 204). In other words, when the circuit 202 is configured to perform a specific operation or perform a specific function, the processor 204 of the circuit 202 can be configured to execute a program code portion stored on the memory 208, where the stored program code portion corresponds to the specific operation or function. In addition, the functions and operations of the circuit 202 can be an independent software application or a part of a software application that performs additional tasks related to the circuit 202. The described functions and operations can be regarded as methods executed by the corresponding device configuration, such as the above in combination with Figure 1 Method 100 discussed. In addition, although the functions and operations described may be implemented in software, these functions may also be performed by dedicated hardware or firmware, or a combination of one or more of hardware, firmware and software. The following operations may be performed by the quality control device 200 and may be stored as functions on a non-transitory computer-readable recording medium. Acquisition function 210, identification function 212 and communication function 214. The circuit 202 may also be configured to perform one or more of collection function 216, retraining function 218 and / or determination function 220. It should be noted that the distribution of functions of the quality control device 200 described herein should be considered as non-limiting examples, as they may be implemented in any suitable manner.

[0089] The acquisition function 210 is configured to acquire image data depicting a section of the web from an image data acquisition device. Figure 1 The image data acquisition device is arranged at a certain position along the packaging material roll. The image data acquisition device can be a linear camera. The linear camera can be arranged at the position of the roll in the roll packaging machine to collect image data of the packaging material roll. The image data can depict the non-printing side of the packaging material.

[0090] The recognition function 212 is configured to use a machine learning model trained to recognize anomalies in the packaging material web image data to recognize anomalies in the section of the packaging material web. Identifying the anomaly by the recognition function 212 may include determining data indicating the anomaly in the image data and classifying the data indicating the anomaly as being indicative of an anomaly or not indicative of an anomaly. Classifying the data indicating the anomaly as being indicative of an anomaly or not indicative of an anomaly may also include determining the type and / or severity of the anomaly.

[0091] The image data may depict an edge section of the packaging material web.The identification function 212 may be configured to identify anomalies in an edge section of the packaging material web.

[0092] The communication function 214 is configured to transmit a signal indicating an operation to be performed to the control system of the wrapping machine in response to the identified anomaly.

[0093] The operation can be any one of the following: (i) rejecting or accepting the package containing the anomaly; (ii) sending a notification signal to the wrapping machine operator; (iii) stopping the wrapping machine; and (iv) adjusting the settings of the wrapping machine.

[0094] The image data of the section of the web can be associated with the package to be formed. Transmitting a signal indicating an operation to be performed via the communication function 214 may also include transmitting a package identifier of the package associated with the image data.

[0095] The determination function 220 can be configured to determine the operation to be performed based on the identified anomaly. The determination function 220 can be configured to determine the operation to be performed based on the type and / or severity of the anomaly.

[0096] The collection function 216 can be configured to collect image data as training data.

[0097] The retraining function 218 can be configured to retrain the machine learning model based on the training data.

[0098] Figure 3 The food production system 300 according to the inventive concept is schematically shown by way of example. The dashed lines indicate optional components. The food production system 300 can be regarded as any type of food processing production line for filling and packaging food.

[0099] The food production system 300 includes a web wrapping machine 304, also referred to as a wrapping machine or a filling machine. The wrapping machine 304 is a web wrapping machine for producing food packages. More specifically, the wrapping machine 304 can be used to package liquid food in cardboard-based packages. As early as the 1940s, Tetra Pak launched this type of wrapping machine, which has now become a well-known method for safely and cost-effectively packaging milk and other liquid foods. This general method can also be used for non-liquid foods, such as potato chips. In addition, this general method is also applicable to non-cardboard-based packages.

[0100] The packaging material is usually printed and prepared at a packaging material production center (also referred to as a processing plant), and then transported to the location where the wrapping machine 304 is placed, such as a dairy. Usually, the packaging material is loaded onto the reel 302 before transportation. After arriving at the site, the reel 302 is placed in the wrapping machine 304, or fed into the wrapping machine 304 as shown herein.

[0101] During the production process, the web 402 of the packaging material can be fed from the reel 302 into the wrapping machine 304 and pass through the wrapping machine. AlthoughFigure 3 This is not shown in the figure, but the packaging material can pass through a sterilization device (such as a hydrogen peroxide bath or an LVEB (low voltage electron beam) station) to ensure that the web 303 is free of harmful microorganisms. Before providing the food, the web 402 can be formed into a tube by longitudinal sealing. The food can be fed into the tube through a filling tube (not shown in the figure), and a valve (not shown in the figure) can be used to regulate the flow rate through the filling tube. The lower end of the tube can be fed into a folding device (not shown in the figure), where transverse sealing is performed, and the tube is folded according to a folding line (also known as a weakened line) and then cut, thereby forming the package 308.

[0102] The packaging machine 304 includes a control system 305. The control system 305 can be configured to comprehensively control the functions and operations of the packaging machine 304.

[0103] The food production system 300 further includes an image data acquisition device 306. The image data acquisition device 306 is disposed at a certain position on the web of packaging material 402. The image data acquisition device 306 can be disposed at any position between the web 302 and the forming device for forming the tube for the food. The image data acquisition device 306 is used to acquire image data depicting a section of the web of packaging material 402. The image data acquisition device 306 can be disposed inside the packaging machine 304. However, it is readily understood by those skilled in the art that the image data acquisition device 306 can also be disposed outside the packaging machine 304. The food production system 300 can include one or more image data acquisition devices, which are disposed at different positions on the web of packaging material 402. The image data acquisition device 306 can be any camera or other image acquisition device suitable for generating image data for identifying abnormalities in the web of packaging material 402. The image data acquisition device 306 can include a light source, which is used to irradiate the part of the package for which the image data is acquired. The light source can use visible light, infrared (IR) light, ultraviolet (UV) light, or any combination thereof. The light source can provide visible light with a wavelength of approximately 380 to 740 nanometers (nm). The light source can provide infrared light with a wavelength of approximately 740 nm to 1 mm. The light source can provide ultraviolet light with a wavelength of approximately 1 to 380 nm.

[0104] The food production system 300 further includes a quality control device 200 for performing quality control on the packages produced in the web-fed packaging machine 304. The quality control device 200 is as described above in connection with Figure 2The quality control device 200 as described. As shown herein, the quality control device 200 can be provided as an integrated device of the packaging machine 304. Herein, the quality control device 200 is shown to be communicatively connected to the control system 305 of the packaging machine 304. The quality control device 200 can be integrated within the control system 305 of the packaging machine 304. Alternatively, the quality control device 200 can be provided as a device independent of the packaging machine 304 and communicatively connected thereto. For example, the quality control device 200 can be provided as a remote server, such as implemented in the cloud 322. The quality control device 200 is communicatively connected to the image data acquisition device 306. Although the image data acquisition device 306 shown in the figure is separated from the quality control device 200, the image data acquisition device 306 can also be a part of the quality control device 200.

[0105] As described above in connection with Figure 1 and Figure 2 the quality control device 200 can acquire image data depicting a section of the packaging material web 402. Here, the quality control device 200 receives the image data from the image data acquisition device 306 via any suitable communication protocol. Optionally, the packaging identifier (ID) of the package associated with the image data can also be received. Alternatively, the quality control device 200 can acquire the image data from the control system 305 of the packaging machine 304, and the control system 305 in turn receives the image data from the image data acquisition device 404. In return, the quality control device 200 transmits a signal indicating the operation to be performed to the control system 305 of the packaging machine 304. Optionally, the quality control device 200 transmits updated settings to the packaging machine 304.

[0106] The food production system 300 can further include a packaging rejection unit 310. The packaging rejection unit 310 can be configured to retain or discard packages based on the received signal. The quality control device 200 can be communicatively connected to the packaging rejection unit 310. The quality control device 200 can transmit the operation of rejecting or accepting a package to the packaging rejection unit 310. The packaging identifier (ID) of the package associated with the operation can also be transmitted. According to the operation, the packaging rejection unit 310 can discard or retain the package. The discarded packages 312 can be collected by the recycling unit 314 to recycle the products in the discarded packages. The accepted packages 316 can be further sent to the food production system 300.

[0107] The food production system 300 can further include a quality control station 318. The quality control station 318 can serve as a secondary quality control system of the food production system 300 for performing other types of quality control on the packages 308 in addition to the quality control device 200 described herein.

[0108] The quality control station 318 may be configured to receive the accepted packages 316 for further evaluation. Although not shown, the quality control station 318 may also be configured to receive the rejected packages 312 for further evaluation. Here, the quality control station 318 is shown as being located downstream of the package rejection unit 310. However, the quality control station 318 may also be located upstream of the package rejection unit 310 for directly evaluating the packages 308 produced by the packaging machine 304. The quality control station 318 may be part of the quality control device 200 for providing feedback to the machine learning model.

[0109] The quality control station 318 may include a user interface 324 (e.g., an operator panel) configured to display image data collected by the image data acquisition device 306 for viewing by a user 320 (e.g., a machine operator). Specifically, the quality control station 318 may allow the user to determine whether any anomalies were correctly identified (if any anomalies exist) or were incorrectly identified (if no anomalies exist). This information (i.e., whether the package is a false negative FN (or true negative TN) or a false positive FP (or true positive TP)) may be sent to the quality control device 200. It goes without saying that if the package is determined to be FN or FP, this may also provide information about TN and TP. The quality control device 200 may use information about false negatives (optionally, true negatives) and / or false positives (optionally, true positives) to retrain the machine learning model. Feedback from the user 320 may also be used to mark any image data collected for retraining the machine learning model.

[0110] It should be noted that one or more parts of the food production system 300 can be implemented in the cloud 322. For example, the machine learning model of the quality control device 200 can be deployed in the cloud 322. Therefore, the same machine learning model can be used for multiple food production systems 300. As another example, any data related to the training of the machine learning model (e.g., image data describing a new type of anomaly) can be transmitted to the cloud 322 and distributed to other food production systems 300 so as to retrain the machine learning models of other food production systems 300.

[0111] Figures 4A to 4D Different positioning modes of the image data acquisition devices 404, 404' relative to the packaging material web 402 are illustrated. Figures 1 to 3 As described above, the image data acquisition device 404 is arranged at a certain position along the packaging material roll 402. It should be noted that, Figures 4A to 4D They should only be regarded as non-limiting examples and for ease of explanation, for example, the sizes and shapes of layers and regions are exaggerated and are therefore only used to illustrate the general structure and concepts rather than a real scene.

[0112] Figure 4 shows a part of the packaging material web 402 within a web-fed packaging machine in a cross-sectional side view (along the x-direction). The feed direction of the web 402 is indicated by the thick arrow. The packaging material web 402 has a first side 408a and a second side 408b. Here, the image data acquisition device 404 is arranged to acquire an image depicting a section of the first side 408a of the web 402. The first side 408a can be the non-printed side of the packaging material. The non-printed side can be the inner side of the subsequently formed package.

[0113] As described above, the image data acquisition device 404 can be a line camera, as shown herein. The field of view of the image data acquisition device 404 is indicated by the dashed line. Since in this example the line camera is shown in a side view, the field of view is indicated by a single line.

[0114] Advantageously, the image data acquisition device 404 can be arranged near the web of the web-fed packaging machine. The vibration or other movement of the web 402 is reduced. In this example, the image data acquisition device 404 is provided at a position on the web 406 of the packaging machine in contact with the packaging material web 402.

[0115] Figure 4B A cross-sectional view along the feed direction of the web 402 (i.e., the y-direction) shows a part of the packaging material web 402. In the currently shown example, the image data acquisition device 404 is arranged to acquire an image of a section of the web 402 that covers the entire width of the web 402. In other words, the field of view of the image data acquisition device 404 in the transverse direction of the web 402 (i.e., the x-direction) can cover the entire first side 408a of the web 402.

[0116] Figure 4C A part of the packaging material web 402 is shown from a top-down (i.e., z-direction) perspective. Figure 4D Shows the same example as Figure 4C but shows a cross-sectional view along the feed direction of the web 402 (i.e., the y-direction). The web 402 also shows a plurality of anomalies 412a, 412b, 412c, which can be identified using the currently disclosed techniques. These anomalies may represent, for example, breaks 412a, 412c and creases 412b in the packaging material. However, as described above, other anomalies of the packaging material web 402 can also be identified.

[0117] Here, the image data acquisition device 404 is arranged at the position of the edge section 410a (also referred to as the first edge section) of the web 402. Thus, the image data acquisition device 404 can be configured to acquire image data depicting the edge section 410a of the web 402. The edge section 410a can be formed by an outer portion of the web 402, for example, within a range of a distance d from the outermost edge 414 of the web 402. It will be readily understood by those skilled in the art that an additional edge section 410a' (also referred to as the second edge section) (opposite to the edge section 410a) of the web 402 can also be part of the quality control. Thus, an additional image data acquisition device 404' can be provided, which is configured to acquire image data depicting the additional edge section 410a' of the web. The additional edge section 410a' can be at a distance d' from the additional edge 414' of the web 402. The distance d' can be the same as or different from the distance d.

[0118] An abnormality of the packaging material web 402 may also occur in the inner section 410b of the web 402. Although Figure 4C not shown, an image data acquisition device can be provided at a position capable of capturing the image data of the inner section 410b.

[0119] Figure 5 An upgrade kit 500 is schematically shown. The upgrade kit 500 can be applied to a web-fed packaging machine, such as the web-fed packaging machine 304 described above in connection with Figure 3 . The upgrade kit 500 is configured to be capable of performing quality control on the packages produced in the web-fed packaging machine 304. The upgrade kit 500 includes an image data acquisition device 306, for example, the image data acquisition device as described above in connection with Figure 3 . The upgrade kit 500 further includes a quality control device, for example, the quality control device 200 as described above in connection with Figure 2 .

[0120] Figure 6 A non-transitory computer-readable storage medium 600 is shown. The non-transitory computer-readable storage medium 600 stores one or more programs, which are configured to be executed by one or more processors of a processing system, and the one or more programs include instructions for performing the method 100 as described above in connection with Figure 1 .

[0121] Although not shown, the method 100 can also be implemented in a computer program product. The computer program product contains instructions that, when executed by a computer, cause the computer to perform the steps of the method 100 (refer to Figure 1 ).

[0122] In addition, those skilled in the art can understand and implement the variations of the disclosed variations by studying the drawings, the disclosure, and the appended claims when practicing the claimed invention.

Claims

1. A method (100) for quality control of packages produced by a reel-fed wrapping machine, wherein, The drum-type packaging machine includes an image data acquisition device provided at a certain position along the packaging material web, and the method (100) includes: obtaining (S102) image data depicting a section of the web from the image data acquisition device; using a machine learning model trained to identify anomalies in the image data of the packaging material web, identifying (S104) anomalies in the section of the packaging material web; and in response to the identified anomalies, transmitting (S112) a signal indicating an operation to be performed to a control system of the packaging machine.

2. The method (100) according to claim 1, wherein: - the image data depicts an edge section of the web, and wherein identifying (S104) the anomalies includes identifying anomalies in the edge section of the packaging material web, and / or - identifying (S104) the anomalies includes identifying the presence of joints and / or anomalies associated with the joints.

3. The method (100) according to claim 1 or 2, wherein identifying (S104) the anomalies includes: determining (S106) data indicating the anomalies in the image data, and classifying (S108) the data indicating the anomalies as representing an anomaly or not representing an anomaly.

4. The method (100) according to claim 3, wherein, Classifying (S108) the data indicating an anomaly as representing an anomaly further includes determining the type and / or severity of the anomaly, and wherein the method (100) further includes determining (S110) the operation to be performed based on the type and / or severity of the anomaly.

5. The method (100) according to any one of claims 1 to 4, wherein, The operation is any one of the following operations: (i) rejecting or accepting a package containing the anomaly, (ii) sending a notification signal to an operator of the packaging machine, (iii) stopping the packaging machine, and (iv) adjusting the settings of the packaging machine.

6. The method (100) according to any one of claims 1 to 5, wherein, The image data of the section of the web is associated with a package to be formed, and wherein transmitting (S112) the signal indicating the operation to be performed further includes transmitting (S114) a package identifier of the package associated with the image data.

7. The method (100) according to any one of claims 1 to 6, further comprising: collecting (S116) the image data as training data, and retraining (S118) the machine learning model based on the training data.

8. The method (100) according to any one of claims 1 to 7, wherein, The image data acquisition device is a line camera.

9. The method (100) according to claim 8, wherein, The line camera is arranged to acquire image data of the packaging material web at a web position in the drum-type packaging machine.

10. The method (100) according to any one of claims 1 to 9, wherein, The image data depicts the non-printed side of the packaging material.

11. A quality control device (200) for quality control of packages produced in a web-fed wrapping machine, wherein, The drum-type packaging machine includes an image data acquisition device provided at a certain position along the packaging material web, and the control device (200) includes a circuit (202), the circuit being configured to execute: an acquisition function (S210) configured to obtain image data depicting a section of the web from the image data acquisition device; An identification function (212) configured to identify an anomaly in the section of the packaging material web using a machine learning model trained to identify anomalies in image data of the packaging material web; and A communication function (214) configured to transmit a signal indicating an operation to be performed to a control system of the packaging machine in response to the identified anomaly.

12. The quality control device (200) according to claim 11, wherein, Identifying the anomaly by the identification function (212) includes: Determining data in the image data indicative of the anomaly, and Classifying the data indicative of the anomaly as representing an anomaly or not representing an anomaly.

13. A food production system (300) comprising: A web-fed packaging machine (304) configured to produce packages (308); An image data acquisition device (306) disposed at a position along a packaging material web (402) in the web-fed packaging machine and configured to acquire image data depicting sections (406a, 406b) of the packaging material web (402); and A quality control device (200) for quality control of packages produced in the web-fed packaging machine according to claim 11 or 12.

14. An upgrade kit (500) for use in a web-fed packaging machine (304) for quality control, the upgrade kit (500) comprising: An image data acquisition device (306); and The quality control device (200) according to claim 11 or 12.

15. A non-transitory computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a processing system, the one or more programs including instructions for performing the method (100) according to any one of claims 1 to 10.

Images