Method and system for quality control of processed food products

Abstract

In a first aspect, the invention relates to a method for quality control of processed food in a production line with a camera, comprising steps such as obtaining a digital base model of a food product, generating a first dataset with variations on this base model, and a second dataset with digital models of defects. These are combined into a third dataset of food products with defects. Furthermore, the method comprises simulating a production environment in which the food models with defects are placed, generating two-dimensional images of the food products in this production environment, training an Al model with these images to recognize defects, and applying the Al model for identifying defective food products in a production line. In a second aspect, the invention comprises an apparatus for quality control with a camera, a computer for image processing, and a graphical user interface. The computer contains a trained Al model that can identify defects in the production line. The invention offers an efficient, cost-saving method to train and deploy an Al model for defect recognition, which ensures accurate quality control without affecting production speed. Thanks to automatic data analysis and flexibility in product changes, the Al model can be quickly adapted to new requirements in the production environment, whereby food safety and consistency remain guaranteed.

Description

[0001] METHOD AND SYSTEM FOR QUALITY CONTROL OF PROCESSED FOOD PRODUCTS

[0002] TECHNICAL FIELD

[0003] The invention relates to a method for the quality control of processed food in a production line with a camera.

[0004] PRIOR ART

[0005] EP4060612 describes a method for generating robot commands for handling physical objects. The method comprises obtaining two images of an object from a set of cameras, such as hyperspectral cameras, which are positioned at different respective angles relative to the object. A voxel representation is generated which is segmented based on the two images and which contains a three-dimensional surface of the object, wherein the segmentation is performed by a segmentation Al model. A main direction is manually determined based on the segmented representation. Based on the determined main direction of the object relative to a reference volume, a robot command is calculated for handling the object. The command is executed by a device with a robot element.

[0006] US8477154 describes a Graphical User Interface (GUI) method for interactive virtual inspection of modeled objects. The method comprises obtaining a three-dimensional model of a modeled object and displaying a primary view of the modeled object, so that the user can identify a point of interest on the surface of the modeled object. A marker tag with user-inputted information is created which annotates the point of interest, and the marker tag is automatically linked to the point of interest on the modeled object. A secondary view of the modeled object is displayed, including the user-identified location of the point of interest and the marker tag.

[0007] W02021105504A1 describes a method for generating a robot command to handle a three-dimensional physical object, such as industrial products. This is done by obtaining images of the object from multiple cameras at different angles. Based on two images, a segmented voxel representation of the 3D surface is generated, after which a 3D reconstruction and segmentation take place via a 3D neural network trained with manual annotations. Subsequently, a robot command is calculated for handling the object, which is executed by a robot element. Traditional methods for annotating data cost a lot of time and are therefore unsuitable for annotating data for the quality control of processed food. The present invention aims to find a solution to this problem.

[0008] SUMMARY OF THE INVENTION

[0009] In a first aspect, the invention relates to a method for quality control of processed food in a production line with a camera, the method comprising the steps of: obtaining a digital base model of a food product; obtaining a first dataset comprising digital models of food products by applying variations to the digital base model; obtaining a second dataset comprising different generated digital models of defects; obtaining a third dataset comprising digital models of food products with defects by combining the first and the second dataset; simulating a production environment, and combining the simulated production environment with at least one digital model of a food product; generating two-dimensional images of one or more digital models of food products with defects in a production environment; training an Al model with the two-dimensional images to recognize food products with defects; obtaining image material of the processed food in the production line; identifying food products with defects by analyzing the image material with the trained Al model; wherein the first dataset is obtained by, using algorithms, varying the dimensions of the digital base model and applying surface characteristics to the digital base model; and wherein during the obtaining of the second dataset different types of defects are generated; and wherein during the obtaining of the third dataset the type of defect and the location of the defect on the digital model are varied per digital model.

[0010] In a second aspect, the invention relates to an apparatus for quality control of processed food in a production line with a camera, the apparatus comprising: at least one camera directed at the processed food in the production line; a computer configured to receive images from the at least one camera, and to identify defective food products based on the received images; a graphical user interface configured to display data based on the identified defective food products; and wherein the computer comprises a memory with an Al model trained with generated labeled data of food products with and without defects in a simulated production environment, and wherein the computer is configured to identify defective food products using the Al model.

[0011] Quality control of processed food products in a production line is essential to ensure food safety, consistency, and customer satisfaction. During processing, products may sustain damage, contaminations, or deviations in texture, shape, or color. Strict controls help to identify and remove defective products in a timely manner, which minimizes health risks for consumers and limits waste. Because in the food production industry large volumes of food products must move quickly through the production line, manual quality control is practically unfeasible and would significantly slow down the production speed. By deploying an Al model to analyze image material of the products, defects can be recognized faster and more accurately. The present invention offers an efficient way to train and deploy an Al model for identifying defective food products, by automatically generating annotated data of defective products. As a result, costs are saved and quality control is improved. Moreover, this method makes it possible to create a larger variation and quantity of defect models, so that the Al model learns to recognize a broader range of defects and the accuracy of the quality control increases. The invention also offers flexibility, because new datasets can be easily generated when the production line or the types of food products change, whereby the Al model can be quickly and effectively adapted to new requirements in the production environment.

[0012] DETAILED DESCRIPTION

[0013] Unless otherwise defined, all terms used in the description of the invention, including technical and scientific terms, have the meaning as commonly understood by a person skilled in the art to which the invention pertains. For a better assessment of the description of the invention, the following terms are explicitly explained.

[0014] As used in this document, the articles "a", "an" and "the" refer to both the singular and the plural, unless the context clearly indicates otherwise. For example, "a segment" means one or more than one segment.

[0015] The terms "comprise", "comprising", "consist of", "consisting of", "provided with", "contain", "containing", "encompass", "encompassing", "include" and "including" are used herein as synonyms and are intended as inclusive or open terms indicating the presence of what follows, without excluding or preventing the presence of other components, characteristics, elements, members or steps known from or disclosed in the prior art.

[0016] By "digital base model" is meant a first digital representation of a food product.

[0017] By the term "digital models" is meant digital representations of food products which are obtained by applying variations to the digital base model.

[0018] In a first aspect, the invention relates to a method for quality control of processed food in a production line with a camera according to claim 1. Quality control of processed food products in a production line is essential for ensuring food safety, consistency, and customer satisfaction. During processing, products may become damaged, become contaminated, or exhibit deviations in texture, shape, or color. Strict quality controls make it possible to detect and remove defective or deviant products early, which helps to minimize health risks for consumers and reduce waste.

[0019] In the food production industry, quality control constitutes a major challenge, because food products must be transported in high volumes and at high speed to achieve the required efficiency. This makes it practically impossible to check quality manually, since this would significantly slow down the production speed. By deploying an Al model for analyzing image material of food products, defects can be identified faster and more accurately. To ensure that the Al model recognizes defective products correctly, it must first be trained with annotated data of defective food products. These annotations help the Al model learn what is to be considered a defect.

[0020] The method according to claim 1 accelerates the training of the Al model by generating annotated data of defective food products, instead of manually annotating data of defective food products. Because manual annotation is superfluous, a lot of time and personnel costs are also saved, and many human errors are avoided. In addition, generating models of defective food products provides more freedom, whereby a larger variation of models of defective food products can be generated, as a result of which the Al model that is trained with said digital models can recognize a broader variation of defects. Said freedom also offers the possibility to generate a larger quantity of digital models per defect, which improves the accuracy of the Al model, and thereby the quality control.

[0021] Furthermore, this method offers the possibility to adapt the Al model flexibly and efficiently to changes in the production line, which is crucial given the dynamic nature of modern food production. Production lines are regularly optimized or adapted to process new types of food products. In addition, food is often seasonal, as a result of which production lines often switch food products. Thus, only a limited time is available for adapting the quality control. With the present method, however, the Al model can be adapted simply and quickly by generating new annotated datasets for the adapted food products.

[0022] Generating digital models of food products and digital models of defects separately, and subsequently combining these offers significant advantages for the effectiveness and flexibility of the dataset. Due to the separated approach, separate control is created over the models of defects and products, whereby in the event of changes only one of the two needs to be adapted, which saves duplicate work. Moreover, this separation makes it possible to generate a larger dataset, because defects and products can be varied independently of one another. This results in a broader and more realistic range of training data for the Al model. In addition, it offers the possibility to specifically analyze and adapt specific defects for different product types, which leads to a more accurate and more targeted detection system, tailored to specific inspection criteria and food products in the production line. Furthermore, it can be more easily controlled in which proportions certain defects occur on a single digital food product model or over a collection of digital models of food products, whereby the dataset reflects the actual defect frequencies in the production environment. This creates a better trained Al model that effectively responds to realistic defect patterns, which increases the reliability and precision of the quality control.

[0023] In an embodiment, the Al model is partially trained with manually annotated data of actual food products. Preferably, the training data for training the Al model comprises at most 50% manually annotated data, and even more preferably at most 35%, and even more preferably at most 20%.

[0024] In an embodiment, both the digital models of food products and the production environment are simulated three-dimensionally. This offers the advantage that various two-dimensional images can be generated from one 3D setup, with variations in both product and environment perspectives. The light incidence, shadow formation, and relative positions of objects in the production environment can be easily adapted without having to create new models or a new environment each time. This approach saves time and increases efficiency, because a wide range of realistic inspection scenarios can be simulated. As a result, the Al model is robustly trained to consistently recognize defects, regardless of variations in the production environment. Through this flexible 3D simulation, the model is better prepared for the actual variations in the production line, which significantly increases the accuracy and reliability of the quality control.

[0025] By adding a simulated production environment to the digital models of food products, a more realistic context is created, whereby the Al model learns to recognize not only individual products, but also products within a production environment. This ensures that the Al model is trained to identify objects in a complete, complex context, without being influenced by random occurrences in the background. These background variations are varied in the simulated environment, so that the Al model learns to focus on the product itself, which prevents the model from drawing wrong conclusions.

[0026] By varying the dimensions of the digital models of the food products, a wide range of shapes and proportions can be generated, whereby the first dataset reflects the natural variation in food products. By varying the dimensions before applying the surface characteristics, the surface characteristics such as texture and color retain their proportions, which prevents them from distorting unnaturally during changes in size. This ensures that the digital models are more realistic, which leads to a better training of the Al model, resulting in better quality control.

[0027] In an embodiment, during the step of obtaining the first dataset, limit values are determined based on observed variations in dimensions of the processed food, and wherein the algorithms are configured to generate variations in dimensions based on the determined limit values. The limit values prevent unrealistic digital models from being generated, and through the use of algorithms, a large variety of dimensions can be generated efficiently.

[0028] In an embodiment, a Monte Carlo algorithm is applied, wherein random values are generated within the established limit values to create variations in dimensions. This algorithm offers a diverse dataset without unrealistic deviations occurring.

[0029] In an embodiment, a Gaussian noise algorithm is applied, which adds small, natural variations around an average to the dimensions of the model. This simulates subtle differences which typically occur in production.

[0030] In a further embodiment, a parametric model is applied, wherein parameters such as height, width, and depth are varied within the limit values according to preset rules. This approach offers precise control over specific dimensions, while the natural variability is preserved. Through the use of these algorithms, a broad set of realistic models can be generated quickly and efficiently which corresponds closely to the actual variations in the processed food product.

[0031] In an embodiment, during the step of applying surface characteristics, different lighting effects and different textures are applied to the digital base model. Lighting effects play a crucial role, because light intensity and direction in the actual production environment may vary, which influences the appearance of the surface of food products. Variations in lighting can change the visibility of textures, as a result of which surfaces look glossier or more matte, depending on the light incidence. By simulating these effects in the dataset, the Al model learns to recognize defects despite changes in the visual presentation of textures under varying lighting conditions. This significantly improves the accuracy and reliability of the model because it is prepared for realistic inspection conditions.

[0032] In an embodiment, the light incidence on the digital models of food products is simulated from different angles, so that unique shadow patterns are created which bring out the texture and shape details more clearly. By applying lighting from various directions, varying shadow zones are created which accentuate both height differences and surface features. This is especially important for making small but crucial details visible, such as subtle irregularities, ridges, or small indentations, which might remain hidden under uniform, diffuse lighting. As a result, the Al model is better trained for recognizing defects when different shadow effects occur on the food products.

[0033] In an embodiment, variations in color temperature are simulated on the digital models of food products, from warm to cool light, to accustom the Al model to possible color shifts in a production environment. This is of importance because factories have different light sources, such as ambient lighting, signal lighting on machines, and specific work lighting, which can each emit a different color temperature. In addition, windows at the production line can admit natural light variations, varying from cool morning light to warm evening light, depending on the time of day and the weather conditions. By applying these light color variations to the digital models, the Al model is better prepared for environmental changes, as a result of which it remains able to reliably recognize defects, regardless of the color of the light. This increases the consistency and accuracy of the inspection under diverse production conditions.

[0034] In an embodiment, both smooth and rough textures are applied to the digital models of food products to simulate the surface characteristics of these products. As a result, the Al model learns to recognize defects, even when the structure of the product surface varies.

[0035] In another embodiment, the reflectivity of the textures on the digital models of food products is adapted, so that surfaces can appear both matte and glossy. This prevents the Al model from incorrectly interpreting specular highlights as defects or overlooking actual defects due to reflection effects on the surface. In an embodiment, color variations are added to the textures of the digital models of food products to simulate natural color differences. This helps the Al model to distinguish between normal color variations and real discolorations that may indicate a defect.

[0036] In an embodiment, ray tracing is used for simulating lighting effects on the digital models of the food products, wherein, inter alia, realistic light rays from different angles are simulated. This is advantageous for accurately representing shadow formation and reflections, so that the Al model learns better how light and shadow influence the visibility of textures and surface details.

[0037] In another embodiment, global illumination is applied for simulating lighting effects on the digital models of the food products, wherein, inter alia, light gradients and ambient light are simulated. This is beneficial for creating subtle shadows and color shifts, which ensures a more natural lighting that is comparable to the conditions in an actual production environment.

[0038] In an embodiment, Phong shading is used for simulating lighting effects on the digital models of the food products, to realistically represent the reflection and gloss of surfaces. This helps in training the Al model to distinguish specular highlights of, for example, smooth surfaces from real defects and prevents erroneous interpretations in the case of light reflections.

[0039] In a subsequent embodiment, digital models is deployed for simulating lighting effects on the digital models of the food products, to enhance shadow zones in corners and tight spaces. This is advantageous for representing depth and texture differences, which helps the Al model to recognize small defects in parts of the product that are difficult to see more accurately.

[0040] In a subsequent embodiment, ambient occlusion is applied to enhance shadow zones in corners and tight spaces, which accentuates the sense of depth and texture differences on the models. This facilitates the detection of small defects in difficult corners or surfaces for the Al model that might otherwise remain unnoticed, which improves the overall defect detection.

[0041] In an embodiment, bump mapping is used for simulating a texture on the surface of the digital models of food products, wherein small height differences are simulated without changing the underlying geometry. This is advantageous for creating a realistic relief on surfaces, so that the Al model learns to better distinguish between natural texture and possible defects.

[0042] In another embodiment, displacement mapping is applied for simulating a texture on the surface of the digital models of food products, wherein the surface is actually deformed based on texture data. This is advantageous for accurately representing coarse textures, such as ridges or deep indentations, which helps to train the Al model to recognize defects on uneven surfaces.

[0043] In an embodiment, specular mapping is used for simulating a texture on the surface of the digital models of food products, wherein reflectivity and gloss are adapted to represent variations in glossy and matte areas. This is advantageous for simulating surfaces that are partially reflective, so that the Al model learns to distinguish light reflections from actual defects.

[0044] In a subsequent embodiment, normal mapping is deployed for simulating a texture on the surface of the digital models of food products, by adjusting normal values to simulate small details and reliefs without changing the geometry. This is especially useful for simulating fine surface details, which helps the Al model to recognize subtle texture differences that may indicate defects.

[0045] In an embodiment, the method comprises the step of applying details to the digital models using a Generative Al model.

[0046] In an embodiment, the method comprises the step of applying details to the simulated production environment using a Generative Al model.

[0047] In an embodiment, the 2D images of one or more digital models of food products with defects in a production environment are generated using a Generative Al model.

[0048] In an embodiment, the Generative Al model concerns Generative Adversarial Networks (GANs). In another embodiment, the Generative Al model concerns a diffusion model.

[0049] This offers the advantage that the realism of both the digital models and the simulated production environment can be increased with this by continuously adapting them, preferably based on image material from the actual production environment. As a result, subtle lighting effects and variations on the food products can be simulated more accurately, whereby the Al model learns to deal with realistic lighting influences. The model can also recognize the finer details of surface characteristics such as texture and gloss better because these are matched to actual product properties. In addition, realistic changes in the production environment itself can be added, such as the intermittent presence of objects or personnel, so that the Al model becomes more robust to variations in the background. Furthermore, this approach offers the possibility to adapt the digital models such that they correspond to the exact perspective of one or more cameras that are used for the actual defect detection in the production line. This ensures that the Al model is trained realistically, which significantly improves the accuracy and reliability in the quality control.

[0050] In an embodiment, during the generation of two-dimensional images, the lighting is varied based on the possible lighting conditions on the production line. This ensures not only more realistic images, but also images that correspond more accurately to the actual lighting in the production environment. As a result, the Al model is trained on specific lighting variations that it actually encounters, such as changes in light intensity, color temperature, or angle of incidence. This increases the accuracy of the model.

[0051] In an embodiment, digital models of food products with and without defects are simulated overlapping in one production environment. Since food products are often transported in bulk through the production line, they regularly overlap one another and often lie in each other's shadow, as a result of which defects are sometimes only partially visible or even partly hidden. By including this overlap and shadow formation in the simulation, the Al model is trained to recognize defects under conditions in which these are not fully visible or are visually disturbed by other products. This significantly improves the accuracy of the Al model.

[0052] In an embodiment, motion blur is applied to the two-dimensional images of the digital models. During transport on the production line, food products are often subject to vibrations and movements, which can result in a certain degree of motion blur in the image material of the camera. By incorporating this blur into the simulation, the Al model is trained to recognize defects under realistic conditions in which the products are not standing completely still. This improves the accuracy of the model because it learns to detect defects despite slight blurring caused by movement, which results in a more reliable inspection process that corresponds better to the actual conditions in the production line.

[0053] In an embodiment, the motion blur is based on actual image material of the food products in the production line. The degree and nature of this blur are, in fact, strongly dependent on the specific properties of the product and the conditions on the production line. Different products exhibit different types of movements and vibrations during transport, depending on their shape, weight, and manner of placement. In addition, factors such as the speed of the production line and the position of the camera (for example, the distance to the production line and the angle at which filming takes place) can have a large influence on the degree of blur in the image material. It is therefore difficult to accurately simulate a realistic motion blur without making use of actual video material. By matching the blur to these specific conditions, the simulation closely aligns with the actual situation and the Al model learns to detect defects under the real visual challenges of the production environment.

[0054] In an embodiment, the two-dimensional images are generated based on one or more videos comprising realistic dynamic interactions between digital models of food products with and without defects, wherein these interactions are based on video material of the actual production line. By adding dynamic interactions between digital models of food products, it is prevented that unrealistic configurations arise, such as models that partially or completely occupy the same space. This ensures that the mutual positions of products on the production line are more realistic and correspond better with actual conditions. An additional advantage is that these interactions automatically generate a wide range of different configurations, as a result of which a larger variety of training data is created faster and more simply. By basing the interactions on actual video material of the production line, the realism is further enhanced, because the complexity and variation of movements and positions between food products are so difficult to estimate without this reference. This ensures that the Al model becomes even more accurate and robust in defect recognition, because it is trained on realistic and varied situations that accurately reflect the actual behavior of the products on the line.

[0055] In a further embodiment, physical interactions such as gravity and collision dynamics are added to the simulation, whereby food products can realistically stack, shift, and fall depending on their position on the production line. This offers the advantage that the Al model is trained on defect detection in situations wherein products physically influence each other, such as during the stacking or the tilting of products due to their weight and shape.

[0056] In another further embodiment, friction is added to the digital models, so that products realistically slide or decelerate upon contact with other objects. This ensures that the Al model is exposed to realistic deceleration and acceleration patterns, which often occur when food products touch each other or move over surfaces. In another further embodiment, the elasticity of products is simulated, so that upon collisions the degree of deflection or deformation can be simulated. This helps the Al model to recognize defects, even when products change shape due to pressure or collisions, which often happens in bulk transport.

[0057] In an embodiment, the perspective is varied during the generation of two-dimensional images, which offers multiple advantages for the quality control. Because production lines are constantly evolving and are often adapted or renewed, the cameras for defect detection are regularly moved or placed in a different setup. By training the Al model on defect recognition from different perspectives, the model does not need to be retrained upon a change of the camera positions, which saves time and costs. A further advantage is that defects look different from each perspective due to the changing configuration of products, light incidence, and shadow, whereby the Al model is trained on a wider variety of data. This increases the accuracy because the model learns to recognize defects in diverse visual conditions. Furthermore, this approach makes it possible to deploy multiple cameras for detecting defective food products, without being restricted to a setup where only a single camera functions optimally. The Al model is, after all, prepared for different perspectives, which offers much freedom in scaling up the quality control.

[0058] In an embodiment, the digital base model of the food product is formed by means of at least 100 photos, taken from multiple perspectives of the product. This offers the advantage that a realistic representation of the food product is obtained immediately. Due to the complexity and irregular characteristics of food products, it is often difficult to design all details manually; by basing the model on photos, however, the generation of digital models starts directly from a true-to-life base model. Moreover, the use of photos from different angles ensures that the shape and texture of the product are captured more completely, which forms a detailed and accurate basis for further model variations.

[0059] In an embodiment, the digital base model of the food product is formed using laser scanning. Laser scanning offers the advantage that it has an extremely high accuracy, whereby even the smallest details of the food product can be captured. This makes it possible to digitize complex shapes and surfaces, such as those of natural products, in great detail, which is crucial for reliable quality control and defect detection.

[0060] In a second aspect, the invention concerns an apparatus according to claim 13. This apparatus offers the great advantage that it requires only one camera to identify defective food products, without extensive adjustments to the production line being necessary. Due to the simple installation of a camera aimed at the food in the production line, the quality control can be quickly integrated into existing production environments, without causing delay in the production process. This setup makes it possible to detect defects in real-time, which improves the quality of the food production without the throughput speed being influenced.

[0061] In an embodiment, the displayed data on the graphical user interface comprises one or a combination of the following data types: a timeline of identified defective food products, an overview of the number of identified defects subdivided based on defect properties, and a live display of the camera image with indicated defects. This variation in data types offers multiple advantages. The timeline of identified defects makes it possible to analyze defect patterns, whereby operators gain insight into recurring problems and can respond quickly to quality issues. The overview of the number of defects subdivided according to properties gives detailed information about common defect types, which contributes to targeted improvements in the production process. The live display of the camera image with indicated defects offers direct visual feedback and enables operators to take immediate action, which further increases the speed and effectiveness of the quality control. Together, these data types offer an extensive and flexible support for quality management within the production line.

[0062] In an embodiment, the apparatus comprises at least two cameras that image substantially the same food products in the production line from different perspectives. This offers the advantage that food products are observed from multiple angles, whereby the Al model can identify defects that might not be fully visible from a single perspective. By combining multiple views, hidden or partially covered defects are detected better, which significantly improves the accuracy of the quality control. In addition, this approach increases the reliability of the defect detection under variable product positions or orientations in the production line, because defects remain in view, even when products rotate or move. This multi-camera configuration thus increases the robustness and accuracy of the apparatus, without the production line losing speed.

[0063] In the following, the invention is described by means of non-limiting examples illustrating the invention, and which are not intended or to be interpreted to limit the scope of the invention.

[0064] EXAMPLES A digital 3D base model of a food product can be created on the basis of photos that have been taken from different angles and perspectives of the product. The product is photographed from multiple positions around the object to capture all sides and details, wherein the photos offer small variations in angle and position. For food products with a more complex shape, smaller variations in angle are taken and more photos per angle. Subsequently, these photos are input into specialized photogrammetry software, which uses algorithms to identify and triangulate common points in the photos, thereby creating a three-dimensional point cloud of the surface. Based on this point cloud, a mesh is created which represents the shape of the product in 3D.

[0065] After generating the digital 3D base model of the food product, the dimensions and shapes of this base model are varied to obtain multiple digital models which represent realistic variations of the original product. These variations can be performed manually or automated by means of algorithms, depending on the required quantity and complexity of the models.

[0066] In a manual implementation, an operator adjusts the dimensions and shapes of the base model within 3D modeling software, wherein full control over the variations is possible. For larger datasets, however, algorithms are more effective, because they can apply changes quickly and consistently, which leads to a more efficient production of realistic model variants.

[0067] Use can also be made of parametric modeling to flexibly adjust base parameters such as height, width, and depth within preset margins. Thereby, variants of the base model are automatically generated. The algorithm can, for example, increase or decrease the height of the model within a margin of 10%, while the proportions are maintained, so that realistic variations arise.

[0068] Perlin Noise or Simplex Noise algorithms can also be used to apply subtle asymmetrical deformations which simulate natural irregularities on the surface. These algorithms add noise to specific points on the model, such as slight bulges or indentations, which results in realistic-looking variations which often occur in food products.

[0069] A Monte Carlo algorithm can be applied to select random variations within predefined limit values. At each iteration, the algorithm chooses new values for the dimensions, whereby a dataset with diverse models is created without repetition of the same variations. This is effective for quickly generating a wide range of shapes. Finally, Gaussian Scaling can be used, wherein subtle fluctuations around an average value are applied. This algorithm ensures that most variations lie close to the base model, while a few larger deviations are added for diversity. Hereby, Gaussian Scaling simulates subtle natural deviations in size and shape.

[0070] For adding realistic surface characteristics to the digital models, diverse properties such as texture, color, reflectivity, and roughness can be adjusted to make the model visually correspond better with an actual food product. Different algorithms and techniques are employed herein to simulate these variations accurately and realistically.

[0071] First, texture and roughness can be created with bump mapping and normal mapping, techniques which add details to the surface without changing the underlying geometry. With bump mapping, for example, a slightly rough texture can be added, which simulates a granular surface, while normal mapping provides more depth in the texture by adjusting the light incidence. This is particularly useful for simulating fine structures such as small ridges or irregularities on the product surface.

[0072] For applying color variations, Perlin Noise or Simplex Noise algorithms can be applied, which generate subtle differences in color and simulate a natural color gradation, such as slight discolorations or irregular tones. These algorithms add noise to the color pattern, whereby a realistic, non-uniform color distribution is created. Moreover, shader algorithms can be deployed to flexibly adjust the intensity and saturation of colors, so that the model resembles the natural color variations of the food product.

[0073] To simulate reflectivity and gloss, specular mapping can be applied. This algorithm makes it possible to adjust the reflectivity of specific parts of the surface, so that parts of the product acquire a slight gloss while other parts retain a matte finish. This is useful for products that have different material structures on one surface, such as glossy and dull spots, which contributes to a more realistic surface.

[0074] For products that are partially transparent or where depth effects are desired, subsurface scattering and opacity mapping can be applied. Subsurface scattering simulates how light spreads under the surface of a product, which gives a softer and more natural effect with semi-transparent surfaces, such as with thin peel or juicy parts of the product. Opacity mapping is used to simulate partial translucency, for example at thin edges or parts of the product that are less dense. By applying a combination of these techniques, a versatile and realistic dataset can be created, which helps the Al model to accurately identify defects under diverse conditions. Each algorithm contributes to enhancing the visual accuracy of the digital models, whereby the model is trained more effectively on defect recognition that reflects realistic variations in surface characteristics.

[0075] For generating 3D defects that represent realistic variants of actual damage to food products, use can be made of different techniques and sources. The process begins with collecting image material of defects that actually occur in food products on the production line. This image material serves as a reference to capture the diverse characteristics of the defects, such as shape, size, texture, color change, and intensity of the damage. Defects such as cracks, discolorations, bruises, and superficial damage can be analyzed in as much detail as possible.

[0076] Based on this data, digital models of the defects are generated which represent each of the captured characteristics. Thus, cracks can be modeled with specific lengths, depths, and texture details, so that the different types of cracks that can occur in food products are faithfully simulated. For discolorations, a model can be built in which color gradations and distribution patterns are realistically represented, based on color information and distribution patterns that have been observed in the reference image material.

[0077] An example of automatically generating a 3D model of a defect on the basis of a photo begins with the analysis of an image on which a specific defect, such as a crack, is visible. Image recognition software can be used to identify the edges, length, and depth indicators of the crack in the photo. This information is subsequently converted into depth and texture data, with which a digital 3D model of the crack can be built. Algorithms such as edge detection mark the contours of the crack, while depth mapping captures the depth profiles. These profiles are converted into a three-dimensional mesh which accurately represents the shape and depth of the crack. To add a realistic texture, the surface of the 3D model can be processed with a bump map or displacement map, in which noise is added to simulate the irregularities and rough edges of the crack as observed in the original photo. This method makes it possible to generate 3D models of diverse defects in an automated manner, based on actual damage patterns that have been captured in image material.

[0078] The use of algorithms such as Perlin Noise and Gaussian distribution is useful to generate variations within each type of defect. These algorithms add subtle random variations to the properties of the defects, such as the size, the intensity of the color change, and the texture of the damage. This ensures that the generated defects are not uniform, but show natural variation which corresponds to what can occur in reality. Perlin Noise can, for example, add small, irregular variations to the edges of a crack, while a Gaussian distribution is used to let the intensity of a discoloration decrease in a realistic manner towards the edges. In this way, a varied dataset of defects is created with natural variations which are representative of what can occur in reality.

[0079] In addition, a dataset of defects can be expanded by making use of rigid body simulations to generate standard deformations such as dents. In this case, a rigid body simulation is employed to simulate the exact shape and depth of a dent on a standalone model, wherein the effects of forces such as pressure or impact on the surface are simulated. This resulting model of the dent can subsequently be stored as a standard defect and applied to different food product models without the full simulation having to be performed again each time.

[0080] The dataset with digital models of food products can be combined with the dataset of digital models of defects to generate a realistic and varied set of digital models of food products with defects. This process begins with selecting a digital model of a food product and choosing a suitable defect model from the defect dataset, such as a crack, dent, discoloration, or superficial damage. Each defect model can be specifically positioned and scaled to accurately simulate the natural occurrence of defects in the food product.

[0081] The defects can be placed at specific locations on the food model to create variation. This placement can be set manually or automated by an algorithm that applies the defects at random or specific positions. By applying random positions and introducing variations in orientation or size of the defect model, a dataset with natural and diverse configurations of the food product with defects is created. For example, defects such as rotten spots, mold spots, or other forms of internal degradation are placed randomly on the food product, because in practice they can occur anywhere, and defects such as damage from collisions are often found on the ends, corners, or edges of a food product, because these spots are more vulnerable to physical impact.

[0082] An example of this process is combining a base model of a food product with a crack model, wherein the crack can be stretched, rotated, or adjusted in scale to match the surface and the shape of the food product. The digital model of the defect can be scaled to the size of the digital model of the food product. Algorithms such as random positioning or Gaussian distribution can be applied to adjust the size, placement, and orientation of the defect in a controlled, variable manner, so that the defect looks realistic on different versions of the food product model.

[0083] After the combination of the food product and the defect, texture adjustment can be performed to soften the transition between the food surface and the defect. By making use of bump mapping, normal mapping, or displacement mapping, the defect can be integrated seamlessly into the surface of the food product, whereby it looks less like an "added" element and more like a natural defect that has originated in the product itself.

[0084] For simulating a production environment around the digital models of food products with defects, a 2D or 3D simulation can be set up, depending on the desired degree of realism and details. In both cases, the goal of the simulation is to imitate a realistic inspection environment, based on image material and observations of the actual production environment.

[0085] In a 3D simulation, a virtual environment is created around the three-dimensional models of food products with defects. The process begins with analyzing image material of the actual production environment, such as videos or photos of the production line. This image material is used to capture relevant details such as the arrangement of conveyor belts, equipment, lighting, and environmental objects. Based on this data, a 3D model of the production environment can be built which forms an accurate representation of the physical environment. Herein, the relative positions of equipment, conveyor belts, supporting structures, and other elements can be reproduced, so that the food products are located in a realistic context.

[0086] The simulation can replicate diverse aspects of the production environment to train the Al model in recognition under varying conditions. Thus, variable lighting can be added, such as strong lamps or natural light falling through windows, with which realistic shadows and light reflections are simulated. The light incidence and intensity can be set variably to imitate different times of the day, such as the influence of morning or evening light on the conveyor belt or the bright artificial light that illuminates the textures of the food product. This helps the Al model to recognize defects under diverse lighting conditions. Variable obstacles and personnel can also be added in the simulation.

[0087] Realistic shadows and reflections can further be added based on lighting and the presence of equipment and personnel in the environment. By imitating, for example, the reflection of metal parts of a machine or the shadow pattern of a moving arm, the model is prepared for the visual distractions that are typical in a production environment.

[0088] A 2D simulation recreates a similar production environment around two-dimensional images of digital models of food products with defects. Here, the images of the food product are placed on a 2D background that visually represents the environment, such as an image of a conveyor belt or sorting system. Lighting and shadow can also be applied in this 2D simulation by filters or shaders that generate different light intensities and shadow effects.

[0089] Multiple digital models of food products, both with and without defects, may be placed in a single simulated production environment to create a realistic image of the production line. This setup simulates, for example, the configuration of food products on a conveyor belt, wherein products are displayed in bulk as this would occur in a real production line. By placing the digital models at specific positions, a detailed representation is created wherein products partially overlap each other or lie at different angles and positions. This simulation of a static image of the production environment ensures that the Al model can be trained on diverse product configurations, including variations in visibility of defects, shadow formation, and lighting effects that further increase the realism.

[0090] In a subsequent step, physical interactions between the digital models of food products can be simulated to imitate the dynamics of a production line. Herein, algorithms such as rigid body dynamics and collision detection are used to imitate the effect of contact between products. These algorithms calculate, for example, how products slide, rotate, or overlap on the conveyor belt when they come into contact with each other or with other objects in the environment. In the event of a collision, slight deformations, tilts, or shifts can occur, depending on the position and direction of the contact, which provides a realistic picture of how products behave relative to each other on the production line.

[0091] With this setup, an animated simulation can be generated, wherein the movement of food products through the simulated production line is captured as a video output. This dynamic video simulates the movement on the conveyor belt, while the environmental elements, such as light and shadow, adapt to the movements.

[0092] The next step is obtaining two-dimensional images of the digital models of food products in the production environment. Herein, it is important to generate a large variation of images that accurately simulate the possible perspectives of the camera in the actual production environment.

[0093] In a 3D simulation, virtual cameras are used to capture the models from different angles and positions. The camera can be positioned in a manner that corresponds to the actual cameras in the production line, so that the angles of incidence and perspectives in the images correspond to the realistic inspection angle. By making use of camera rotation and zoom functions, a wide range of perspectives can be captured, from top views and side views to slight oblique angles. In addition, variable lighting settings can be used to adjust the lighting in the 3D environment, so that defects are visible under diverse lighting conditions. This method yields a large diversity of images which simulate a lifelike inspection environment.

[0094] In a 2D simulation, certain areas can be zoomed in on, and filters and perspective distortions can be applied to display the images from different angles and positions, wherein the angle and position of a typical inspection camera on the production line are simulated. In the 2D simulation, too, lighting effects can be added, for example by applying lighting filters to vary the intensity and direction of the light in the images. As a result, a set of two-dimensional images is created which simulate diverse inspection angles and lighting conditions, comparable to the configurations in the 3D simulation.

[0095] To prepare the Al model for the motion blur which often occurs in image material of the production line, motion blur can be simulated in the two-dimensional images. To simulate these effects, special blur filters and algorithms such as motion blur can be applied to the images. These algorithms simulate the blurring effect that occurs when products move during photographing.

[0096] The type and the intensity of the motion blur can be matched to actual image material of the production environment, whereby the degree of blur in the simulation corresponds realistically to the actual conditions. By analyzing, for example, videos of the production line, the specific patterns of blur, such as the direction and length of the blur streaks, can be simulated. This ensures that the Al model learns to recognize defects despite the presence of motion blur, which is essential for an accurate inspection on a dynamic production line.

[0097] The Al model can be trained with the collected two-dimensional images by means of a machine learning process, wherein the model learns to distinguish between food products with and without defects and simultaneously to accurately identify different types of defects. For this, a deep neural network, such as a convolutional neural network (CNN), is typically used, which can effectively analyze visual patterns and features. During the training phase, the Al model receives a series of images which are labeled as "defect" or "non-defect," and possibly also with detailed labels such as "crack," "discoloration," "bruises," or "surface damage." In addition, additional labels can be used which provide valuable information, such as "severity of defect" (light, moderate, severe), "location of defect" (center, edge, surface), and "dimensions of defect" (large, small, micro-defect).

[0098] The Al model learns through these labels to make increasingly precise distinctions between different types of defects and associated properties, which contributes to a comprehensive analysis of the product quality. By iteratively optimizing the weights within the model, the model develops an increasing accuracy in recognizing defects, including relevant details.

[0099] Once trained and optimized, the Al model can be deployed in a production inspection system for real-time detection of defective food products on the production line. This system consists of one or more cameras that continuously capture images of the food products while they move over the conveyor belt. The images are forwarded directly to a computer on which the Al model runs, which processes the images to identify and classify defective food products according to the previously learned labels. Thanks to the real-time processing, the system can signal, classify, and analyze defects in depth within a few seconds based on location, severity, and type.

[0100] The system comprises a graphical user interface (GUI) that displays the defect detection results in real-time and can be accessible in different ways, such as via a screen next to the production line, a web interface, or a mobile app. The GUI offers multiple forms of feedback on the detected defects. Possibly a live display of the camera images, which show the food products in the production line, wherein defective products are marked with visual indicators, such as red outlines or color codes that indicate the type of defect. This enables operators to see directly where defects are located on the production line and which types of defects are present.

[0101] The GUI possibly also contains a timeline of detected defects which captures the moments at which specific types of defects were detected. This offers insight into trends or patterns over a certain period, which is useful for monitoring the production and quality consistency. In addition, the GUI possibly displays statistics such as the number of detected defects over a specific period and an overview of the defects distributed by type, such as "crack" or "discoloration," with additional data regarding location and severity. These statistics help operators and quality managers in identifying recurring defects, so that structural problems in the production line can be addressed. Furthermore, the GUI can provide a warning when the number or the severity of defects exceeds a preset threshold, which can be a signal for potential problems in the production process that must be addressed. Besides direct visual feedback, the system offers options for monthly reports and automatic data export, which enables the management team to analyze trends and quality data in the long term. This system not only enables operators to intervene directly when defects are detected, but also offers valuable insights that support quality monitoring and process optimization. Through continuous monitoring and feedback, the system contributes to a consistent food quality on the production line and enables timely intervention to minimize quality problems.

Claims

CLAIMS1. A method for quality control of processed food in a production line with a camera, the method comprising the steps of:- obtaining a digital base model of a food product;- obtaining a first dataset comprising digital models of food products by applying variations to the digital base model; obtaining a second dataset comprising different generated digital models of defects;- obtaining a third dataset comprising digital models of food products with defects by combining the first and the second dataset; simulating a production environment, and combining the simulated production environment with at least one digital model of a food product; generating two-dimensional images of one or more digital models of food products with defects in a production environment;- training an Al model with the two-dimensional images to recognize food products with defects; obtaining image material of the processed food in the production line; identifying food products with defects by analyzing the image material with the trained Al model; wherein the first dataset is obtained by, using algorithms, varying the dimensions of the digital base model and applying surface characteristics to the digital base model; and wherein during the obtaining of the second dataset different types of defects are generated; and wherein during the obtaining of the third dataset the type of defect and the location of the defect on the digital model are varied per digital model.

2. The method according to claim 1, wherein during the step of obtaining the first dataset limit values are determined based on observed variations of dimensions of the processed food, and wherein the algorithms are configured to generate variations in dimensions based on the determined limit values.

3. The method according to any one of the preceding claims, wherein during the step of applying surface characteristics different lighting effects and different textures are applied to the digital base model.

4. The method according to any one of the preceding claims, further comprising the step of generating digital models and / or simulating a production environment using a Generative Al model.

5. The method according to any one of the preceding claims, wherein during the generating of two-dimensional images the lighting is varied based on the possible lightings on the production line.

6. The method according to any one of the preceding claims, wherein digital models of food products with and without defects are simulated overlapping in one production environment.

7. The method according to any one of the preceding claims, further comprising the step of applying a motion blur to the two-dimensional images.

8. The method according to any one of the preceding claims, wherein the digital base model and the digital models of the defects and of the food products are three-dimensional.

9. The method according to any one of the preceding claims, wherein the two- dimensional images are generated based on one or more videos comprising realistic dynamic interactions between digital models of food products with and without defects, wherein said dynamic interactions are based on video material of processed food in the production line.

10. The method according to any one of the preceding claims, wherein during the step of generating the two-dimensional images the perspective is varied.

11. The method according to any one of the preceding claims, wherein the digital base model of the food product is obtained based on at least 100 photos, taken from multiple perspectives of the food product.

12. The method according to claim 11, wherein during obtaining the digital base model, a temporary matte spray is applied to food products with transparent, reflective, or very smooth surfaces.

13. An apparatus for quality control of processed food in a production line with a camera, the apparatus comprising:- at least one camera directed at the processed food in the production line; a computer configured to receive images from the at least one camera, and to identify defective food products based on the received images;- a graphical user interface configured to display data based on the identified defective food products; and wherein the computer comprises a memory with an Al model trained with generated labeled data of food products with and without defects in a simulated production environment, and wherein the computer is configured to identify defective food products using the Al model.

14. The apparatus according to claim 13, wherein the displayed data comprises one or a combination of the following data types: a timeline of identified defective food products, an overview of the number of identified defective food products subdivided based on properties of the defect, a live display of the image of the camera on which defective food products are indicated.

15. The apparatus according to claim 13 or 14, comprising at least 2 cameras which capture substantially the same food products in the production line from a different perspective.