Method and system for assisting an additive manufacturing process

Abstract

A system for assisting an additive manufacturing process of a workpiece (22) includes an interface configured to receive a first data stream (30) and a second data stream (31), where the first data stream (30) and the second data stream (31) are generated while the workpiece (22) is manufactured by the additive manufacturing process, wherein the first data stream (30) comprises first data provided by a control device (23) controlling the additive manufacturing process, the first data representing a first form of the produced part of the workpiece (22), and the second data stream (31) comprises second data provided by a sensor system (24) monitoring the additive manufacturing process, the second data representing a second form of the produced part of the workpiece (22). The machine executable component comprises a data fusion component configured to receive the first data stream (30) and the second data stream (31) and to generate a merged data stream (32) by associating the first data stream with the second data stream, such that there is a common time reference between the first data stream and the second data stream in the merged data stream; a geometric classification component configured to determine a second form of the produced portion of the workpiece based on the merged data stream; and a quality inspection assembly configured to compare the second form with the first form to assess the quality of the produced portion of the workpiece.

Description

Technical Field

[0001] The presently disclosed subject matter relates to systems and methods for assisting additive manufacturing processes. Background Art

[0002] With the widespread use of additive manufacturing (3D printing), in which material is added to build a workpiece layer by layer, the need for quality assurance of printed parts is increasing.

[0003] If there are qualitative printing problems in additive manufacturing due to machine, process or material problems, the workpiece quality suffers and leads to scrap. And if the printing error is not detected or even hidden by adding layers over the problematic area, such hidden quality issues can easily lead to a large number of subsequent problems when the wrong workpiece is used in the product, especially in availability-critical or safety-critical products such as in the automotive, railway or aviation sectors.

[0004] Therefore, there is a need for in-process error detection and correction of the AM process. Summary of the invention

[0005] This need is met by a system for assisting an additive manufacturing process of a workpiece (in-process), for example by monitoring the quality of a produced portion of the workpiece, in other words, the quality of a partially produced workpiece. The additive manufacturing process comprises a plurality of printing steps, and the system comprises

[0006] - memory (volatile or non-volatile) that stores machine-executable components,

[0007] - a processor operably coupled to the memory and configured to execute the machine-executable components, and

[0008] - an interface configured to receive a first data stream and a second data stream, wherein the first data stream and the second data stream are generated while manufacturing the workpiece by the additive manufacturing process.

[0009] The first data stream comprises first data provided by a control device controlling the additive manufacturing process, the first data representing a first form of a produced part of the workpiece. The first form consists of a first geometric shape associated with a printing step of the additive manufacturing process.

[0010] The second data stream includes second data provided by the image sensor system monitoring the additive manufacturing process, the second data representing a second form of the produced portion of the workpiece, wherein

[0011] The machine executable components include

[0012] a data fusion component configured to receive (as input) the first data stream and the second data stream and to generate / provide (as output) a merged data stream by associating / linking the first data stream with the second data stream such that there is a common time reference between the first data stream and the second data stream in the merged data stream,

[0013] a geometric classification component configured to determine a second form of the produced portion of the workpiece based on the merged data stream and to decompose the second form of the produced portion of the workpiece into at least one second geometric shape, and

[0014] - A quality inspection component configured to compare the second geometric shape with the (corresponding) first geometric shape in order to assess the quality of the produced part of the workpiece.

[0015] The term "produced portion of a workpiece" emphasizes the fact that the workpiece undergoes a production process, and the system disclosed in the present invention is designed to assist the additive manufacturing process in real time (eg, online). For example, each produced portion may correspond to a predetermined stage of the production process.

[0016] The data fusion component establishes a correspondence between the first form and the second form at any time. This makes it possible, for example, to compare the execution results of a specific line of a partial program executed by the control unit with a frame (image) in the image data stream.

[0017] One or more components disclosed in the present invention may be implemented as software, software modules, software packages, etc., including, for example, human or machine readable code in a programming language written in a declarative form of commands. The component may be, for example, compiled and / or run.

[0018] The quality assessment can, for example, conclude that the quality is (still) good, has deteriorated, or has even improved. For example, a quality degradation can be detected when errors occur during the additive manufacturing process of a workpiece, such as printing errors.

[0019] In other words, assessing the quality includes detecting possible (early) errors that occur during the additive manufacturing process, ie during the process of (stepwise) adding material to a portion of a workpiece, for example.

[0020] Errors can include geometric errors, such as errors in the thickness and / or width of the printed circuit board (PCB), breaks, missing material in the printed layer (which can result in grooves and / or cavities in the workpiece, incomplete powder layers, missing coatings, etc.).

[0021] The in-process (real-time) quality assessment disclosed herein allows for the detection of early process errors and, for example, enables in-process error correction, thereby improving the quality of the final workpiece.

[0022] In one embodiment, the geometric classification component is further configured to receive the merged data stream and detect a produced portion of the workpiece in a second data stream of the merged data stream.

[0023] In one embodiment, each printing step may be defined or determined by a predefined or predetermined action performed by an additive manufacturing device or machine. This may be, for example, a certain cycle (certain lines of G-code) in a part program (e.g., a part program of an NC program) that causes the machine's print head to print a layer, such as a single material layer, or some predefined form.

[0024] The geometry classification component may be further configured to match and / or associate the second geometry with a printing step of the additive manufacturing process.

[0025] The second geometric shape may be defined in terms of a predetermined set of (eg substantially printable) geometric (2D and / or 3D) shapes.

[0026] Basic shapes can be shapes that cannot be further decomposed into simpler shapes, such as a circle. Printable refers to the printability of the shapes by an additive manufacturing machine, for example they can be matched and / or associated with some G-code printing functions.

[0027] In one embodiment, the quality inspection component may be configured to compare the second geometric shape with the first geometric shape after each printing step to assess the quality of the produced portion of the workpiece.

[0028] In one embodiment, the geometric classification component comprises a machine learning algorithm, in particular a classification algorithm, which is used to detect produced parts of the workpiece in the second data stream of the merged data stream. The classification algorithm may be a trained algorithm.

[0029] In one embodiment, the machine learning algorithm may be (pre-)trained on a predefined training dataset representing the form of the produced parts of the workpiece.

[0030] In one embodiment, a predefined training data set represents a decomposition of a form of a produced portion of a workpiece into geometric shapes, and a machine learning algorithm is used to determine at least one decomposition of a second form of the produced portion of the workpiece into second geometric shapes.

[0031] In one embodiment, the classification algorithm may include a reasoner based on descriptive logic. The reasoner may be configured to perform formal and mathematical reasoning by using a predetermined set (e.g., a set of axioms). In particular, the reasoner may be an untrained reasoner.

[0032] In one embodiment, the predetermined set (for the reasoner) includes semantic descriptions of geometric properties and relations between geometric shapes according to a descriptive logic.

[0033] The semantic description in the predetermined dataset may include properties of (basic) shapes, such as size, shape (eg number of corners, etc.), (possible) relationships to each other (on top, seamlessly connected, centered, etc.).

[0034] Furthermore, the predetermined data set may comprise a description (or semantic definition) of (equivalence) classes of shapes (eg a "sphere" is a shape without corners - semantic definition of geometric shapes - basic shape ontology).

[0035] In other words, semantics can be part of the quality inspection component.

[0036] This facilitates semantic classification of, for example, 2D and / or 3D geometric shapes based on a classification algorithm and an algorithm configuration using equivalence class reasoning.

[0037] In one embodiment, the machine-executable components further include an action feedback component configured to provide a suggestion of at least one action to be performed on an additive manufacturing machine performing an additive manufacturing process of a workpiece.

[0038] In one embodiment, for example, providing advice to an operator (of the additive manufacturing system) may include displaying the advice on a display device. This may provide ongoing guidance to the operator of the additive manufacturing machine.

[0039] In one embodiment, the at least one action comprises stopping the additive manufacturing process, and in particular performing (manually, for example by an operator, or automatically) a corrective action after stopping the additive manufacturing process.

[0040] In one embodiment, the corrective action is associated with a (known / predetermined) material defect and a (known / predetermined) workpiece quality degradation and / or comprises a corrective machining program (G-code) executable by the control device.

[0041] In other words, in addition to the production stop feature, the system can also be used for corrective actions. This is particularly interesting for long run times and expensive printing processes. Not only can it stop or interrupt the additive manufacturing process, the system can also influence the upcoming printing steps to correct the errors found. For example, more material is printed to the next layer where the quality inspection identified a material defect.

[0042] In one embodiment, the action feedback assembly is further configured to suggest at least one corrective action to the additive manufacturing process (without interrupting the manufacturing process) and cause the control device to perform the at least one corrective action. This particularly allows for correction of early failures in the printing process.

[0043] In other words, for example for some known material defects and workpiece quality degradations, the corrective action can be fully automatic. Such automatic correction procedures can again be derived from known or predefined semantic equivalence classes.

[0044] In one embodiment, semantic equivalence classes may be defined based on input from a CAD / CAM system that may decompose a workpiece into individual geometric shapes.

[0045] For example, with a predefined validation routine for the G-code steps, automatic corrective actions can be more easily implemented. For example, if a printed line is missing some material in the middle of the line, the system will simply identify the location of the gap and initiate a corrective line printing action at that location (e.g., on the current or upcoming 3D print layer).

[0046] In another embodiment, the defined / trained equivalence classes are shared with other users, who can reuse them in their 3D printing processes, thereby reducing their own definition / training phase work and the technical equipment / infrastructure costs required. This could be the basis for new business models.

[0047] In one embodiment, the system can provide decision support in the form of notifications or warnings to the operator and / or step-by-step manual correction instructions at a safe point selected by the system, for example through a decision support component, from the perspective of the 3D printing process or simply after the manufacturing process of the workpiece is completed.

[0048] In one embodiment, the system can gain additional geometric classification accuracy (during inference) when additional inputs from other sensors are added (e.g., external vibration sensors, ultrasonic sensors, 3D laser scanners, etc.).

[0049] In one embodiment, the clock of a control device controlling the additive manufacturing process is a common time reference, for example, in order to know which G-code line corresponds to which frame in the data stream.

[0050] In one embodiment, the second data comprises image data (data obtained by an imaging process; monitoring is an imaging process), in particular video stream data (of the surface of the produced part of the workpiece), a laser scanning data stream, an ultrasound data stream.

[0051] In one embodiment, the system may provide a report of the produced workpieces, their production quality, the application of any in-process corrective actions, and (if so) the root causes of the decisions for those corrective actions. This may be performed by a reporting component.

[0052] The above needs are also met by a computer-aided method for additive manufacturing of a workpiece, for example by monitoring the quality of a produced portion of the workpiece, the method comprising

[0053] - while producing a workpiece by an additive manufacturing process comprising a plurality of printing steps, receiving a first data stream and a second data stream, wherein

[0054] The first data stream comprises first data provided by a control device controlling the additive manufacturing process, the first data representing a first form of a produced part of the workpiece, wherein the first form consists of a first geometric shape associated with a printing step of the additive manufacturing process, and

[0055] The second data stream comprises second data provided by the image sensor system monitoring the additive manufacturing process, the second data representing a second form of the (identical!) produced part of the workpiece, wherein

[0056] - generating a merged data stream by associating / linking the first data stream with the second data stream such that there is a common time reference between the first data stream and the second data stream in the merged data stream,

[0057] - determining a second form of the produced part of the workpiece and at least one decomposition of the second form of the produced part of the workpiece into second geometric shapes based on the merged data stream, and

[0058] - comparing the second geometry with the (corresponding) first geometry in order to assess the quality of the produced part of the workpiece.

[0059] Furthermore, this need is met by a computer program comprising instructions which, when executed by a computing device, cause the computing device to perform the steps of the above-mentioned method.

[0060] Furthermore, this need is satisfied by a computer readable medium comprising such a computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The above and other objects and advantages of the present invention will become apparent upon consideration of the following detailed description of certain aspects, which are merely indicative of several possible ways in which they may be practiced. The description is made in conjunction with the accompanying drawings, in which like reference numerals refer to like parts throughout, and in which:

[0062] Figure 1 shows an additive manufacturing system,

[0063] Figure 2 A flowchart showing the additive manufacturing process,

[0064] Figure 3 Shows Figure 2 The data flow in

[0065] Figure 4 The inference phase of the additive manufacturing process is shown. DETAILED DESCRIPTION

[0066] Figure 1 An additive manufacturing system 1 is shown. The system 1 comprises a 3D printer 2 and a computing device 3 for assisting an additive manufacturing process, such as a process performed by the 3D printer 2.

[0067] The 3D printer 2 may comprise a machining (printing) area 20 in which a print head 21 is arranged to print a workpiece.

[0068] Figure 2 The process of printing a workpiece is shown, wherein a portion of a workpiece 22 has been printed.

[0069] The 3D printer 2 further comprises a control unit 23, which can be designed as a CNC control unit. The control unit 23 controls the additive manufacturing process. In particular, the control unit causes the print head 21 to perform specific movement and printing actions, for example according to a part program executed on the control unit 23 during the additive manufacturing process. The part program can include a code, also called a G-code.

[0070] For example, the code may include a command or group of commands that, when executed by the control unit 23, causes the 3D printer 2 to print a specific geometric form or shape.

[0071] During the additive manufacturing process, the workpiece 22 is created step by step. At each printing step P1, P2, P3, P4, etc., the control unit 23 may execute one or more lines of the part program and cause the print head 21 to print some geometric form / shape on the portion 22 of the workpiece.

[0072] In other words, there may be a correspondence between the part program code (G-code) and the printed geometric form or shape. For example, the part program code may include a command such as "print a cylinder of radius R and height H".

[0073] To monitor the additive manufacturing process, the 3D printer 2 may include a monitoring system 24, which may include one or more cameras. The monitoring system 24 generates a stream, i.e. a sequence of image data 31 of the surface of the produced portion of the workpiece 22, such as video stream data, and may be part of a computer vision system.

[0074] The monitoring system 24 may be integrated into the 3D printer 2 and be located, for example, in the printing area 20 .

[0075] The monitoring system 24 may also include sensors for generating image data, in particular a laser scanner generating laser scanning data (stream), an acoustic scanner generating ultrasound data, or the like.

[0076] Data, in particular data in the form of a continuous data stream 30 (first data) from the control unit 23 are submitted to the calculation device 3 .

[0077] The first data 30 may include parts of a program (e.g. a G-code) executed by the control unit 23. Furthermore, the first data may include target data, which have been optimized in the process, for example. Furthermore, the first data may also include actual data measured in the process from the perspective of the control unit 23, i.e. values ​​from the running process. For example, a curve-optimized axis setpoint and an actual value interpreted from the measured current. The setpoint may be optimized by the control unit 23.

[0078] In other words, the first data 30 represents the form of the produced portion of the workpiece 22 from the perspective of the control unit 23 , that is, the first form of the produced portion of the workpiece 22 .

[0079] The second data 31 represent the form of the produced part 22 of the workpiece from the perspective of the monitoring system 24 .

[0080] Using the first data 30, image data 31 (second data) from the monitoring system 24 is also submitted to the computing device 3 associated with the 3D printer 2. The data between the computing device 3 and the 3D printer 2 is usually bidirectional, which is caused by Figure 1 The double arrows “data flow” are shown in FIG.

[0081] The computing device 3 can be designed as an edge computing device, such as an edge industrial computer or some other circuit, for example, including one or more field programmable gate array-based integrated circuits, i.e., FPGA-based integrated circuits. Those skilled in the art will understand that the computing device 3 and similar devices include a processor that executes a machine executable component.

[0082] The computing device 3 can interpret the received data 30 , 31 .

[0083] Figure 1 Schematically shows a sequence of images corresponding to printing steps P1, P2, P3, P4, etc., which are performed by a 3D printer during an additive manufacturing process. T is the time axis. For example, each printing step may correspond to printing a single layer.

[0084] Figure 1 It is shown that based on the first data 30 at each point in time T, the form of the produced part 22 of the workpiece from the perspective of the control unit 23 can be generated, in particular visualized.

[0085] The computing device 3 comprises a data fusion component configured to receive as input a first data stream 30 and a second data stream 31 and to generate as output a merged data stream 32. To produce the merged data stream 32, the data fusion component associates the first data stream 30 with the second data stream 31 such that a common time reference exists between the first data stream 30 and the second data stream 31 in the merged data stream.

[0086] In other words, the data fusion component links the two data flows 30 , 31 to one another so that, for any point in time of the additive manufacturing process, in particular for any printing step P1 , P2 , P3 , P4 , etc. , the form of the portion 22 of the workpiece from the perspective of the control unit 23 (the first form of the portion of the workpiece 22 ) is associated with the form of the portion 22 of the workpiece from the perspective of the monitoring system 24 (the second form of the portion 22 of the workpiece) and is therefore comparable thereto.

[0087] Thereby, a correspondence between the first form and the second form is established at any time T. Thereby, during / after each printing step P1, P2, P3, P4, etc., the form of the produced portion 22 of the workpiece from the perspective of the control unit 23 can be compared with the form of the produced portion of the workpiece 22 from the perspective of the monitoring system 24.

[0088] For example, the association or linking can be performed as follows. The G-code (which can be contained in the first data stream 30) has a logical time - the G-code line number. The G-code line can be assigned to the clock of the control unit 23. The second data stream 31 (for example a video stream) has its natural time. The camera can be designed as a network time protocol client, i.e., an NTP client. In this way, the clock of the control unit 23 in the first data stream 30 can be related / associated with the time and the second data stream 31.

[0089] In other words, the clock of the control device 23 controlling the additive manufacturing process may be a common time reference.Thereby, a correspondence between G-code lines in frames in the image data stream 31 is provided.

[0090] In one embodiment, multiple 3D printers 2 are associated with a computing device 3. In this case, data streams from a particular 3D printer may include metadata that may be used to tag those data streams and associate them with their correct 3D printer.

[0091] Furthermore, information about the position and / or orientation of the workpiece and / or the monitoring system, such as the camera and / or the print head and / or the printing area, such as the printing area of ​​a 3D printer, can be provided. This can be achieved, for example, by obtaining information about the relationship between the workpiece and / or the 3D printer (print head) and / or the coordination system of the camera using movement commands.

[0092] The merged data stream 33 is then provided to a geometric classification component of the computing device 3. The geometric classification component is configured to determine the second form of the produced part 22 of the workpiece based on the merged data stream.

[0093] The geometric classification component can thus determine the geometrical properties of the produced portion of the workpiece 22 .

[0094] Furthermore, the computing device 3 comprises a quality checking component configured to compare the second form with the (corresponding) first form in order to assess the quality of the produced part of the workpiece 22 .

[0095] To this end, the quality inspection component can analyze the data stream 32 and, for example, extract from the first data stream information about a first form, i.e. the form of the produced part of the workpiece 22 from the point of view of the control unit 23, corresponding to a second form of the produced part 22 of the workpiece provided to the quality inspection component by the geometric classification component. The information about the first form can, for example, be part of a part program in the first data stream 30, according to which, for example, a cylinder of height H and radius R is printed at step Px. The second form of the produced part of the workpiece 22 as a result of the analysis of the image stream data by the geometric classification component can be a Celinda of height H and radius R printed at step Px. In this case, no error has occurred in the additive manufacturing process and it can continue.

[0096] In one embodiment, the computing device 3 may include an action feedback component configured to provide a suggestion of at least one action to be performed on the 3D printer 2 for the situation when the quality inspection component determines that there is a difference between the first form and the second form of the produced part of the workpiece 22. This means that there is a difference between the produced part of the workpiece (the first form) as interpreted / seen by the controller unit 23 and the produced part of the workpiece (the second form) as seen by the monitoring unit 24.

[0097] The action that may be suggested to, for example, an operator of the 3D printer 2 may include stopping the additive manufacturing process, in particular performing (manually, for example by an operator, or automatically) a corrective action after stopping the additive manufacturing process.

[0098] In one embodiment, the corrective action is associated with a (known / predetermined) material defect and a (known / predetermined) workpiece quality degradation and / or comprises a corrected machining program (G-code) executable by the control device.

[0099] In one embodiment, the action feedback component may be further configured to suggest at least one corrective action to the additive manufacturing process and cause the control device to perform the at least one corrective action. This allows for fully automatic correction of the additive manufacturing process without any human support.

[0100] In one embodiment, the system may provide, for example, through a quality inspection component or through some other component, a set of technical statements in the form of an automatically improved 3D printer program (G-code) for modifying the engineering cycle of the printed workpiece to further reduce printing problems and the need for corrective actions during production (automatic self-improvement process in the engineering phase).

[0101] In particular, in this embodiment, model-based reasoning on correlations between controller (G-code, print head movement, current), sensor data (temperature, flow, pressure), and 3D geometric scene detection can be used to understand the root causes, improvement options, and the impact / benefits of the options. The results can be presented to the operator / G-code programmer in the form of a report.

[0102] In the case of long running production or in the case of production with expensive materials, immediate stopping of the machine when a printing problem is detected early saves costs. Automatic correction of the problem during the process reduces waste generation. In the case of correctable errors, immediate automatic problem analysis and instruction of automatic corrective actions during or at the end of the current printing step and before starting the next printing step mitigates artifact problems in quality before the next printing step continues to cover the current printing layer and thus the area of ​​immediate concern.

[0103] To this end, the motion feedback component can generate a corrected partial program and cause the control unit 23 to execute the corrected G code.

[0104] In one embodiment, the geometry classification component may be configured to receive the merged data stream and detect a produced portion of the workpiece in a second data stream of the merged data stream.

[0105] As mentioned above, the additive manufacturing process comprises a plurality of printing steps P1, P2, P3, P4, etc., wherein at each step some geometric form or shape is printed. In particular, at each printing step, at least one geometric shape, in particular only one geometric shape, is printed, for example according to a partial G-code defining the printing step P1, P2, P3, P4, etc.

[0106] Thus, the first form may consist of a first geometric shape associated with a printing step of the additive manufacturing process, ie a geometric shape “seen” from the perspective of the control unit 23 of the 3D printer 2 .

[0107] In this case, the geometric classification component can be configured to determine a decomposition of the second form of the produced portion 22 of the workpiece into at least one decomposition of a second geometric shape and match / associate the second geometric shape with a printing step P1, P2, P3, P4, etc. of the additive manufacturing process.

[0108] The second geometric shape may be selected from a predetermined set of geometric shapes, ie geometric shapes that can be printed by a 3D printer, which means that there is a G-code for printing each geometric shape from the predetermined set of geometric shapes.

[0109] The predetermined set of (printable) geometric shapes may include basic printable geometric shapes, ie geometric shapes that cannot be further decomposed.

[0110] The geometry can be a 2D or 3D geometry.

[0111] The quality inspection component may be configured to compare the second geometry with the first geometry, in particular after each printing step, in order to assess the quality of the produced portion 22 of the workpiece.

[0112] In an ideal situation, when the 3D printer 2 prints what it is supposed to print, there will be a one-to-one correspondence between the first geometric shape and the second geometric shape.

[0113] If there is a deviation or inconsistency between the first geometry and the second geometry, corrective action may be suggested and / or implemented, as described above.

[0114] In one embodiment, the geometric classification component may include a machine learning algorithm, in particular a classification algorithm, which is used to detect the produced portion of the artifact in the second data stream of the merged data stream.

[0115] The machine learning algorithm may be a trained algorithm. In particular, the machine learning algorithm may be trained on a predefined training data set representing the form of the produced part of the workpiece.

[0116] In one embodiment, the predefined training data set may represent a decomposition of the form of the produced portion of the workpiece into geometric shapes, and the machine learning algorithm is used to determine at least one decomposition of a second form of the produced portion of the workpiece into second geometric shapes.

[0117] A machine learning algorithm, such as a trained machine learning algorithm, can be designed to classify 3D printing situations and / or objects. Such a machine learning algorithm can be trained for all the different steps of printing geometries such as spheres, cylinders, single points, lines, or curves.

[0118] The training input data may be obtained from equivalence class definitions or from training of equivalence classes on reference printer data in the time domain or path domain (e.g., axis movements of set points, energy consumption, speed and acceleration, and actual values ​​of the print head over time or along the printing path) and / or input video data labeled during the definition / training phase, where for example each equivalence class describes a normalized geometric object such as a cylinder with variable attributes such as height, width, depth or circumference. By automatically defining a dynamic aggregation of such equivalence classes based on semantic combinations of underlying equivalence class classifiers, the resulting equivalence classes may be used individually or in combination during inference (e.g., online classification) on a computing device.

[0119] For example, techniques known from 3D polygon combination in computer graphics, which use multiple base polygons to approximate a highly detailed final 3D geometry, can be applied by the machine learning algorithm disclosed in the present invention. The greater the number of polygons (or in our case, the greater the number of equivalence classes), the greater the number of equivalence classes that are finally aggregated, and therefore the higher the specificity of the correct match classification of the actual image of the workpiece during or after processing.

[0120] In one embodiment, the quality inspection component may also include a machine learning algorithm, such as a trained machine learning algorithm, particularly for classifying the quality of the produced portion of the workpiece 22 .

[0121] Figure 2 In more detail, it is shown that Figure 1 Flow chart of an additive manufacturing process performed by system 1.

[0122] The computing device may be designed as an edge computing system 3 for performing workpiece quality inspection and online correction of additive manufacturing during the process of adding material to the workpiece 22 .

[0123] The additive manufacturing process (eg, 3D printing) is performed by a 3D printer 2 , which includes a printing area 20 , a print head 21 , a workpiece 22 , a controller 23 , and a monitoring system 24 .

[0124] The controller 23 may receive data from analog and digital sensors (eg, vibration sensors).

[0125] The monitoring system 24 may include one or more cameras, 3D laser scanners, and the like.

[0126] Those skilled in the art will appreciate that the edge computing system 3 may receive and process data from more than one 3D printer performing different additive manufacturing processes.

[0127] In other words, the input data of the edge computing system 3 can come from different machines and / or operational technology (OT) devices in the workshop.

[0128] The controller 23 may provide data regarding the current position in the numerical control program of the machine 2, print axis information, feed rates, and acceleration values ​​according to the tool path of the 3D printer along the additive process.

[0129] The camera provides image data comprising a plurality of images showing the motion field as the basis for the quality classification process on the edge computing system 3. At least one video frame is available for each printed layer to allow a correct classification and thus a correct understanding of the printing situation.

[0130] Optionally, data from analog and digital sensors (e.g., vibration sensors) can be provided. For example, by correlating analog vibration data or digital information about start / stop events or opening / closing machine doors. This input data can be merged on the edge computing system 3 by a data fusion component.

[0131] Optionally, the monitoring system 24 may include one or more 3D laser scanners, which provide further image data. This may improve the accuracy of the classification.

[0132] In one embodiment, image data is provided rather than camera data.

[0133] The input data from the laser scanners, the so-called 3D point cloud over time, can be merged on the edge computing system via a data fusion component.

[0134] To receive data, the edge computing system may include one or more interfaces. The "southbound" side of the edge computing system 3 represents a logical view of the physical computer network facing the machine 2. This portion of the edge computing system 3 includes a logical representation of the real hardware and devices on the shop floor, which is abstracted by so-called "adapters" to integrate these devices into the system architecture at runtime. Each adapter can act as a digital substitute / proxy for the device connected to it via the computer network, integrate data (such as camera data), and provide feedback to the 3D printing controller 23 (such as providing a change request to a machining operation or changing the 3D printing program during material processing or applying a print head 21 feed rate override request).

[0135] The southbound side can include adapters and integrate them with the data fusion component.

[0136] Those skilled in the art will appreciate that the data fusion component and the action feedback component can be designed as one module (see Figure 2 ).

[0137] The data fusion component can be configured to integrate and synchronize different data sources in a temporal and, for example, semantic manner. This can allow semantic classification of operational scenarios using 3D geometry equivalence classes. For example, deterministic real-time cycle counters, position data of the print head 21 (from the 3D printer controller 23), and / or line position information of a running NC program can be used as reference data to synchronize all other data.

[0138] The action feedback component allows actions to be sent back to the 3D printer 2 which will have a corrective / positive effect on the printing process. The action can be: setting a shared system or user variable (which can be used by the NC programmer to react to feedback from the edge computing system 3), providing an NC subroutine for the main program to call (e.g. via an EXTCALL function), providing an entire NC program to be started next, providing a machine action such as a STOP instruction to stop the printing process, a MOVEMENT instruction to move the print head 21 to a specific position (optionally along a given safe path), a PRINTERHEAD instruction to stop or affect the flow of material addition, or a FEEDRATE instruction to override the feed rate of the print head 21 movement, or a combination or sequence of multiple such actions.

[0139] Furthermore, the edge computing system 3 comprises a service space. In the service space there are one or more services for supplying and storing input data describing the actual 3D printing process in motion.

[0140] The service space may include a streaming data supply component that, for example, provides low-latency, near real-time (in another embodiment: deterministic real-time) data for consumption by components in the App space of the edge computing system 3 (see further below). This data 30, 31 provides an always up-to-date view of the machining process performed by the 3D printer 2 with very low latency (e.g., below 100 ms) in the information stream. These low latencies increase the reliability of the edge computing system 3 because they are calculated in a closed-loop control that includes input data transmission, situation classification, decision-making, and back-propagation of the resulting actions to the machine to improve the 3D printing process.

[0141] Optionally, the service space can include a persistent batch data view that can store incoming data persistently for a predefined amount of time (e.g., 3 months) and provide a batch data-based information source for historical data. This component can be used by components on the app space to benchmark current data with historical information, to derive reasoning results describing the current situation based on past situations and their relationships, and to explain the root causes of process and artifact degradation over time based on situation state changes within a certain time range.

[0142] The edge computing device includes an app space or application space that runs the logic for situation classification for workpiece quality assessment and decision making for in-process workpiece quality correction.

[0143] The App space includes a 3D printed artifact in-process quality analysis and correction function that manages and tracks the main logic control flow by accepting stream-based input data via a streaming data supply component and / or by querying a "persistent batch data view" whenever time-window-based historical data is needed for situation understanding. It interacts with other components in the App space to evaluate artifact quality in an iterative incremental situation understanding process by running analytical and / or machine learning algorithms with analytical algorithm configurations (also called analytical models) on data from the streaming data supply and / or the persistent batch data view. When / after the edge computing system 3 is first set up, the algorithms and algorithm configurations come from the "northbound" (see below) and can be updated to newer versions later.

[0144] The quality analysis and correction functions in the process of 3D printing workpieces are implemented by a geometric classification component, a quality inspection and / or classification component, and a decision component. Optionally, the decision component can be integrated into the quality inspection component.

[0145] The geometry classification may be configured to run at least one algorithm to classify 3D geometry in input data (e.g. camera image 31), wherein at least one algorithm is configured to run on fused input data 32 to identify 3D geometry in the data 31. The input data 30, 31 may include one or more of the following data types: G-code data of an NC program, machine data (e.g. axis movements with position setpoints and actual values, speed, acceleration, current, print head properties such as temperature, material flow, pressure, etc.), camera image 31, optional 3D laser scanner, optional analog and digital inputs from other sensors such as external vibration sensors, ultrasonic sensors, audio sensors and other sensors.

[0146] The quality classification is configured to run at least one algorithm to classify workpiece quality in input data 32 (e.g., camera images), wherein at least one algorithm is run on the 3D geometry identified by the geometry classification component and the fused input data to identify current (and future) workpiece quality in the data.

[0147] The decision can be configured to execute at least one algorithm having at least one algorithm configuration on the identified current and future workpiece qualities to determine whether a feedback action of the 3D printer 2 is required and which feedback action is required to affect the current printing process, thereby terminating printing or improving the printing quality by changing the 3D printing parameters along the upcoming sequence of printing steps of the remaining printing layers of the workpiece that affects the current suboptimal workpiece quality.

[0148] The "northbound" side of the edge computing system 3 represents a logical view of the physical computer network towards backend systems that allow the storage and analysis of (huge) amounts of data (so-called big data management and analysis systems). This part may contain a logical representation of public endpoints for data export and deployment artifact import tasks.

[0149] In one embodiment, the edge computing system 3 includes a deployment component importer component (entire app / algorithm / configuration) that is configured to import and / or install entire components, such as replacing old versions of components, algorithms, and / or algorithm configurations in the App space based on a specific 3D printer, available data sources, and target artifacts (depending on the scope and complexity of the replacement) to improve the performance of the edge computing system 3.

[0150] In one embodiment, the edge computing system 3 includes a streaming data exporter adapter component configured to forward data from the streaming data supply component to an external data storage capable of storing large amounts of data.

[0151] Optionally, the edge computing system 3 may include a batch data exporter component configured to export data from the persistent batch data view component to an external data storage capable of storing large amounts of data.

[0152] The edge computing system 3 may be connected to an intranet / Internet information technology (IT) infrastructure, such as IT systems in a customer's premises or an IT backend infrastructure in the cloud.

[0153] The IT infrastructure may include a data store (or data lake) where input data, such as merged data 32 from one or more edge systems, may be integrated, stored, and aggregated.

[0154] The IT infrastructure may include the selection of an inference algorithm, where the correct inference algorithm may be selected for classification and decision making on the edge computing system 3 , for example based on the 3D printer type and / or model, available camera and sensor data, and workpiece type.

[0155] The reasoning algorithms may include selection or combination from description logic reasoning, temporal logic reasoning, complex event processing, mathematical optimization, statistics, rule-based reasoning, automata theory, neural networks and / or simulation.

[0156] The IT infrastructure may include the definition / training of analytical model functionality, which allows, for example, the definition and / or training of analytical models in the form of algorithm configurations. In particular, it may allow the manual / semi-automatic or automatic definition or training of analytical models (i.e., configuration of analytical algorithms) based on the selected inference algorithm, 3D printer type and / or model, available camera and sensor data, and workpiece type. The resulting analytical configuration can be deployed to the App space of the edge computing system 3 together with the analytical algorithm (if not previously deployed in the required version) for classifying geometry, workpiece quality, and making decisions on whether to stop the 3D printer and how to repair / improve the quality of the workpiece.

[0157] In addition, packaging and deployment of, for example, reasoner-based or semantic-based reasoning classification algorithms and algorithm configuration components can be provided, which packages the deployment components into an app for the App space, or into a combination of an algorithm and a corresponding algorithm configuration, or into separate data packages of the algorithm and the algorithm configuration, depending on which components are changed in the backend and therefore need to be updated on the edge computing system 3, for example by incrementally updating separate or combined deployment components. Deployment can be accomplished by providing the packaged artifacts to a "deployment component importer" component on the edge computing system 3, which can then install / update the reasoning components in the App space.

[0158] In another embodiment, Figure 2 All components displayed in the App space in can be combined into a single logical module executed on the edge computing system 3.

[0159] Figure 3 Shows about Figure 2 The data flow of input and output data of the edge device system 3.

[0160] As southbound input, the edge device system 3 can acquire 3D printer 2 process data from the controller 23, including NC program code related to current operations, set points and actual values ​​and data from other sensors (first data stream 30), and acquire image data, such as video stream data from one or more digital cameras (second data stream 31).

[0161] As input from the northbound, the edge device system 3 may obtain one or more improvements to its logic, such as new or improved analysis algorithms and / or improved algorithm configurations.

[0162] In the southbound direction, the edge device system 3 can output feedback to the 3D printer controller 23. The feedback can include a proposal to change one or more parts of the NC program and / or a suggestion to change one or more set points for the next operation (e.g., the next printing step) and / or to bring the machine to a specific / predefined state (e.g., stop it).

[0163] In the northbound direction, the edge device system 3 can output data to the IT backend and clients, for example, for global KPI dashboards and reports and / or training of lyrical models used on the edge device system 3, such as through quality classification and / or decision components.

[0164] Situation understanding and decision making on the edge device system 3 includes functionality for classification of 3D object geometry and / or classification of workpiece quality and / or calculation of countermeasures / mitigation for poor workpiece quality.

[0165] In other words, the disclosed system and method allow to identify incorrect printing steps by analyzing 3D printer controller data and captured camera images during the actual process. In another embodiment, further data is used during this analysis process to even further improve the quality of this analysis inspection process: further analog and digital sensor inputs and 3D laser scanner data can be fused by logical combination and time correlation to the always same 3D printer tool path position and NC program line of the current printing step.

[0166] Furthermore, performance optimization is achieved through the reusability of geometric base equivalence classes and scalability of the training process (train once and reuse multiple times / training steps can be parallelized).

[0167] Furthermore, the proposed approach may allow to explain the root causes of potential printing quality issues or errors by tracing the overall image classification back to the individual decision processes of mapping the 3D printed geometry to the aggregated set of individual geometries, or sequentially mapping the individual geometries to the corresponding 3D printing steps during the printing process when the workpiece is created step by step using basic geometries. These geometries can be mapped to equivalence classes which can be defined in the definition phase (respectively trained in the training phase) and now used as individual or combined classifiers (based on reasoners) during the actual 3D printing process in the so-called reasoning phase.

[0168] Instead of classifying the camera snapshots in one go, the presently disclosed subject matter can classify each individual additive manufacturing step by mapping it to a probabilistic means matching predefined equivalence classes with the variability of geometric properties of some classes expressed by semantic properties like height, depth, or just a generic scaling factor.

[0169] Equivalence classes can be semantically defined in the form of description logics. This allows automatic consistency checking of the classification input to the reasoning phase by converting defined / trained assertions based on common geometric terms into mathematical class satisfiability problems. Similarly, the definition / training phase and the classification input can be optimized by automatically minimizing the number of equivalence classes according to the class equivalence / class inheritance axioms inherent to description logics, and applying automatic class satisfiability checks again. This allows identification and elimination of duplicate geometric equivalence classes, thereby reducing the number of equivalence classes in the classifier library, speeding up processing during the reasoning phase.

[0170] Figure 4 The inference phase is shown in more detail. In the definition and training phases, equivalence classes of geometric shapes / forms are defined ( Figure 4 ), and, for example, offline, train the (untrained) algorithm ( Figure 4 The training phase may include a phase for defining geometric equivalence classes. For example, the equivalence classes may be predefined equivalence classes stored in a database, such as a knowledge base, such as in the cloud or on a server in a workshop.

[0171] Machine Learning Algorithms ( Figure 4 The "algorithm" in the above description may be an untrained or pre-trained algorithm. As disclosed in the present invention, the machine learning algorithm may include a reasoner. In particular, the machine learning algorithm may include a classification algorithm, which may be based on a reasoner.

[0172] The reasoner uses descriptive logic and a predetermined dataset consisting of semantic descriptions of equivalence classes of geometric forms and shapes.

[0173] In other words, the machine learning algorithm can be based on a reasoner and can be trained on image data representing workpieces in production. It is trained to recognize in images: workpieces in production and (printable) geometric shapes and / or forms (cubes, cylinders, etc.), which are defined in equivalence classes and into which the workpieces can be decomposed.

[0174] During the inference phase (see Figure 4 ), applying a trained machine learning algorithm to image data, such as 3D printing image data during a process, and in particular to the second data stream 31 of the merged data stream 32, to perform a decomposition and classification of the detected form of the manufactured portion 22 of the workpiece.

[0175] Optionally, if the algorithm does not produce a clear result, further reasoning can be performed, for example interrupting the production process (in Figure 4 marked with a “crossed out hand” in the text).

[0176] When the 3D printer controller 23 runs / executes the numerical control program that moves the print head, it serves as a source of information. This program is usually provided in the form of a set of so-called "G-code" / program code instructions for a printed line with the following parameters: start and end position and thickness, for example, these parameters can now be used by the verification algorithm of the printed line, and using this information, the algorithm can correct the camera's observation based on the captured camera image and / or improve the classification of such printed line (comparing the camera image of the printed line with the corresponding G-code information and checking whether it was printed correctly). Therefore, the G-code input from the numerical control unit (controller) 23 of the 3D printer 2 is used to obtain processing instructions in order to keep this input consistent with the classification program when reasoning about the camera image. This allows higher accuracy than using only the camera input and provides a good trade-off for batch production of identical or very similar workpieces.

[0177] In the event of a printing error being detected, the algorithm can stop the printing process and save material and production time.

[0178] The subject matter disclosed in the present invention may allow explaining the cause of an error by tracing back to the first class (or first group of classes) matching an early printing fault through classification matching from aggregations of equivalent classes on a class sequence, which constitutes an aggregation with individual geometric attributes of these classes.

[0179] This method allows early fault detection and also improves classification accuracy, since classification can be performed multiple times (even continuously) during workpiece processing.

[0180] In short, the presently disclosed subject matter allows for higher accuracy and better performance automated workpiece quality control and decision support for automatic self-healing / improving workpiece quality during the 3D printing process.

[0181] The step-by-step process includes

[0182] • Motion controller data acquisition using G-code integration

[0183] • Camera stream-based data acquisition using image segmentation and object detection

[0184] • Temporal and causal data fusion for contextualized controller, camera and sensor inputs

[0185] • Situation understanding through semantic classification of operational scenarios using 3D geometric equivalence classes

[0186] • Automated in-process decision making using near real-time or real-time feedback to 3D printing controllers

[0187] •Automatic, in-process repair of printed parts

[0188] •Use model-based correlations between controller, external sensor, and video data to provide transparency and decision support for improved engineering processes.

[0189] The subject matter disclosed in the present invention can stop production at the first error and / or automatically / semi-automatically apply corrective actions during the 3D printing process, thereby saving costs (not only the production costs of faulty products, but also the costs of recalls), and additionally improve production planning accuracy, product quality, production reliability, and reduce the risk of product failure when used in assembled composite products such as aircraft, automobiles, trains, etc.

[0190] With corrective actions, the system can take immediate action to restore the workpiece to full or partial quality, either as early as when an early degradation of the workpiece quality is detected or after an actual degradation is detected while still in processing, thus saving huge costs in series production or the production of small-size, high-value workpieces and thus reducing the amount of waste produced.

[0191] The system and method disclosed in the present invention also provide automatic feedback to improve the PLM process, provide decision support for modifying G-code, thereby improving 3D printing quality and reducing the need for countermeasures in the future process.

[0192] The above embodiments of the present invention are presented for the purpose of illustration rather than limitation. In particular, the embodiments described with respect to the drawings are only a few examples of the embodiments described in the introduction. The technical features described with respect to the system can be applied to enhance the method disclosed in the present invention, and vice versa.

Claims

1. A system for assisting an additive manufacturing process of a workpiece (22), wherein: The additive manufacturing process includes a plurality of printing steps (P1, P2, P3, P4), wherein the system includes - memory, storing machine executable components, - a processor operatively coupled to said memory and configured to execute said machine-executable components, and - an interface configured to receive a first data stream (30) and a second data stream (31), wherein the first data stream (30) and the second data stream (31) are generated while manufacturing the workpiece (22) by the additive manufacturing process, wherein The first data stream (30) comprises first data provided by a control device (23) controlling the additive manufacturing process, the first data representing a first form of a produced part of the workpiece (22), wherein the first form consists of a first geometric shape associated with the printing step (P1, P2, P3, P4) of the additive manufacturing process, and The second data stream (31) comprises second data provided by an image sensor system (24) monitoring the additive manufacturing process, the second data representing a second form of the produced part of the workpiece (22), wherein: The machine executable components include: a data fusion component configured to receive the first data stream (30) and the second data stream (31) and generate a merged data stream (32) by associating the first data stream with the second data stream such that a common time reference exists between the first data stream and the second data stream in the merged data stream, a geometric classification component configured to determine the second form of the produced portion of the workpiece based on the merged data stream and to decompose the second form of the produced portion of the workpiece into at least one second geometric shape, and - a quality inspection component configured to compare the second geometry with the first geometry in order to assess the quality of the produced portion of the workpiece.

2. The system according to claim 1, wherein: The geometry classification component is configured as - receiving said combined data stream (32), and - detecting the produced part (22) of the workpiece in the second data stream of the merged data stream.

3. The system according to claim 1 or 2, wherein: The geometry classification component is configured as - Matching the second geometry to the printing step of the additive manufacturing process.

4. The system according to any one of claims 1 to 3, wherein: The geometric classification component comprises a machine learning algorithm, in particular a classification algorithm, comprising a reasoner based on a descriptive logic and using a predetermined set comprising a semantic description of geometric properties and relations between geometric forms and / or shapes according to the descriptive logic.

5. The system according to claim 4, wherein: The machine learning algorithm is trained on a predefined training data set representing a form of produced parts of a workpiece.

6. The system according to claim 5 in combination with claim 3, wherein: The predefined training data set represents a decomposition of the form of the produced part of the workpiece into geometric shapes, and the machine learning algorithm is used to determine at least one decomposition of the second form of the produced part of the workpiece into second geometric shapes.

7. A system according to any one of claims 1 to 6, wherein: The machine-executable components also include an action feedback component configured to provide a suggestion of at least one action to be performed on an additive manufacturing machine performing the additive manufacturing process of the workpiece.

8. The system according to claim 7, wherein: The at least one action includes stopping the additive manufacturing process.

9. The system according to claim 8, wherein: The at least one action comprises performing a corrective action after stopping the additive manufacturing process, wherein in particular the corrective action is associated with material defects and workpiece quality degradation and / or comprises a corrected machining program executable by the control device (23).

10. The system according to any one of claims 7 to 9, wherein: The action feedback component is further configured to suggest at least one corrective action to the additive manufacturing process and cause the control device (23) to perform the at least one corrective action.

11. The system according to claim 10, wherein: The at least one corrective action is derived from semantic equivalence classes, wherein the semantic equivalence classes are preferably defined based on input from a CAD / CAM system capable of decomposing the workpiece into individual geometric shapes.

12. The system according to any one of claims 1 to 11, wherein: The clock of the control device controlling the additive manufacturing process is the common time reference.

13. The system according to any one of claims 1 to 12, wherein: The second data (31) includes image data, in particular a video data stream, a laser scanning data stream, or an ultrasound data stream.

14. The system according to any one of claims 1 to 13, wherein: The data fusion component is configured to establish a correspondence between the first form and the second form, so as to facilitate comparison of the execution result of a predetermined line of the partial program executed by the control unit with the frame image in the image data stream.

15. The system according to any one of claims 1 to 14 further comprises a decision support component configured to provide decision support in the form of notifications or warnings to an operator at a safe point selected by the system and / or in the form of step-by-step manual correction instructions from the perspective of the 3D printing process or after the manufacturing process of the workpiece is completed.

16. A computer-aided method for additive manufacturing of a workpiece, comprising: - while producing the workpiece (22) by an additive manufacturing process comprising a plurality of printing steps (P1, P2, P3, P4), receiving a first data stream (30) and a second data stream (31), wherein The first data stream (30) comprises first data provided by a control device (23) for controlling the additive manufacturing process, the first data representing a first form of a produced part (22) of the workpiece, wherein: The first form consists of a first geometric shape associated with the printing step (P1, P2, P3, P4) of the additive manufacturing process, and The second data stream (31) comprises second data provided by an image sensor system (24) monitoring the additive manufacturing process, the second data representing a second form of the produced portion (22) of the workpiece, wherein - generating a merged data stream (32) by associating the first data stream (30) with the second data stream (31) such that there is a common time reference between the first data stream and the second data stream in the merged data stream (32), - determining the second form of the produced part (22) of the workpiece based on the merged data stream (32) and at least one decomposition of the second form of the produced part of the workpiece into second geometric shapes, and - comparing said second geometry with said first geometry in order to assess the quality of said produced portion of said workpiece.

17. A computer program comprising instructions which, when executed by a computing device, cause the computing device to perform the steps of the method according to claim 16.

18. A computer readable medium comprising a computer program according to claim 17.

Images