An additive manufacturing system forming monitoring system and method based on digital twin

Abstract

The present invention discloses a forming monitoring system and method for an additive manufacturing system based on digital twin, including a CCD high-speed industrial camera for full-range monitoring on the top of the machine tool, a CCD high-speed industrial camera for side monitoring of the arc gun, a three-dimensional laser profilometer, a strip laser emitter, a sound acquisition sensor, an infrared thermal imaging camera, a camera fixture with multiple degrees of freedom, and an annular fixing device. Corresponding data acquisition methods and steps are proposed, and corresponding data optimization algorithms and processing flows are further designed, providing a strong basis for constructing a monitoring and control system with multiple framework modes. At the same time, based on a variety of collected real physical data, digital twin data is formed, a digital twin model in the virtual space is constructed, the whole process of processing and manufacturing is monitored and controlled in real time, the closed-loop feedback control of the manufacturing system is realized, the forming accuracy of parts is improved, and the input costs of manpower and material resources are saved for the design and manufacture of new products.

Description

Technical Field

[0001] The present invention relates to the technical field of arc additive manufacturing, and particularly to a forming monitoring system and method for an additive manufacturing system based on digital twin. Background Art

[0002] Wire and Arc Additive Manufacture (WAAM) technology has the characteristics of high deposition rate, rapid forming, and the ability to manufacture medium and large-sized complex structure parts, which greatly saves manufacturing costs and reduces labor consumption. Therefore, WAAM technology is widely used in the manufacture of various medium and large-sized complex parts, especially in the manufacture of aerospace complex structure parts. Most of the parts in the aerospace field are large-sized and complex parts, and the manufacturing time often takes several days or even months. Therefore, the quality monitoring of the formed parts and the simulation and control of the design data for manufacturing are particularly important, especially the monitoring of the forming process of the formed parts.

[0003] As the number of stacked layers increases and heat is continuously input, the heat dissipation rate of the formed part becomes slow, and the solidification time of the molten pool for forming is extended, resulting in difficult control of the bead shape. The ideal goal of arc additive manufacturing forming is that the bead size is exactly the same as the size of the layer-by-layer slicing model. However, WAAM technology has the characteristics of weak arc directivity, high heat input, relatively long stacking process cycle, and rapid cooling; at the same time, the heat source has a long traversal time in complex paths, and for the formed part that undergoes complex thermal cycles, mechanical property problems such as residual stress are inevitably generated inside. It can be seen that if the molten pool and welding parameters are monitored in real time during the manufacturing process, the process parameters are adjusted at any time, and the manufacturing conditions with reduced heat accumulation are achieved, the bead size can always maintain relative consistency with the pre-set size, and the quality of the formed part will be further improved.

[0004] Digital twin is a technology that synthesizes the attributes of multiple disciplines, multiple physical quantities, multiple scales, and multiple probabilities, uses data such as physical models and sensor updates to complete mapping in the digital virtual space, forms a twin relationship between virtual equipment and real equipment, and thus realizes the full life cycle management of physical equipment.

[0005] Existing patents such as Patent CN112084570A propose an additive manufacturing coupled digital twin ecosystem for creating an optimized manufacturing process to fabricate repair-defined parts; a replacement model corresponding to a physics-based model of the defined part is updated in real time;

[0006] Patent CN111795977A provides an online real-time monitoring system for multiple monitoring devices in metal additive manufacturing, aiming to solve the problem that the existing metal additive manufacturing monitoring devices cannot timely discover the causes of processing defects due to incomplete information acquisition;

[0007] Patent CN112162519A provides a composite machine tool digital twin monitoring system, which simplifies the monitoring process of the operation of the composite machine tool, improves the monitoring accuracy of the machine tool system, and realizes the active predictive maintenance of the operation of the composite machine tool.

[0008] In the descriptions of the above various patents, systems based on digital twin technology can basically well solve a large number of monitoring and regulation problems, as well as the problem of real-time online optimization of corresponding processes to improve production efficiency. However, there are few systems and methods for monitoring and regulating the forming process of metal additive manufacturing using digital twin, especially those related to the intelligent system of arc additive manufacturing. There is currently no forming monitoring system and method for additive manufacturing system based on digital twin on the market.

[0009] In response to the requirements such as the deformation of the formed parts and the inability to guarantee the mechanical properties caused by various factors in the WAAM forming process, innovative designs are carried out. Summary of the Invention

[0010] The purpose of the present invention is to provide a forming monitoring system and method for an additive manufacturing system based on digital twin, so as to solve the problems in the above-mentioned background technology that too little information on the manufacturing forming process is collected in the traditional arc additive manufacturing process, the accuracy is difficult to guarantee, and the entire additive manufacturing process and the full life cycle management of the finished product cannot be visualized.

[0011] To achieve the above purpose, the present invention provides the following technical solution: A forming monitoring system for an additive manufacturing system based on digital twin. The hardware part of the system includes an alarm module, a top CCD high-speed industrial camera, a five-axis machine tool vertical telescopic mechanism, a three-dimensional laser profile scanner, an infrared camera, a sound sensor, a ring-shaped fixing device, an arc gun, a substrate, a strip-shaped laser emitter, and an arc side monitoring camera;

[0012] The top CCD high-speed industrial camera, the three-dimensional laser profile scanner, the infrared camera, the sound sensor, the strip-shaped laser emitter, and the arc side monitoring camera are installed around the arc gun and the substrate in cooperation with the ring-shaped fixing device, and simultaneously monitor the additive welding forming process. Based on the monitored and collected real physical data, digital twin data is formed, and a digital twin model in the virtual space is constructed to monitor and regulate the entire process of machining and manufacturing in real time, and realize the closed-loop feedback control of the manufacturing system.

[0013] Preferably, the top CCD high-speed industrial camera is used to monitor the arc light change situation in the full range, and at the same time, through the corresponding visual image processing algorithm, the arc light change situation in the manufacturing process is processed in real time, and a mathematical correlation model is established between the arc light change situation and the part forming process.

[0014] The visual image processing algorithm includes grayscale processing, threshold segmentation, and shape screening of the collected images; filtering the feature distribution at the strong arc light to obtain the image of the frequency flash arc in the five-axis machine tool processing chamber. Further, the image is split into RGB channels and further HSV color transformation is performed. Further, the pixels of each channel are successively subjected to image difference operation with the pixels at the corresponding positions of the previous frame of the original image to obtain the corresponding T value, and the size of the T value is judged against a preset threshold. If the T value is greater than the threshold, the current image is subjected to first-level wavelet transform processing to extract low-frequency signals and filter high-frequency signals, all valid low-frequency signals are stored, and are transmitted to the central system through the transmission module for further decision-making and judgment.

[0015] Preferably, for the side arc monitoring camera, firstly, the arc light is used to irradiate the molten pool to form a passive vision monitoring state; secondly, the strip laser emitter is used to irradiate the molten pool to form an active laser vision monitoring state, and the main and passive visions are combined for information collection. The key lies in the usage methods and process steps of the passive and active visions.

[0016] The usage methods and process steps of the main and passive visions include firstly using a CCD high-speed industrial camera to collect the arc light image of the current arc, calculating the current threshold using the image processing algorithm, and comparing it with the preset threshold. Further, based on the comparison result, a decision is made on whether to continue using passive vision or laser active vision for image collection in the next step. Finally, the image data is transmitted to the central processing system through the transmission module.

[0017] With the above technical solution, since the image information obtained by the main and passive vision monitoring system in cooperation with the side arc monitoring camera is limited to the molten pool and the welding torch, it may ignore that the ambient light may also have a certain relationship with the arc light. At the same time, in order to create a more realistic virtual simulation manufacturing environment, it is quite important to collect the environmental factors in the processing machine tool chamber. Further, the top CCD high-speed industrial camera collects image information in the full range, and a corresponding visual image processing algorithm is designed to process the changes of the arc light during the manufacturing process in the processing machine tool in real time, providing strong data support for establishing the mathematical model of the change of the arc light in the processing machine tool and the part forming process.

[0018] Preferably, the infrared camera is used to monitor the temperature change of the molten pool during the manufacturing process in real time. By analyzing the relationship between the temperature data and the forming quality of the molten pool, an association model with temperature and quality is established, and corresponding control strategies are adopted for regulation. At the same time, the relationship model between the collected temperature information and the forming quality is recorded in the data storage module. The central processing system makes a comprehensive judgment and decision based on the actual working conditions and the previous data, and synchronously adjusts the forming welding process parameters and the forming process and steps.

[0019] Adopting the above technical solution, by collecting the data of the temperature change of the molten pool during the forming process, a mathematical correlation model between temperature and forming quality is established. At the same time, combined with other welding process parameters, corresponding control strategies are selected. Further, the control strategy module includes adjusting the rhythm of welding manufacturing, changing the stacking path of manufacturing welding, changing the wire feeding speed of the welding wire and the moving speed of the welding torch, and starting corresponding manual active intervention means, etc. Further, the manual active intervention means include manufacturing a cold source by a thermoelectric device to cool the substrate, transporting a cooling gas to cool the molten pool, etc.

[0020] Preferably, multi-degree-of-freedom camera fixtures are provided at the installation positions of the top CCD high-speed industrial camera, the infrared camera and the arc side monitoring camera. Through clamping by the multi-degree-of-freedom camera fixture, the shooting angle is adjusted in real time to adapt to the complex situation of additive manufacturing process monitoring.

[0021] The multi-degree-of-freedom camera fixture includes a large "U" - shaped plate and a small "U" - shaped plate, and the large "U" - shaped plate is located outside the small "U" - shaped plate. The large "U" - shaped plate is the support plate of the slider, and the small "U" - shaped plate is mainly for clamping the end of the camera. A sliding groove for the slider to slide is left at the middle position of the large "U" - shaped plate to adjust the left - right and front - back positions of the camera by the sliding of the slider. The large "U" - shaped plate and the small "U" - shaped plate are connected by a spherical hinge connecting rod. The small "U" - shaped plate and the clamped camera have a pitching adjustment function, and the upper end of the large "U" - shaped plate is connected to the machine tool through a connecting fixing block, and the connecting fixing block also has a left - right rotation adjustment function. Based on the adjustment of each degree of freedom described, the clamped camera can be adjusted omnidirectionally in space.

[0022] Adopting the above technical solution, it is applicable to the clamping of various cameras, and the overall angle can be adjusted left - right and up - down. Among them, at the contact end of clamping the camera, it is connected to the overall part through horizontal sliding and spherical hinge; it can achieve fine adjustment of the angle and position in all directions in space, and realize the adjustment effect of various positions and angles of the camera.

[0023] Preferably, the three - dimensional laser profilometer is vertically installed above the formed part to monitor the geometric dimensions of the molten pool forming. Through corresponding image - processing algorithms, the process of defect formation and various defects of the formed part are monitored quickly. The three - dimensional information of the molten pool extracted is compared with the three - dimensional information of the ideal molten pool effect, the changes of the corresponding process parameters during the defect - generating process are analyzed, and the corresponding relationship between the defects and the process parameters is recorded and stored in the data storage module. In each update of the twin data, the central processing system automatically adjusts the corresponding manufacturing process parameters according to the recorded relationship between the defect type and the process parameters.

[0024] With the above technical solution, the three-dimensional laser contour scanner can be used to collect the three-dimensional contour information of the formed part, including the three-dimensional point cloud image data during and after the forming process. Design corresponding point cloud processing algorithms to detect and classify the defects of the formed parts with quality problems. The classification of defects mainly includes bead collapse, porosity in the bead, slag inclusion in the weld, crack in the weld, and discontinuity of the weld, etc. Further, during the forming process of each formed part, the welding parameters and environmental influencing factors during the forming process are recorded throughout the process, that is, the whole process of the generation of each type of defect will be recorded and stored in the data storage module. When manufacturing parts next time, the data in the data storage module is compared, and the forming welding parameters for the new time are adjusted in real time to continuously improve the accuracy of the manufactured parts and avoid more defects in subsequent manufacturing.

[0025] Preferably, the sound sensor is installed on the side of the arc gun and is used to collect the sound signals during the arc additive manufacturing process in real time. The collected sound signals are processed by corresponding sound signal algorithms to filter out the interfering sound signals and extract the effective sound signals, and a mathematical correlation model reflecting the part forming process and the sound signals is established and stored in the data storage module as a kind of marker data for the central processing system to make comprehensive decision-making and judgment for regulation.

[0026] With the above technical solution, by collecting the arc sound during the manufacturing process, analyzing and processing the corresponding sound signals, and according to the analysis results, an appropriate filter is selected for noise removal processing to obtain the relationship between the corresponding sound signals and the bead geometry information. The filter includes various conventional filtering algorithms and also includes adaptive filters, such as the adaptive filter based on neural network.

[0027] Preferably, the ring-shaped fixing device is used for the sensor that needs to be fixed on the side of the arc gun. The sensor can be adjusted in a circumferential position around the arc gun, and the ring-shaped fixing device can be adjusted vertically relative to the vertical telescopic mechanism of the five-axis machine tool to a large extent, realizing the adjustment effect of the sensor in various positions in space;

[0028] The ring-shaped fixing device is mainly composed of an upper ring and a lower ring, and the upper ring and the lower ring are connected by flat side plates. The upper ring makes a circular motion around the end of the flat side plate, thereby realizing the relative position change of the upper ring and the lower ring in space, and thus realizing the change of the distance between the sensor and the arc gun. Grooves are provided on the inner sides of the upper ring and the lower ring, and the grooves are slidably connected to the sensor, thereby realizing the adjustment of rotation around the central axis of the ring. The ring-shaped fixing device is connected to the vertical telescopic mechanism of the five-axis machine tool through the side of the mounting plate, and the ring-shaped fixing device can make a vertical movement through the vertical telescopic mechanism of the five-axis machine tool.

[0029] With the above technical solution, the framework layout installation of multiple sensors is realized.

[0030] A forming monitoring method for an additive manufacturing system based on digital twins maps the data collected by sensors into digital space to form twin data, and further constructs a digital twin model for predictive control and online real-time modification of forming processing parameters, thereby achieving the manufacturing effect from real data to digital space, and then from digital virtual space to real space, so as to realize all-round monitoring of the full life cycle management of additively formed parts.

[0031] Preferably, the digital twin model collects the data collected by the sensor to form twin data, and further constructs the digital twin model;

[0032] The data twin model has a complete digital mapping relationship with the physical space entity equipment. The virtual model controlled by the computer can synchronously monitor the working status of the physical equipment and issue new manufacturing instructions for real-time parameter control and manufacturing;

[0033] The digital twin model has a variety of complex mixed data. The embedded intelligent algorithm can perform large-scale data operations based on the complex mixed data to update the information of the twin data in real time and update the digital twin model at the same time, so that the digital twin model can be iteratively updated in real time according to the environmental changes in the physical space, so that the model is more effective.

[0034] The real-time updated digital twin model readjusts the equipment working parameters based on the processing of real-time data by intelligent algorithms, and quickly delivers them to the composite additive and subtractive manufacturing system through the data transmission module to achieve closed-loop control manufacturing of the entire workpiece process.

[0035] By adopting the above technical solution, the digital twin model corrects the manufacturing parameters by using machine learning and other algorithms for online decision-making, and uses one of the intelligent algorithms to explain the working process, such as the prediction module of the neural network model for parameter correction. Based on the data set collected by multiple sensors, the association model between the welding process parameters and the weld bead forming dimensions is trained online, and the optimal process parameters are generated. The optimal process parameters are fed back to the five-axis composite additive and subtractive manufacturing system through the data transmission module, and the process parameters are readjusted for manufacturing. The parameters input by the neural network model include variables such as wire feeding speed, welding gun movement speed, welding current and voltage, arc flash frequency during welding, and sound generated by the arc during welding. The output is the geometric dimensions of the weld, including weld bead width and height. The variables output by the neural network are not limited to the geometric dimensions of a single-layer weld, but also include the layer height and layer width of multiple layers and various factors that may affect the manufacturing process.

[0036] Compared with the prior art, the beneficial effects of the present invention are: the WAAM forming process monitoring system and method of the five-axis composite additive and subtractive manufacturing system based on digital twin,

[0037] 1. The present invention collects information about the manufacturing process based on a multi-frame sensor layout, as well as various environmental factors in the additive manufacturing environment, to build a digital twin model of the entire arc additive manufacturing process, so as to monitor and manage the entire life cycle of the formed parts, further improve the manufacturing accuracy of additive manufacturing, avoid unnecessary waste of manpower and material resources, and realize a more intelligent manufacturing system;

[0038] 2. By adjusting the camera fixture with multiple degrees of freedom, the camera can present a variety of shooting postures in space to adapt to the shooting angle requirements of additive manufacturing in complex environments; at the same time, a multi-frame sensor real-time data acquisition and monitoring system is built to meet the data acquisition and monitoring requirements under actual complex environment operations, and a variety of implementation methods and process steps are given; it solves the problems of too little information collection in the traditional arc additive manufacturing forming process, difficulty in ensuring the accuracy of formed parts, inability to monitor the entire process of formed parts, lack of multi-directional adjustment fixtures for sensors, and inability to fully visualize the additive manufacturing process and the entire life cycle management of finished products. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the overall layout of multi-sensor monitoring of the present invention;

[0040] Figure 2 This is a general diagram of the multi-sensor data collection and processing process of the present invention;

[0041] Figure 3 A flow chart for active and passive vision use of the present invention;

[0042] Figure 4 This is a flow chart of the arc flash processing algorithm of the present invention;

[0043] Figure 5 A schematic diagram of a method for selecting a temperature control strategy of the present invention;

[0044] Figure 6 This is a schematic diagram of the temperature control strategy module of the present invention;

[0045] Figure 7 This is a flow chart of sound signal processing of the present invention;

[0046] Figure 8 This is a schematic diagram of a sound filtering processing module of the present invention;

[0047] Figure 9 This is a schematic diagram of a welding defect classification module of the present invention;

[0048] Figure 10 This is a schematic diagram of the calling flow of the defect classification module of the present invention;

[0049] Figure 11 This is a schematic diagram of the structure of the multi-degree-of-freedom camera fixture of the present invention;

[0050] Figure 12 Schematic diagram of the ring fixing device structure of the present invention;

[0051] Figure 13 Schematic diagram of the data storage module of the present invention;

[0052] Figure 14 Schematic diagram of the working process of the digital twin model of the present invention;

[0053] Figure 15 Schematic diagram of the neural network model training of the present invention.

[0054] In the figure: 1. Alarm module; 2. Top CCD high-speed industrial camera; 3. Five-axis machine tool vertical telescopic mechanism; 4. Three-dimensional laser profile scanner; 5. Infrared camera; 6. Sound sensor; 7. Ring fixing device; 71. Upper ring; 72. Lower ring; 73. Flat side plate; 74. Notch; 75. Mounting plate; 8. Arc gun; 9. Substrate; 10. Strip laser emitter; 11. Arc side monitoring camera; 12. Multi-degree-of-freedom camera fixture; 121. Large "U"-shaped plate; 122. Small "U"-shaped plate; 123. Slide block; 124. Slide groove; 125. Ball hinge connecting rod; 126. Connecting and fixing block. Specific embodiments

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0056] Please refer to Figures 1-15 , the present invention provides a technical solution:

[0057] An additive manufacturing system forming monitoring system based on digital twin. The hardware part of the system includes an alarm module 1, a top CCD high-speed industrial camera 2, a five-axis machine tool vertical telescopic mechanism 3, a three-dimensional laser profile scanner 4, an infrared camera 5, a sound sensor 6, a ring fixing device 7, an arc gun 8, a substrate 9, a strip laser emitter 10, and an arc side monitoring camera 11. The overall layout of the system is as Figure 1 shown;

[0058] The top CCD high-speed industrial camera 2, 3D laser profile scanner 4, infrared camera 5, sound sensor 6, strip laser emitter 10, and arc side monitoring camera 11 are installed around the arc gun 8 and the substrate 9 in cooperation with the annular fixing device 7 to monitor the additive welding forming process at the same time. Based on the monitored and collected real physical data, digital twin data is formed, a digital twin model in the virtual space is constructed, the whole process of manufacturing is monitored and regulated in real time, and the closed-loop feedback control of the manufacturing system is realized. The multi-sensor data acquisition and processing workflow is as Figure 2 shown.

[0059] For the arc side monitoring camera 11, firstly, the arc light is used to irradiate the molten pool to form a passive vision monitoring state; secondly, the strip laser emitter 10 is used to irradiate the molten pool to form an active laser vision monitoring state. The information is collected by combining the active and passive visions. The key lies in the usage methods and process steps of the passive and active visions;

[0060] The usage methods and process steps of the active and passive visions include: firstly, the CCD high-speed industrial camera is used to collect the arc light image of the current arc, and the image processing algorithm is used to calculate the current threshold and compare it with the preset threshold. Further, based on the comparison result, a decision is made on whether to continue using passive vision or laser active vision for image acquisition in the next step. Finally, the image data is transmitted to the central processing system through the transmission module. The specific workflow is as Figure 3 shown.

[0061] The top CCD high-speed industrial camera 2 is used to monitor the arc light changes in the full range, and at the same time, the arc light changes in the manufacturing process are processed in real time through the corresponding visual image processing algorithm, and a mathematical correlation model is established between the arc light changes and the part forming process.

[0062] The visual image processing algorithm includes: grayscale processing, threshold segmentation, and shape screening of the collected images; filtering the feature distribution at the strong arc light to obtain the image of the frequency flash arc in the five-axis machine tool processing room. Further, the image is split into RGB channels and further HSV color transformation is performed. Further, the pixels of each channel are sequentially subjected to image difference operation with the pixels at the corresponding positions of the previous frame of the original image to obtain the corresponding T value, and the size of the T value and the preset threshold are judged. If the T value is greater than the threshold, the current image is subjected to the first-level wavelet transform processing, the low-frequency signal is extracted and the high-frequency signal is filtered, all the effective low-frequency signals are stored, and are transmitted to the central system through the transmission module for further decision-making judgment. The specific algorithm steps are implemented as Figure 4 shown.

[0063] Since the image information obtained by the active and passive vision monitoring system in cooperation with the arc side monitoring camera 11 is limited to the molten pool and the welding torch, it may ignore the possible relationship between the ambient light and the arc light. At the same time, in order to create a more realistic virtual simulation manufacturing environment, it is quite important to collect the environmental factors in the machining workshop. Further, the top CCD high-speed industrial camera collects image information in the whole range, and designs corresponding visual image processing algorithms to process the changes of the arc light during the manufacturing process in the machining workshop in real time, providing strong data support for establishing the mathematical model of the change of the arc light in the machining workshop and the part forming process.

[0064] The infrared camera 5 is used to monitor the temperature change of the molten pool during the manufacturing process in real time. By analyzing the relationship between the temperature data and the forming quality of the molten pool, an association model with temperature and quality is established, and corresponding control strategies are adopted for regulation. At the same time, the relationship model between the collected temperature information and the forming quality is recorded in the data storage module. The central processing system makes a comprehensive judgment and decision according to the actual working conditions and the previous data, and synchronously adjusts the forming welding process parameters and the forming process and steps.

[0065] By collecting the data of the temperature change of the molten pool during the forming process, a mathematical association model between temperature and forming quality is established. At the same time, combined with other welding process parameters, corresponding regulation strategies are selected. The specific working process of selecting the regulation strategy is as Figure 5 shown. Further, the regulation strategy module includes adjusting the rhythm of welding manufacturing, changing the stacking path of manufacturing welding, changing the wire feeding speed of the welding wire and the moving speed of the welding torch, and starting corresponding manual active intervention means, etc. Further, the manual active intervention means include manufacturing a cold source by a thermoelectric device to cool the substrate, transporting a cooling gas to cool the molten pool, etc. The specific regulation strategy module is as Figure 6 shown.

[0066] The sound sensor 6 is installed on the side of the arc gun 8 and is used to collect the sound signal during the arc additive manufacturing process in real time. The collected sound signal is processed by a corresponding sound signal algorithm to filter out the interfering sound signal and extract the effective sound signal, and a mathematical association model reflecting the part forming process and the sound signal is established, which is stored in the data storage module as a kind of marker data for the central processing system to make a comprehensive decision and judgment for regulation.

[0067] By collecting the arc sound signal during the manufacturing process, analyzing and processing the corresponding sound signal, according to the analysis result, an appropriate filter is selected for noise removal processing, and the relationship between the corresponding sound signal and the bead geometry information is obtained. The working process of the sound signal processing is as Figure 7 shown, where the filter includes various conventional filtering algorithms and also includes adaptive filters, such as the adaptive filter based on neural network. The sound signal filter module is as Figure 8as shown

[0068] The three-dimensional laser profile scanner 4 is vertically installed above the formed part to monitor the geometric dimensions of the molten pool forming. Through corresponding image processing algorithms, it monitors the process of defect formation and quickly detects various defects of the formed part. It compares the extracted three-dimensional information of the molten pool with the three-dimensional information of the ideal effect, analyzes the changes in the corresponding process parameters during the defect generation process, and records the corresponding relationship between the corresponding defects and process parameters and stores it in the data storage module. In each update of the twin data, the central processing system automatically adjusts the corresponding manufacturing process parameters according to the recorded relationship between the defect type and process parameters.

[0069] The three-dimensional laser profile scanner 4 can be used to collect the three-dimensional contour information of the formed part, including the three-dimensional point cloud image data during and after the forming process. Design corresponding point cloud processing algorithms to detect and classify the defects of the formed parts with quality problems. The classification of defects mainly includes defects such as bead collapse, pores in the bead, slag inclusions in the weld, cracks in the weld, and weld discontinuity. The defect classification module for manufacturing the formed part is as Figure 9 as shown. Further, during the forming process of each formed part, the welding parameters and environmental influence factors during forming are recorded throughout the process, that is, the whole process of the generation of each type of defect will be recorded and stored in the data storage module. When manufacturing parts the next time, the data in the data storage module is compared, and the forming welding parameters for the new time are adjusted in real time to continuously improve the accuracy of the manufactured parts and avoid more defects in subsequent manufacturing. The specific implementation steps and work flow are as Figure 10 as shown

[0070] Multi-degree-of-freedom camera fixtures 12 are installed at the installation positions of the top CCD high-speed industrial camera 2, the infrared camera 5, and the arc side monitoring camera 11. They are clamped by the multi-degree-of-freedom camera fixture (12) to adjust the shooting angle in real time to adapt to the complex situation of additive manufacturing process monitoring;

[0071] The multi-degree-of-freedom camera fixture 12 includes a large "U"-shaped plate 121 and a small "U"-shaped plate 122. The large "U"-shaped plate 121 is located outside the small "U"-shaped plate 122. The large "U"-shaped plate 121 is the support plate for the slider 123, and the small "U"-shaped plate 122 is mainly used to clamp the end of the camera. A chute 124 for the slider 123 to slide is left at the middle position of the large "U"-shaped plate 121, which is used for the slider 123 to slide to adjust the left-right and front-back positions of the camera. The large "U"-shaped plate 121 and the small "U"-shaped plate 122 are connected by a ball hinge connecting rod 125. The small "U"-shaped plate 122 and the clamped camera have a pitching adjustment function. The upper end of the large "U"-shaped plate 121 is connected to the machine tool through a connecting fixing block 126, and the connecting fixing block 126 also has a left-right rotation adjustment function. Based on the adjustment of each degree of freedom described above, the clamped camera can be adjusted omnidirectionally in space. The schematic diagram of the multi-degree-of-freedom camera fixture 12 is as follows Figure 11 as shown

[0072] It is applicable to the clamping of various cameras, and the whole can adjust the angle left and right and up and down. Among them, at the contact end of the clamped camera, by laterally sliding and connecting with the overall part through a ball hinge, fine adjustment of the angle and position in all directions in space can be achieved, realizing the adjustment effect of various positions and angles of the camera

[0073] The ring-shaped fixing device 7 is used for the sensor that needs to be fixed on the side of the arc gun 8. The sensor can make a circumferential position adjustment around the arc gun 8. The ring-shaped fixing device 7 can make a large vertical adjustment relative to the five-axis machine tool vertical telescopic mechanism 3, realizing the adjustment effect of the sensor in various positions in space

[0074] The ring-shaped fixing device 7 is mainly composed of an upper ring 71 and a lower ring 72, and the upper ring 71 and the lower ring 72 are connected by a flat side plate 73. The upper ring 71 makes a circular motion around the end of the flat side plate 73, thereby realizing the relative position change of the upper ring 71 and the lower ring 72 in space, and thus realizing the change of the distance between the sensor and the arc gun 8. Notches 74 are opened on the inner sides of the upper ring 71 and the lower ring 72, and the notches 74 are slidably connected to the sensor, thereby realizing the adjustment of rotation around the central axis of the ring. The ring-shaped fixing device 7 is connected to the five-axis machine tool vertical telescopic mechanism 3 through the side of the mounting plate 75. The ring-shaped fixing device 7 can make a vertical movement through the five-axis machine tool vertical telescopic mechanism 3, realizing the installation and setting of the frame layout of multiple sensors. The specific ring-shaped fixing device 7 is as Figure 12 shown

[0075] A method for monitoring the forming process of an additive manufacturing system based on digital twin, which maps the data collected by sensors into the digital space to form twin data, and further constructs a digital twin model for predicting and regulating, and online real-time modifying the forming process parameters, so as to achieve the manufacturing effect from real data to the digital space and then from the digital virtual space to the real space, in order to realize the full-life cycle management of comprehensively monitoring the additive formed parts.

[0076] The results after various signal processes are stored in the data module as a new data set, and correspond one by one to the geometric information of various formed parts, for the central processing system to make comprehensive decision-making judgments. Further explanation, the proposed data storage module is not limited to the listed signal processes. This module is just a framework for storing the results of multi-signal data, and can further expand the signal data to more comprehensively describe various influencing factors of forming. The specific framework data storage module is as Figure 13 shown.

[0077] The digital twin model collects the data collected by sensors to form twin data, and further constructs a digital twin model. The data twin model has a complete digital mapping relationship with the physical space entity device. The virtual model controlled on the computer side can synchronously monitor the working state of the entity device and issue new manufacturing instructions for real-time parameter regulation manufacturing. The digital twin model has a variety of complex mixed data, and the embedded intelligent algorithm can perform large-scale data operations based on the complex mixed data to update the information of the twin data in real time, and at the same time update the digital twin model, so that the digital twin model can be iteratively updated in real time according to the environmental changes in the physical space, so that the model is more practical. The real-time updated digital twin model adjusts the parameters of the device work again according to the processing of the real-time data by the intelligent algorithm, and quickly sends them to the hybrid additive and subtractive manufacturing system through the data transmission module to realize the closed-loop control manufacturing of the whole process of the workpiece. The specific work process is as Figure 14 shown.

[0078] When the digital twin model corrects the manufacturing parameters, it is to make online decisions using algorithms such as machine learning. Taking one of the intelligent algorithms to elaborate the working process, for example, the prediction module of the neural network model corrects the parameters. Based on the data set collected by multiple sensors, an association model between the welding process parameters and the bead forming size is trained online, and the optimal process parameters are generated, and fed back to the five-axis hybrid additive and subtractive manufacturing system through the data transmission module to re-adjust the process parameters for manufacturing. The parameters input into the neural network model include variables such as wire feeding speed, welding torch moving speed, welding current and voltage, frequency of arc flash during welding process, and sound generated by the arc during welding. The output is the geometric size of the bead, including bead width and height. The schematic diagram of the training model of the neural network is as Figure 15, the variables output by the neural network are not limited to the geometric dimensions of a single-layer weld bead, but also include the layer height and layer width of multiple stacked layers, as well as various factor variables that may affect the manufacturing process, etc.

[0079] The content not described in detail in this specification belongs to the prior art well-known to those skilled in the art. Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A forming monitoring system for an additive manufacturing system based on digital twin, comprising a vertical telescopic mechanism (3) of a five-axis machine tool and an alarm module (1), characterized in that: The hardware part of the system includes a top CCD high-speed industrial camera (2), a three-dimensional laser profile scanner (4), an infrared camera (5), a sound sensor (6), a ring-shaped fixing device (7), an arc gun (8), a substrate (9), a strip laser emitter (10), and an arc side monitoring camera (11); The top CCD high-speed industrial camera (2), three-dimensional laser profile scanner (4), infrared camera (5), sound sensor (6), strip laser emitter (10), and arc side monitoring camera (11) are installed around the arc gun (8) and the substrate (9) in cooperation with the ring-shaped fixing device (7), and simultaneously monitor the additive welding forming process. Based on the monitored and collected real physical data, digital twin data is formed, a digital twin model in the virtual space is constructed, the whole process of manufacturing is monitored and regulated in real time, and the closed-loop feedback control of the manufacturing system is realized; The top CCD high-speed industrial camera (2) is used to monitor the arc light change in the whole range, and at the same time, the change of the arc light in the manufacturing process is processed in real time through the corresponding visual image processing algorithm, and a mathematical correlation model is established between the arc light change and the part forming process; The visual image processing algorithm includes graying the collected image, threshold segmentation, and shape screening; filtering the feature distribution at the strong arc light, obtaining the image of the frequency flash arc in the five-axis machine tool processing room. Further, the image is split into RGB channels and further HSV color transformation is performed. Further, the pixel difference operation of each channel with the pixel at the corresponding position of the previous frame of the original image is performed to obtain the corresponding T value, and the size of the T value and the preset threshold are judged. If the T value is greater than the threshold, the current image is subjected to a first-level wavelet transform to extract the low-frequency signal and filter the high-frequency signal, all effective low-frequency signals are stored, and are transmitted to the central system through the transmission module for further decision-making and judgment; The arc side monitoring camera (11), one is to use the arc light to irradiate the molten pool to form a passive vision monitoring state; the other is to use the strip laser emitter (10) to irradiate the molten pool to form an active laser vision monitoring state, and information is collected by combining the active and passive visions, specifically including the usage methods and process steps of the passive and active visions; The usage methods and process steps of the active and passive visions include first using the CCD high-speed industrial camera to collect the arc light image of the current arc, calculating the current threshold using the image processing algorithm, and comparing it with the preset threshold. Further, according to the comparison result, it is decided whether to continue using the passive vision or the laser active vision for image collection in the next step. Finally, the image data is transmitted to the central processing system through the transmission module; The infrared camera (5) is used to monitor the temperature change of the molten pool during the manufacturing process in real time. By analyzing the relationship between the temperature data and the forming quality of the molten pool, an association model with temperature and quality is established, and corresponding control strategies are adopted for regulation. At the same time, the relationship model between the collected temperature information and the forming quality is recorded in the data storage module. The central processing system makes a comprehensive judgment and decision according to the actual working conditions and the previous data, synchronously adjusts the forming welding process parameters and the forming process and steps; Multi-degree-of-freedom camera fixtures (12) are provided at the installation positions of the top CCD high-speed industrial camera (2), the infrared camera (5) and the arc side monitoring camera (11). Through the clamping of the multi-degree-of-freedom camera fixture (12), the shooting angle is adjusted in real time to adapt to the complex situation of additive manufacturing process monitoring; The multi-degree-of-freedom camera fixture (12) includes a large "U" - shaped plate (121) and a small "U" - shaped plate (122), and the large "U" - shaped plate (121) is located outside the small "U" - shaped plate (122). The large "U" - shaped plate (121) is the support plate of the slider (123), and the small "U" - shaped plate (122) is the end for clamping the camera. A chute (124) for the slider (123) to slide is left at the middle position of the large "U" - shaped plate (121), which is used for the slider (123) to slide to adjust the left - right and front - back positions of the camera. The large "U" - shaped plate (121) and the small "U" - shaped plate (122) are connected by a ball - hinge connecting rod (125). The small "U" - shaped plate (122) and the clamped camera have a pitching adjustment function, and the upper end of the large "U" - shaped plate (121) is connected to the machine tool through a connecting fixed block (126). At the same time, the connecting fixed block (126) also has a left - right rotation adjustment function. Based on the adjustment of each degree of freedom described, the clamped camera can be adjusted omnidirectionally in space; The three - dimensional laser profile scanner (4) is vertically installed above the formed part, which is used to monitor the geometric dimensions of the molten pool forming. Through corresponding image - processing algorithms, it monitors the process of defect formation and quickly detects various defects of the formed part. The three - dimensional information of the molten pool extracted is compared with the three - dimensional information of the ideal molten pool effect, analyzes the change of the corresponding process parameters during the defect generation process, and records the corresponding relationship between the defects and the process parameters in the data storage module. In each update of the twin data, the central processing system automatically regulates the corresponding manufacturing process parameters according to the recorded relationship between the defect type and the process parameters; The sound sensor (6) is installed on the side of the arc gun (8), which is used to collect the sound signals during the arc additive manufacturing process in real time. The collected sound signals are processed by corresponding sound - signal algorithms to filter out the interfering sound signals and extract the effective sound signals, and a mathematical association model reflecting the part forming process and the sound signals is established, which is stored in the data storage module as a kind of marker data for the central processing system to make a comprehensive decision - making judgment and regulation basis; The annular fixing device (7) is used for a sensor that needs to be fixed on the side of the arc gun. The sensor can be adjusted in a circumferential position around the arc gun. The annular fixing device can be vertically adjusted relative to the vertical telescopic mechanism of the five-axis machine tool to a large extent, achieving the adjustment effect of the sensor in various spatial positions; The annular fixing device (7) is composed of an upper ring (71) and a lower ring (72), and the upper ring (71) and the lower ring (72) are connected by flat side plates (73). The upper ring (71) makes a circular motion around the end of the flat side plate (73), thereby realizing the relative position change of the upper ring (71) and the lower ring (72) in space, and thus realizing the change in the distance between the sensor and the arc gun (8). Grooves (74) are provided on the inner sides of the upper ring (71) and the lower ring (72), and the grooves (74) are slidably connected to the sensor, thereby realizing the adjustment of rotation around the central axis of the ring. The annular fixing device (7) is connected to the vertical telescopic mechanism (3) of the five-axis machine tool through the side of the mounting plate (75), and the annular fixing device (7) can move vertically through the vertical telescopic mechanism (3) of the five-axis machine tool.

2. A forming monitoring method for an additive manufacturing system based on digital twin, characterized in that: Collect data using the sensor of a forming monitoring system of an additive manufacturing system based on digital twin as described in claim 1, map the data collected by the sensor to the digital space to form twin data, and further construct a digital twin model for predicting and controlling, and online real-time modifying the forming processing parameters, achieving the manufacturing effect from real data to the digital space, and then from the digital virtual space to the real space, so as to realize the full-life cycle management of comprehensively monitoring the additive formed parts; The digital twin model collects the data collected by the sensor, aggregates them to form twin data, and further constructs a digital twin model; The data twin model has a complete digital mapping relationship with the physical space entity device. The virtual model controlled on the computer side can synchronously monitor the working state of the entity device and issue new manufacturing instructions for real-time parameter control manufacturing; The digital twin model has a variety of complex mixed data, and the embedded intelligent algorithm can perform large-scale data operations based on the complex mixed data to real-time update the information of the twin data, and at the same time update the digital twin model, so that the digital twin model can be iteratively updated in real time according to the environmental changes in the physical space, making the model more practical; The real-time updated digital twin model, according to the processing of the real-time data by the intelligent algorithm, re-adjusts the parameters of the equipment operation, and quickly sends them to the composite additive and subtractive manufacturing system through the data transmission module to realize the closed-loop control manufacturing of the whole process of the workpiece.

Images