A method, system, and storage medium for monitoring the quality of a part formed by additive manufacturing

Abstract

The application discloses a kind of additive manufacturing component forming quality monitoring method, system and storage medium, method includes: carrying out additive manufacturing experiment, experimental molten pool image is collected and is preprocessed, constitutes molten pool dataset;Neural network model is built and trained, obtain pre-trained neural network model and it is optimized, and forming quality detection model is generated;According to current process parameter, carry out additive manufacturing, collect current molten pool image, and the quality of additive manufacturing component is predicted by forming quality detection model, and quality prediction information is obtained.The application uses the wetting angle feature and dilution rate feature of cladding path as the feature parameter for predicting forming quality, which can better reflect the quality of cladding path;The obtained prediction model has higher accuracy and generalization ability, can quickly and accurately identify and classify the molten pool in different states in additive manufacturing process, and predict the forming quality of molten pool.The application is applied to the technical field of additive manufacturing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of additive manufacturing quality monitoring, in particular to an additive manufacturing part forming quality monitoring method and system and a storage medium. BACKGROUND

[0002] Laser direct deposition (L-DED) is an emerging additive manufacturing (AM) technology, also known as laser cladding technology. Laser cladding technology refers to a process method in which different filler materials are placed on the surface of the substrate, and the selected coating materials are melted with the shallow layer of the substrate surface by laser irradiation, and then rapidly solidified to form a surface coating with extremely low dilution and metallurgical bonding with the substrate material, thereby significantly improving the wear resistance, corrosion resistance, heat resistance, oxidation resistance and electrical properties of the substrate material surface.

[0003] In the process of laser cladding, the laser beam acts on the powder bed, and the powder particles are fused together here to form a molten pool. In the molten pool, a small amount of metal evaporation will form a cavity in the center of the molten pool. If this cavity becomes unstable, it may have an adverse effect on the quality of the part. Therefore, it is very necessary for the field to monitor and predict the state of the part during additive manufacturing, especially the state of the molten pool.

[0004] The current additive manufacturing quality monitoring method usually uses a CCD camera to collect molten pool images during printing, and analyzes the molten pool images through a traditional deep learning network to find defects in the additive manufacturing process. However, the traditional image processing method has high requirements for the clarity of the molten pool image, the complexity of the background and the defect mapping form, i.e. the hardware requirements for image processing are high, and the monitoring cost of the existing monitoring method is high. In addition, the traditional deep learning model has the problems of poor performance and low accuracy, and the detection accuracy and speed of the model are difficult to achieve the expected effect, and it is difficult to predict the forming quality.

[0005] Moreover, the existing monitoring method usually uses the characteristic parameters of plume and spatter as evaluation parameters of additive manufacturing quality. For example, a Chinese invention patent with the patent number CN110789128B, named "Additive manufacturing part forming quality prediction and control system and method", discloses a part monitoring and prediction method, which predicts the part forming quality after n seconds by combining the LSTM network with three characteristic parameters of plume and spatter. However, plume and spatter are external characteristics of the molten pool, and there is no clear measurement definition in the field, and they cannot reflect the quality of the cladding path, thereby making the accuracy of the prediction result of the forming quality low, and unable to accurately reflect the forming quality. SUMMARY

[0006] The purpose of the present application is to provide an additive manufacturing workpiece forming quality monitoring method, system and storage medium to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0007] The solution to the technical problem of the present application is: in a first aspect, the present application provides an additive manufacturing workpiece forming quality monitoring method, comprising the following steps:

[0008] controlling the additive manufacturing robot to perform an additive manufacturing experiment, collecting experimental molten pool images during the additive manufacturing experiment and preprocessing the experimental molten pool images to form a molten pool data set;

[0009] building a neural network model, training the neural network model through the molten pool data set, obtaining a pre-trained neural network model, and optimizing the pre-trained neural network model to generate a forming quality detection model;

[0010] controlling the additive manufacturing robot to perform additive manufacturing according to current process parameters, collecting current molten pool images, and predicting the quality of the additive manufacturing workpiece through the current molten pool images and the forming quality detection model to obtain quality prediction information;

[0011] The quality prediction information includes the process parameters corresponding to the additive manufacturing, the position information and category of the molten pool, and the quality number of the corresponding forming quality.

[0012] As a further improvement of the above technical solution, the control of the additive manufacturing robot to perform the additive manufacturing experiment, and the collection of the experimental molten pool images during the additive manufacturing experiment, comprises:

[0013] controlling the additive manufacturing robot to perform i times of additive manufacturing experiments, and the process parameters of each additive manufacturing experiment are different, obtaining i additive workpieces;

[0014] The i-th additive workpiece corresponds to the i-th set of additive manufacturing experiments.

[0015] During the additive manufacturing experiment, a plurality of experimental molten pool images are collected by a side-shaft high-speed camera to form an i-group molten pool experimental data set;

[0016] The i-group molten pool experimental image data set corresponds to the i-group additive manufacturing experiment.

[0017] As a further improvement of the above technical solution, the preprocessing of the experimental molten pool images to form a molten pool data set comprises:

[0018] According to the experimental time of the additive manufacturing experiment, the process of the additive manufacturing experiment is equally divided into ten progress intervals;

[0019] The ten progress intervals are [0%, 10%), [10%, 20%), [20%, 30%), [30%, 40%), [40%, 50%), [50%, 60%), [60%, 70%), [70%, 80%), [80%, 90%), and [90%, 100%], respectively.

[0020] In each set of the experimental molten pool data set, the experimental molten pool images with progress of [0%, 10%), [10%, 20%), [70%, 80%), [80%, 90%), and [90%, 100%] are discarded, and a certain number of experimental molten pool images are randomly selected from the [20%, 30%) progress interval, the [30%, 40%) progress interval, the [40%, 50%) progress interval, the [50%, 60%) progress interval, and the [60%, 70%) progress interval, respectively.

[0021] According to the wetting angle threshold, the additive manufacturing parts with a wetting angle less than the wetting angle threshold are screened, and the screened additive manufacturing parts are classified according to the dilution rate to determine the quality number corresponding to each additive manufacturing part.

[0022] The wetting angle threshold is 80°.

[0023] The randomly selected experimental molten pool images are labeled to obtain label data corresponding to the experimental molten pool images, the quality number is added to the label data of the corresponding experimental molten pool images, and the labeled experimental molten pool images are data augmented to form a molten pool data set.

[0024] As a further improvement of the above technical solution, the classification of the screened additive manufacturing parts according to the dilution rate to determine the quality number corresponding to each additive manufacturing part comprises:

[0025] The additive manufacturing parts with a dilution rate less than 10% are classified as number zero, the number zero is used to map that the dilution rate of the additive manufacturing part is too small, and the quality level of the additive manufacturing part is failed;

[0026] The additive manufacturing parts with a dilution rate greater than or less than 10% and less than or equal to 15% are classified as number one, the number one is used to map that the dilution rate of the additive manufacturing part is general, and the quality level of the additive manufacturing part is passed;

[0027] the additive manufacturing part with the dilution rate greater than 15% and less than or equal to 25% is classified as No. 2, the No. 2 is used for mapping that the dilution rate of the additive manufacturing part is excellent, and the quality level of the additive manufacturing part is excellent;

[0028] the additive manufacturing part with the dilution rate greater than 25% is classified as No. 3, the No. 3 is used for mapping that the dilution rate of the additive manufacturing part is too large, and the quality level of the additive manufacturing part is failed.

[0029] As a further improvement of the above technical solution, the randomly selected experimental molten pool image is labeled to obtain the label data corresponding to the experimental molten pool image, the quality number is added to the label data of the corresponding experimental molten pool image, and the experimental molten pool image after label processing is data augmented, including:

[0030] The target in the randomly selected experimental molten pool image is labeled by an image labeling tool, the target is a molten pool, the category and location information of the target are located, the label data of the experimental molten pool image is obtained, and the quality number is added to the label data of the corresponding experimental molten pool image;

[0031] The experimental molten pool image is linearly transformed and blurred and filtered;

[0032] The experimental molten pool image after processing is data augmented to obtain a data augmented image, and the resolution of the data augmented image and the experimental molten pool image is adjusted so that the resolution of the data augmented image and the experimental molten pool image is the same.

[0033] As a further improvement of the above technical solution, the neural network model is built, the neural network model is trained by the molten pool data set, and a pre-trained neural network model is obtained, including:

[0034] The molten pool training set is divided into a training set and a validation set according to a 1:1 ratio;

[0035] A neural network model is built based on YOLOv5, the training set and the validation set are used as inputs of the neural network model, the neural network model is pre-trained to obtain a pre-trained neural network model and a weight file YOLOv5.pt of the pre-trained neural network model.

[0036] As a further improvement of the above technical solution, the pre-trained neural network model is optimized to generate a forming quality detection model, including:

[0037] controlling the additive manufacturing robot to perform a plurality of additive manufacturing tests, during the additive manufacturing tests, a plurality of coaxial molten pool images are collected by a coaxial CCD camera, and a plurality of off-axis high-speed camera images are collected by an off-axis high-speed camera;

[0038] The coaxial molten pool images and the off-axis molten pool images are preprocessed to form a test set and an optimized training set.

[0039] The optimized training set is used as an input of the pre-trained neural network model, and the pre-trained neural network model is trained based on the weight file YOLOv5.pt.

[0040] The performance of the neural network model is evaluated using the test set, and a forming quality prediction model is output.

[0041] As a further improvement of the above technical solution, the preprocessing of the coaxial molten pool images and the off-axis molten pool images to form a test set and an optimized training set comprises:

[0042] According to the dilution rate, a plurality of test additive manufacturing parts obtained by the additive manufacturing test are classified according to forming quality, and the quality number corresponding to each test additive manufacturing part is determined.

[0043] The off-axis molten pool images are labeled to obtain label data corresponding to the off-axis molten pool images, and the quality number corresponding to the test additive manufacturing part is added to the corresponding label data.

[0044] The off-axis molten pool images after label processing are data augmented, and the augmented off-axis molten pool images are classified into a test set and a to-be-optimized training set.

[0045] The molten pool parameter information in the coaxial molten pool images is extracted, and the molten pool parameter information includes the process parameters corresponding to the additive manufacturing test, the width, height and depth of the molten pool.

[0046] The molten pool parameter information is added to the label data of the corresponding off-axis molten pool image in the to-be-optimized training set to form the optimized training set.

[0047] In a second aspect, the present application provides an additive manufacturing part forming quality monitoring system, comprising:

[0048] An image acquisition unit comprising an off-axis high-speed camera and a coaxial CCD camera, for acquiring experimental molten pool images, coaxial molten pool images and off-axis molten pool images.

[0049] A data processing unit for preprocessing the experimental molten pool images, the coaxial molten pool images and the off-axis molten pool images to form a molten pool data set, an optimized training set and a test set.

[0050] a pre-training unit configured to build a neural network model, train the neural network model through the molten pool dataset, and obtain a pre-trained neural network model;

[0051] an optimization unit configured to perform optimization processing on the pre-trained neural network model through the optimization training set, perform performance testing on the neural network model after the optimization processing through the test set, and generate a forming quality detection model;

[0052] a high-speed camera real-time monitoring system, which is loaded with the forming quality detection model, is configured to collect a current molten pool image, perform quality detection on the current molten pool image through the forming quality detection model, and output quality prediction information through model prediction.

[0053] In a third aspect, the present application also provides a storage medium having processor-executable instructions stored therein, wherein the processor-executable instructions, when executed by a processor, are configured to perform the additive manufacturing part forming quality monitoring method.

[0054] The present application has the following beneficial effects: The additive manufacturing part forming quality monitoring method, system and storage medium provided by the present application use the molten pool or cladding path formed in the laser cladding process, extract the wetting angle feature and dilution rate feature of the molten pool or cladding path as the feature parameters of the molten pool, and can more comprehensively reflect the quality of the cladding path. The neural network model obtained through optimization training has higher accuracy and generalization ability, can quickly and accurately identify and classify the molten pool in different states in the additive manufacturing process, and can predict the forming quality of the molten pool, thereby realizing the early prediction of the quality of the part and the correction of the process parameters, reducing the production cost, avoiding the long time period of post-processing work, and providing data support for the forming quality prediction of different additive manufacturing materials. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 A flowchart of the additive manufacturing part forming quality monitoring method provided by the embodiment of the present application;

[0056] Figure 2 Structural schematic diagrams of the additive manufacturing robot, the coaxial high-speed camera and the paraxial high-speed camera provided by the embodiment of the present application;

[0057] Figure 3 A molten pool topography diagram in the additive manufacturing process provided by the embodiment of the present application;

[0058] Figure 4 A flowchart of the pre-processing experimental molten pool image provided by the embodiment of the present application;

[0059] Figure 5A cladding channel cross-sectional view provided for the embodiments of the present application;

[0060] Figure 6 Different single-channel processed cross-sectional schematic diagrams provided for the embodiments of the present application;

[0061] Figure 7 Different single-channel surface quality schematic diagrams provided for the embodiments of the present application;

[0062] Figure 8 Performance curve diagrams of the forming quality prediction model provided for the embodiments of the present application;

[0063] Figure 9 Result diagrams of the high-speed camera real-time monitoring system provided for the embodiments of the present application. DETAILED DESCRIPTION

[0064] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0065] The present application is further described below in combination with the drawings and specific embodiments. The described embodiments should not be considered as limiting the present application, and all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0066] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0068] Additive Manufacturing (AM), also known as 3D printing. 3D printing refers to the fusion of computer-aided design, material processing and forming technology, based on digital model files, through software and numerical control system to place special metal materials, non-metal materials and medical biological materials according to extrusion, sintering, melting, light curing, spraying and other ways, to manufacture solid objects. In short, 3D printing is a process of manufacturing entities by layer-by-layer forming and stacking of materials. Unlike traditional machining methods of removing-chipping and assembling of raw materials, 3D printing is a bottom-up manufacturing method through material accumulation. This makes it possible to manufacture complex structures that were previously constrained by traditional manufacturing methods. Currently, 3D printing is commonly used in industrial, biological manufacturing and other technical fields.

[0069] The basic process of 3D printing is:

[0070] Modeling: Modeling is the first step of the entire additive manufacturing. There are usually two ways to establish a data model: one is to model through three-dimensional modeling software; the other is to obtain by scanning the real object with a scanner.

[0071] Pre-printing processing: Convert the model file to stl format and slice the model through slicing software, while setting the printing parameters or process parameters, such as setting custom support, model placement direction, layer thickness, etc.

[0072] Printing: Import the file processed by the slicing software into the 3D printer and start printing. Before printing, check the hopper to avoid foreign objects in the printing area.

[0073] Post-printing processing: After printing is completed, the model is taken out from the printer, and the support is still hanging on the model at this time, which needs to be pulled off or cut off with professional tools, and the support surface of the model needs to be polished. This step often needs skilled personnel to operate in a suitable environment with the help of tools. Then, according to different printing processes, the model is post-cured with ultraviolet light, the metal parts are annealed to relieve stress, and finally the model is polished, polished, painted and other operations according to the final requirements.

[0074] Laser energy deposition (L-DED) is a new additive manufacturing technology, also known as laser cladding technology. Laser cladding technology refers to placing selected coating materials on the surface of the substrate in different ways, melting the coating materials and the shallow layer of the substrate surface at the same time by laser irradiation, and forming a surface coating with extremely low dilution and metallurgical bonding with the substrate material after rapid solidification, so as to significantly improve the wear resistance, corrosion resistance, heat resistance, oxidation resistance and electrical properties of the substrate material surface.

[0075] During the process of laser cladding, the laser beam acts on the powder bed, where the powder particles are fused together to form a molten pool. In the molten pool, a small amount of metal evaporation can form a cavity in the center of the molten pool. If this cavity becomes unstable, it may collapse on its own, resulting in the formation of pores in the printed part. In addition, if collapse occurs, steam will be ejected upward from the keyhole and form a plume. This can affect some un-fused particles in the powder bed, possibly disturbing the top layer of material, which can lead to a large number of small defects in the entire part, resulting in poor quality of the part. Therefore, it is necessary for the field to monitor and predict the state of the part during the process of additive manufacturing, especially the state of the molten pool.

[0076] In the field, the commonly used quality monitoring method of additive manufacturing is usually to use a CCD (Charge Coupled Device) camera to collect molten pool images during the printing process, and to analyze the molten pool images through a traditional deep learning network to find defects in the additive manufacturing process. For example, metal laser melting deposition molten pool state recognition based on PPCNN. However, the existing quality monitoring method has limitations, which has the following defects:

[0077] (1) The traditional image processing method is used to process and convert the molten pool image. Since the traditional image processing method has high requirements for the clarity, background complexity and defect mapping form of the molten pool image, i.e. the hardware requirements for image processing are high, thereby increasing the monitoring cost of additive manufacturing.

[0078] (2) The traditional deep learning model has the problems of poor performance and low accuracy. The detection accuracy and speed of the model are difficult to achieve the expected effect, and it is difficult to predict the forming quality;

[0079] (3) In the prior art, the characteristic parameters of the plume and the spatter are usually used as evaluation parameters of the quality of additive manufacturing. However, in actual application, the plume and the spatter belong to the external characteristics of the molten pool, and cannot accurately reflect the forming quality, nor have a clear measurement definition. Moreover, in the application of directed energy deposition, since the final goal is to print a workpiece of a specific quality and a specific shape, the manufacturing process of the workpiece often includes the step of overlapping the cladding path, and the characteristic parameters of the plume and the spatter cannot reflect the quality of the cladding path.

[0080] In view of the above technical problems, the present application provides a forming quality online monitoring method and system based on YOLOv5 and molten pool tracking, which is mainly applied in the field of additive manufacturing technology, and particularly applied in the field of L-DED technology. Referring to Figures 1 to 3 , as shown, Figure 1The diagram shown is a flowchart of a method for monitoring the forming quality of additive manufacturing parts according to an embodiment of this application; Figure 2 The diagram shown is a structural schematic of the additive manufacturing robot, coaxial high-speed camera, and rangefinder high-speed camera provided in the embodiments of this application. Figure 3 The image shown is a molten pool morphology diagram during the additive manufacturing process provided in an embodiment of this application. An embodiment of this application will be described and explained below regarding the additive manufacturing part forming quality monitoring method. The forming quality monitoring method may include, but is not limited to, the following steps.

[0081] S100 controls the additive manufacturing robot to conduct additive manufacturing experiments. During the additive manufacturing experiments, it collects experimental molten pool images and preprocesses the experimental molten pool images to form a molten pool dataset.

[0082] In this specific embodiment, the additive manufacturing robot is a six-axis KUKA robot, such as... Figure 2 As shown. The additive manufacturing robot includes: a laser, a cladding head 101, a chiller, a powder feeder, a gas cylinder, and a motion control system. The laser provides a high-energy laser heat source. The cladding head 101 is used to output laser light and powder, with the substrate corresponding to the experimental material located 13 mm from the output port of the cladding head 101. The chiller ensures the stable operation of the laser and the laser cladding head 101. The powder feeder provides a continuous supply of raw materials for laser cladding. The motion control system is used to control the directional movement of the cladding head 101 in space. In addition, a rangefinder high-speed camera 103 and a coaxial CCD camera 102 are arranged near the cladding head 101, with the coaxial CCD camera 102 mounted above the cladding head 101.

[0083] This step involves acquiring the data used to train the neural network. The data for training the neural network is obtained through several additive manufacturing experiments. During the additive manufacturing process, a powder bed forms... Figure 3 The molten pool shown.

[0084] S200: Build a neural network model, train the neural network model using the molten pool dataset to obtain a pre-trained neural network model, and optimize the pre-trained neural network model to generate a forming quality detection model.

[0085] This step involves training and optimizing the neural network model. During neural network training, the number and types of samples in the dataset are increased to optimize the network's parameters.

[0086] S300 controls the additive manufacturing robot to perform additive manufacturing according to the current process parameters, collects the current molten pool image, and performs quality prediction on the additively manufactured part by using the current molten pool image and the forming quality inspection model to obtain quality prediction information.

[0087] It should be noted that the quality prediction information includes the process parameters corresponding to the additive manufacturing, the position information and the category of the molten pool, and the quality number corresponding to the forming quality.

[0088] This step is a step of real-time forming quality monitoring by the trained neural network model. A high-speed camera real-time monitoring system is built by the trained forming quality prediction model, which can output molten pool tracking information and forming quality prediction information in real time.

[0089] In one embodiment of the present application, the collection of experimental molten pool images in S100 will be further described and explained. The process of collecting experimental molten pool images is as follows:

[0090] First, control the additive manufacturing robot to perform i times of additive manufacturing experiments, and the process parameters of each additive manufacturing experiment are different, thereby obtaining i additive manufacturing parts. The i-th additive manufacturing part corresponds to the i-th set of additive manufacturing experiments.

[0091] It should be noted that the process parameters refer to the process parameters used to control and guide the additive manufacturing robot to print. The process parameters can include but are not limited to:

[0092] The laser power refers to the working power of the laser of the additive manufacturing robot, and the unit is usually watts.

[0093] The scanning speed refers to the rate at which the additive manufacturing robot scans the slices in the slice file, and the unit is usually mm / s.

[0094] The powder feeding speed, also known as the powder mass flow, refers to the mass of the powder fed into the cladding head 101 of the additive manufacturing robot per unit time when the sprayed material for 3D printing is powder.

[0095] When performing additive manufacturing experiments, a plurality of experimental molten pool images are collected by the off-axis high-speed camera 103 to form an i-th set of molten pool experimental data. The i-th set of molten pool experimental image data corresponds to the i-th set of additive manufacturing experiments.

[0096] It should be noted that the off-axis high-speed camera 103 is installed near the cladding head 101 of the additive manufacturing robot. Optionally, the color mode of the off-axis high-speed camera 103 is black and white, the captured image format is tif, the image size is 832*600, the exposure time is 25 microseconds, and the shooting frequency is 4000 frames per second.

[0097] In this specific embodiment, the additive manufacturing experiment refers to laser printing a straight line cladding path. The value of i is selected as 48, that is, the additive manufacturing robot is controlled to laser print 48 straight line cladding paths. When laser printing, the off-axis high-speed camera 103 captures molten pool pictures of 48 single-channel experiments, and more than 500,000 off-axis molten pool pictures are collected.

[0098] Referring to Figure 4 shown, Figure 4 shown is a flowchart of a pre-processing experimental molten pool image provided by an embodiment of the present application. An embodiment of the present application will be further described and explained below for the process of pre-processing the experimental molten pool image in S100. The pre-processing experimental molten pool image can include but is not limited to the following steps.

[0099] S110, progress classification: according to the experimental time of the additive manufacturing experiment, the process of the additive manufacturing experiment is equally divided into ten progress intervals.

[0100] In this step, the experimental time refers to the total duration required for one additive manufacturing experiment. The process of additive manufacturing can be understood as the progress of additive manufacturing. At each 3D printing, the operating system of 3D printing usually displays the printing progress, and when the printing progress is displayed as 100%, it means that this 3D printing is completed. Therefore, the progress of additive manufacturing is equally divided into ten intervals, and the difference between the front end point and the rear end point of each interval is 10%, in order to facilitate the subsequent processing of the experimental molten pool image. The ten progress intervals are:

[0101] [0%, 10%); in this specific embodiment, corresponding to 0%-10% progress; which does not include progress 10%;

[0102] [10%, 20%); in this specific embodiment, corresponding to 10%-20% progress; which includes progress 10% and does not include progress 20%;

[0103] [20%, 30%); in this specific embodiment, corresponding to 20%-30% progress; which includes progress 20% and does not include progress 30%;

[0104] [30%, 40%); in this specific embodiment, corresponding to 30%-40% progress; which includes progress 30% and does not include progress 40%;

[0105] [40%, 50%); in this specific embodiment, corresponding to 40%-50% progress; which includes progress 40% and does not include progress 50%;

[0106] [50%, 60%); in this specific embodiment, corresponding to 50%-60% progress; which includes progress 50% but does not include progress 60%;

[0107] [60%, 70%); in this specific embodiment, corresponding to 60%-70% progress; which includes progress 60% but does not include progress 70%;

[0108] [70%, 80%); in this specific embodiment, corresponding to 70%-80% progress; which includes progress 70% but does not include progress 80%;

[0109] [80%, 90%); In this specific embodiment, it corresponds to 80%-90% progress; This includes progress of 80%, but does not include progress of 90%;

[0110] [90%, 100%]; In this specific embodiment, it corresponds to a progress of 90%-100%; including progress of 90% and 100%.

[0111] For example, if an additive manufacturing experiment takes ten minutes, these ten minutes can be divided into nine progress intervals. The first interval is from zero to one minute, excluding the first minute. The second interval is from one to two minutes, including one minute but excluding two minutes. The third interval is from two to three minutes, including two minutes but excluding three minutes. The fourth interval is from three to four minutes, including three minutes but excluding four minutes. The fifth interval is from four to five minutes, including four minutes but excluding five minutes. The sixth interval is from five to six minutes, including five minutes but excluding six minutes. The seventh interval is from six to seven minutes, including six minutes but excluding seven minutes. The eighth interval is from seven to eight minutes, including seven minutes but excluding eight minutes. The ninth interval is from eight to nine minutes, including eight minutes but excluding nine minutes. The tenth interval is from nine to ten minutes, including both nine and ten minutes.

[0112] It should be noted that in other embodiments of this application, the progress of additive manufacturing can be divided into other numbers of progress intervals, either equally or unequally, and the progress included in the progress interval can also be determined according to the actual situation.

[0113] S120, Discarding and Selecting Data: In each set of molten pool experimental datasets, discard experimental molten pool images with progress rates of [0%, 10%), [10%, 20%), [70%, 80%), [80%, 90%), and [90%, 100%]. Randomly select a certain number of experimental molten pool images from the progress ranges of [20%, 30%), [30%, 40%), [40%, 50%), [50%, 60%), and [60%, 70%).

[0114] It is understandable that "certain quantity" refers to a preset number of samples. For example, if the preset number of samples is 40 images, then 40 melt pool images will be randomly selected from the images belonging to the 20%-30%, 30%-40%, 50%-60%, and 60%-70% progress stages.

[0115] In this step, in the actual additive manufacturing process, the printing progress is 0-20%, the material used for printing has not yet melted, or the material melting state has not yet stabilized. Therefore, the molten pool image of the progress of 0-20% has certain error, and cannot accurately reflect the additive manufacturing process. When the printing progress is 70-100%, the additive manufacturing operation is about to end, and due to the camera field of view problem, the lens of the paraxial camera is blocked by the workpiece, and the molten pool image cannot be collected. Therefore, the molten pool image of the printing progress of 0-20% and the printing progress of 70-100% is discarded in each experiment.

[0116] In practical application, the discarded molten pool image can not be limited to the image of the progress of 0-20% and 70-100%. The robot used for additive manufacturing can be determined according to the actual situation, and the parameters of the robot are different from the embodiment of the application. Therefore, the discarded molten pool image can be determined according to the actual situation. However, it should be noted that the discarded molten pool image must meet the following two conditions: if the material in the image has not yet melted or the material melting state has not yet stabilized, then discard this image; if the image cannot reflect the additive manufacturing operation, then discard this image.

[0117] In addition, the random selection of image data is to enhance the instability of the data and prevent overfitting in the model training process.

[0118] In the application, in order to ensure that each single experiment has corresponding image data mapping the process of additive manufacturing, and to prevent the randomly selected data from being concentrated in a certain progress interval, the application selects equally divided time points during the shooting of each single experiment, that is, the process of additive manufacturing is equally divided into several progress intervals, and an equal amount of molten pool pictures are selected near each progress time point. At the same time, considering that the pictures of the first 20% of the progress are mostly pictures that have not yet started to melt or have not yet stabilized, and the pictures of the last 20% of the progress cannot be shot due to the camera field of view problem, therefore, the images of these two parts are discarded. Finally, the extraction rule of the data set is as follows: in each group of original molten pool experimental data set, starting from the progress of 20%, a certain amount of images are randomly taken in the five intervals of 20%-30%, 30%-40%, 40%-50%, 50%-60%, and 60%-70%.

[0119] In this embodiment, for each group of molten pool experimental data set, 40 experimental molten pool images are randomly taken in the five intervals of 20%-30%, 30%-40%, 40%-50%, 50%-60%, and 60%-70%. A total of 9600 experimental molten pool images are taken for 48 single experiments.

[0120] S130, quality grading: screening the additive manufactured part with a wetting angle less than the wetting angle threshold value according to the wetting angle threshold value, and classifying the screened additive manufactured part according to the dilution rate to determine the quality number corresponding to each additive manufactured part.

[0121] It should be noted that the wetting angle threshold value is 80°. In actual application, the weting angle threshold value can be determined according to actual conditions, and the wetting angle threshold value is not limited in the present application.

[0122] In this step, the quality of the cladding path is reflected by the dilution rate, wetting angle and width-height ratio of the molten pool. The dilution rate, wetting angle and width-height ratio can be measured by the cladding path cross-section diagram, which is obtained by wire cutting, sample grinding, polishing, etching treatment and optical microscope shooting. The final goal of directed energy deposition is to print a workpiece with a specific quality or a specific shape, which often needs to be cladded, and the dilution rate, width-height ratio and wetting angle are all indicators that can measure whether the cladding path can be cladded with high quality.

[0123] The dilution rate refers to the percentage of the metal area of the base material melted into the weld cross-sectional area to the weld cross-sectional area. Generally, the composition of the filler metal is often different from that of the base material, especially when welding or alloy cladding is performed. When the alloy composition of the cladding metal mainly comes from the filler metal, the effect of the partially melted base material in the weld can be considered as dilution. Therefore, the fusion ratio is also commonly referred to as the dilution rate.

[0124] In the specific embodiment, the dilution rate can be calculated by the molten pool depth b and the molten pool height h. Referring to FIG. 1, Figure 5 , Figure 5 the cladding path cross-section diagram provided by the embodiment of the present application, the figure reflects the wetting angle θ, the molten pool depth b, the molten pool height h and the molten pool width w of the cladding path. Therefore, the dilution rate satisfies the following formula:

[0125]

[0126] Wherein, D% represents the dilution rate.

[0127] The wetting angle refers to the angle between the liquid-solid interface at the contact point of the liquid phase and the solid phase and the tangent of the liquid surface. When the angle is equal to 0°, it means complete wetting, i.e. the liquid can spread freely on the surface of the solid. When the angle is less than 90°, it means wetting, and the wetting effect becomes worse as the angle increases. When the angle is greater than 90°, it is basically not wetted due to the small wetting tension. When the angle is equal to 180°, it means complete non-wetting. Referring to FIG. 2, Figure 5 , Figure 5 the wetting angle in the figure is represented by the θ angle.

[0128] According to the wetting angle threshold, the wetting angle less than 80° is screened. Then, according to the preset quality classification rule and the dilution rate, the remaining additive parts are classified in quality, and the mapping quality number is confirmed. In this way, the quality of the additive parts of each experiment is added to the corresponding label data to improve the data set.

[0129] In the embodiment, the quality classification rule is as follows:

[0130] The quality level is divided into four levels, corresponding to four quality numbers: number 0, number 1, number 2 and number 3. Specifically, number 0 represents that the dilution rate of the additive part is too small, and the quality level is failed. The additive part with a dilution rate D% < 10% is classified as number 0. Number 1 represents that the dilution rate of the additive part is general, and the quality level is passed. The additive part with a dilution rate of 10%≤D%≤15% is classified as number 1. Number 2 represents that the dilution rate of the additive part is excellent, and the quality level is excellent. The additive part with a dilution rate of 15% < D% ≤ 25% is classified as number 2. Number 3 represents that the dilution rate of the additive part is too large, and the quality level is the same as that of number 0, both of which are failed. The additive part with a dilution rate D% > 25% is classified as number 3.

[0131] In the present application, since the process parameters of each experiment are different, the quality number actually reflects that different process parameters correspond to different forming quality, that is, the quality number maps the relationship between process parameters and forming quality. Referring to Figure 6 and Figure 7 , it is shown that Figure 6 is the cross-sectional view of the different single-channel processing provided by the embodiment of the present application; Figure 7 is the schematic diagram of the different single-channel surface quality provided by the embodiment of the present application. As shown in Figure 6 and Figure 7 , according to the quality classification rule, the experimental molten pool image can be classified into different categories corresponding to different quality numbers.

[0132] S140, data processing: label processing is performed on the randomly selected experimental molten pool image to obtain the label data corresponding to the experimental molten pool image, the quality number is added to the label data of the corresponding experimental molten pool image, and the experimental molten pool image after label processing is data augmented to form a molten pool data set.

[0133] Specifically, the steps of data processing are as follows:

[0134] S141, the target in the randomly selected experimental molten pool image is labeled by an image labeling tool, the category and position information of the target are located, the label data of the experimental molten pool image is obtained, and the quality number is added to the label data of the corresponding experimental molten pool image.

[0135] Note that the target is the molten pool. The image labeling tool is Labeling.

[0136] In this step, the purpose of the target detection algorithm is to find the target of interest, i.e., the molten pool, in the image captured by the high-speed camera, and to determine the category and position of the molten pool. Therefore, when preparing the data set used for target detection, first, the randomly selected experimental molten pool images need to be information-labeled by the LabelImg tool, and the corresponding label data is generated. The label data includes the category of the molten pool, the position information of the molten pool in the image, and the area information of the molten pool, etc. At the same time, the quality number obtained in S130 is added to the corresponding label data.

[0137] For example, the first set of molten pool data corresponds to the first additive manufacturing experiment, and the first additive manufacturing part is produced in this additive manufacturing experiment. After quality evaluation, the quality of the first additive manufacturing part is “excellent”, and the quality number is three. Then the quality number three is added to the label data of all experimental molten pool images in the first additive manufacturing experiment. The second set of molten pool data corresponds to the second additive manufacturing experiment, and the second additive manufacturing part is produced in this additive manufacturing experiment. After quality evaluation, the quality of the second additive manufacturing part is “general”, and the quality number is two. Then the quality number two is added to the label data of all experimental molten pool images in the second additive manufacturing experiment. In this way, until all the label data of the molten pool images are added with the corresponding quality number.

[0138] In this embodiment, the LabelImg tool is used to label 9600 experimental molten pool images.

[0139] S142, linear transformation and blur filtering processing is performed on the experimental molten pool image.

[0140] In this step, the experimental molten pool image is a grayscale image, and the linear transformation is one of the ways of grayscale transformation. The linear transformation refers to that the original image grayscale value is x, and the pixel grayscale value is transformed by the linear transformation function f(x) = kx + b.

[0141] Specifically, the function is defined as s = T(r), where T is the grayscale transformation function, r is the grayscale before transformation, and s is the pixel after transformation. The linear transformation equation is f(x) = kx + b, where f(x) is the grayscale value after transformation, and x is the grayscale value before transformation. When the slope k is greater than one, the linear transformation will increase the contrast between the gray levels; when k is greater than zero and less than one, the linear transformation will reduce the contrast between the gray levels. By changing the values of the two variables k and b, the transformation result of the experimental molten pool image is adjusted. The purpose of the linear transformation of the experimental molten pool image in this application is to enhance the contrast of the image.

[0142] Optionally, the experimental molten pool image is segmented, and different linear transformation functions are used to transform each segment of the image. In this way, the contrast of the region of interest in the experimental molten pool image can be increased, and the contrast of the non-interest region can be compressed.

[0143] In addition, image blur filtering is calculated by a convolution operator on the image, so it is also called linear filtering. The convolution calculation satisfies the following formula:

[0144]

[0145] Where h(k, l) is the convolution operator, and f(i, j) is the pixel of the image.

[0146] In this step, the experimental molten pool image is blurred by median filtering to make the experimental molten pool image smoother. Median filtering refers to reordering the pixels in the convolution kernel and replacing the pixel value of the center point with the middle value. Median filtering has good inhibitory effect on salt and pepper noise of the image.

[0147] S143, the processed experimental molten pool image is data augmented to obtain a data augmented image, and the resolution of the data augmented image and the experimental molten pool image is adjusted so that the resolution of the data augmented image and the experimental molten pool image is the same.

[0148] In this step, one of the reasons for the poor performance of the target detection model is the poor reproducibility of the target in the training. In order to improve the performance of the target detection model and ensure the reproducibility of the molten pool target in the training process, the present application uses multiple data augmentation methods to augment the experimental molten pool image. After data augmentation, the data augmented image obtained by data augmentation and the experimental molten pool image are adjusted to have a resolution of 640*640. Finally, the molten pool dataset is formed.

[0149] Optionally, the data augmentation method can include but is not limited to image splicing, target molten pool random pasting, random affine transformation, etc. Among them:

[0150] Image splicing refers to splicing two or more molten pool images together to form a new image.

[0151] Target molten pool random pasting refers to randomly copying and pasting the molten pool target in one molten pool image to any position in another molten pool image, so that the other molten pool image becomes a new image. The number of copying and pasting can be multiple times.

[0152] Random affine transformation refers to scaling, folding and rotating any molten pool image to form a new image.

[0153] One embodiment of the present application will be described below. The process of training the neural network in step 200 will be described and explained. The process of training the neural network of the present application can be divided into two steps: the first step is to pre-train the neural network; the second step is to fine-tune the pre-trained neural network (hereinafter referred to as a pre-trained model) and perform secondary training to obtain a quality prediction model.

[0154] Specifically, the process of pre-training the neural network includes:

[0155] First, the training set of the molten pool is divided into a training set and a validation set according to a 1:1 ratio.

[0156] In this step, 50% of the experimental molten pool images in the molten pool training set are randomly selected as the training set, and the remaining 50% of the experimental molten pool images are used as the validation set.

[0157] Then, based on YOLOv5, a neural network model is built, the training set and the validation set are used as the input of the neural network model, the neural network model is pre-trained, and a pre-trained neural network model is obtained.

[0158] It should be noted that the YOLOv5 network model mainly includes four modules: an input end, a backbone network, a Neck network, and a prediction end. It uses GIOU_Loss as the loss function and uses non-maximum suppression (NMS) to screen the target frame. The calculation speed of the SPPF module of YOLOv5 is faster than that of the previous SPP module. In addition to SPPF, the Neck network also uses a top-down FPN feature pyramid and a bottom-up PAN feature pyramid to improve the feature extraction capability of the network.

[0159] The loss function GIOU satisfies:

[0160]

[0161]

[0162] Where M represents the intersection between the target real frame and the predicted frame, N represents the union of the target real frame and the predicted frame, and IOU represents the ratio of the intersection to the union. The minimum bounding rectangle of the target real frame and the predicted frame is represented by C, and D is the difference set between C and the union N. Therefore, GIOU_Loss can be represented by the following formula:

[0163]

[0164] In this step, the training parameters of the network model are set, such as modifying the number of iterations, initializing the learning rate, and setting the number of image channels. After setting the training parameters, the training set obtained by the drawing is input into the YOLOv5 network for training to obtain the weight file YOLOv5.pt of the network. In this embodiment, the network model parameters are set as follows: the initial learning rate is 0.01;

[0165] Learning rate momentum (Momentum) Batch-Size IOU loss coefficient (Box) 0.937 32 0.5 Weight decay coefficient (weight_decay) Classification loss coefficient (CIs) Number of iterations (Epochs) 0.0005 0.5 250

[0166] After obtaining the pre-trained model, the network model is optimized. Specifically, the process of network optimization includes:

[0167] Firstly, the additive manufacturing robot is controlled to perform multiple additive manufacturing tests, and during the additive manufacturing tests, a plurality of coaxial molten pool images are collected by the coaxial CCD camera 102, and a plurality of off-axis high-speed camera 103 are collected by the off-axis molten pool image.

[0168] In this step, different process parameters are set, and the additive manufacturing robot is controlled to perform several additive manufacturing tests. During the test, the coaxial CCD camera 102 is used to take multiple coaxial molten pool images, and at the same time, the off-axis high-speed camera 103 is used to take multiple off-axis molten pool images. The off-axis molten pool image is used to test the performance of the trained neural network. The coaxial molten pool image is used to optimize the pre-trained model.

[0169] Furthermore, the coaxial molten pool image and the off-axis molten pool image are preprocessed to form a test set and an optimization training set.

[0170] Specifically, the off-axis molten pool image is processed using the steps of S130 and S140. First, the quality of the additive manufacturing test is evaluated and classified according to the quality classification rule to generate a quality number; then the off-axis molten pool image is labeled and data augmented, and the quality number corresponding to the off-axis molten pool image is added to the label data. The processed off-axis molten pool image is divided into a test set and a to-be-optimized training set.

[0171] The coaxial molten pool image is processed, and the molten pool parameter information is extracted, which includes the process parameters corresponding to the additive manufacturing test, the width, height and depth of the molten pool, etc. The molten pool parameter information of the coaxial molten pool image is added to the label data of the corresponding off-axis molten pool image in the to-be-optimized training set, and then the optimization training set is formed.

[0172] Then, the optimization training set is used as the input of the pre-trained neural network model to train the pre-trained neural network model.

[0173] In this step, the tuning training set is input into the pre-trained model, and the optimization is continued on the basis of the weight file YOLOv5.pt. Before the end of the training optimization, a small amount of CCD camera abnormal information needs to be manually corrected. The network tuning of the present application focuses on the optimization of the learning rate. In the warm-up stage, one-dimensional linear interpolation is used to update the learning rate of each iteration. After the warm-up stage, the cosine annealing algorithm is used to update the learning rate. Finally, the learning rate is reduced to 0.01*0.01.

[0174] Finally, the performance of the neural network model is evaluated by using the test set, and a forming quality prediction model is output.

[0175] Referring to Figure 8 , the performance curve of the forming quality prediction model provided by the embodiment of the present application is shown. Figure 8 Figure 8 The model training experimental data of the embodiment of the present application is as follows:

[0176] “train / box_loss” refers to the training set bounding box loss. In the present embodiment, the training set bounding box loss is 0.0116.

[0177] “train / obj_loss” refers to the training set target detection loss mean. In the present embodiment, the training set target detection loss mean is 0.0032.

[0178] “train / cls_loss” refers to the training set classification loss mean. In the present embodiment, the training set loss mean is 0.0001.

[0179] “metrics / precision” refers to the precision. In the present embodiment, the precision is 0.9995.

[0180] “metrics / Recall” refers to the recall. In the present embodiment, the recall is 0.9993.

[0181] “val / box_loss” refers to the validation set bounding box loss. In the present embodiment, the validation set bounding box loss is 0.0168.

[0182] “val / obj_loss” refers to the validation set target detection loss mean. In the present embodiment, the validation set target detection loss mean is 0.0036.

[0183] ​"val / cls_loss" refers to the validation set classification loss mean. In this embodiment, the validation set classification loss mean is 0.0001.

[0184] "metrics / mAP_0.5" refers to the mean average precision mean with a threshold value greater than 0.5. In this embodiment, the mean average precision mean is 0.9950.

[0185] "metrics / mAP_0.5:0.95" refers to the mean average precision mean with a threshold value in the threshold value interval [0.5, 0.95] and a step size of 0.05. In this embodiment, the mean average precision mean is 0.7544.

[0186] By Figure 8 It can be seen that the accuracy of the model is continuously improved and the loss is gradually reduced with the increase of the number of iterations. Through optimization, the bounding box loss of the neural network model of the training set is reduced to 0.0116, the target detection loss mean is reduced and stabilized to 0.0032, the classification loss mean is reduced and converged to 0.0001, and the accuracy of the network predicting the corresponding forming quality of the molten pool image reaches 99.9%. The forming quality monitoring method provided by the present application takes less than 0.005 seconds from acquiring a single molten pool image to completing quality prediction. Figure 8 In addition to the experimental data described above, not only does the performance of the forming quality prediction model of the present application outperform most neural network models (such as CNN models, PNCNN models, and LSTM models) in the prior art, but also the model optimization method of the present application can improve the performance of the prediction model.

[0187] In an embodiment of the present application, S300 will be further described and explained below. S300 can include but is not limited to the following steps.

[0188] The current molten pool image of the current additive manufacturing is collected by the high-speed camera real-time monitoring system, and the current molten pool image is input into the forming quality detection model.

[0189] In this step, the forming quality detection model is carried on the high-speed camera real-time monitoring system, the monitoring system shoots the current image, and the current image is input into the system for detection. The installation position of the high-speed camera real-time monitoring system is the same as the installation position of the aforementioned off-axis high-speed camera 103.

[0190] The quality prediction information is obtained by performing quality prediction on the current molten pool image by the forming quality detection model.

[0191] It should be noted that the quality prediction information includes: process parameters corresponding to this time of additive manufacturing, position information and categories of the molten pool, and a quality number of the forming quality corresponding to the molten pool. Among them, the process parameters refer to laser power, scanning speed and powder feeding speed.

[0192] Referring to Figure 9 , the Figure 9 The result graph of the high-speed camera real-time monitoring system provided by the embodiment of the application is shown. Through Figure 9 It can be known that the method provided by the application can predict the single-track forming quality of the cladding track in real time and accurately, can avoid the long-time post-processing work, and can provide data support for the forming quality prediction of different additive manufacturing materials.

[0193] Based on the above embodiment, the application further provides an additive manufacturing component forming quality monitoring system, and the above monitoring method is applied to the system. The system is composed of the following unit modules:

[0194] The image acquisition unit includes a paraxial high-speed camera 103 and a coaxial CCD camera 102.

[0195] Among them, the coaxial CCD camera 102 is used to acquire coaxial molten pool images. The paraxial high-speed camera 103 is used to acquire experimental molten pool images and paraxial molten pool images.

[0196] The data processing unit is used to pre-process the experimental molten pool image, the paraxial molten pool image and the coaxial molten pool image, and form a molten pool data set, a tuning training set and a test set;

[0197] The pre-training unit is used to build a neural network model, and pre-train the neural network model through the molten pool data set;

[0198] The tuning unit is used to tune the pre-trained neural network model through the tuning training set, test the performance of the tuned neural network model through the test set, and generate a forming quality detection model;

[0199] The high-speed camera real-time monitoring system is equipped with the forming quality detection model, which is used to acquire a current molten pool image, perform quality detection on the current molten pool image through the forming quality detection model, and output quality prediction information through the model prediction.

[0200] Optionally, the data processing unit, the pre-training unit and the tuning unit are integrated in a PC end.

[0201] In addition, the application further provides a storage medium, wherein the storage medium stores processor-executable instructions, and the processor-executable instructions are used to execute the additive manufacturing component forming quality monitoring method when executed by the processor.

[0202] The application has the following technical effects:

[0203] (1) The wetting angle feature and dilution rate feature of the molten pool or the cladding path formed in the laser cladding process are extracted as the feature parameters of the molten pool, and the wetting angle and the dilution rate can more comprehensively reflect the quality of the cladding path.

[0204] (2) The neural network model for prediction is obtained through pre-training and optimization training, which has higher accuracy and generalization ability, can quickly and accurately identify and classify the molten pool in different states in the additive manufacturing process, and predict the forming quality of the molten pool.

[0205] (3) The process of additive manufacturing is monitored and the quality is predicted through the high-speed camera monitoring system equipped with the prediction model, the process parameters can be corrected through the prediction results, the production cost is reduced; at the same time, the long time period of post-processing work is avoided, and data support can be provided for the forming quality prediction of different additive manufacturing materials.

[0206] The terms "first", "second", "third", "fourth" and the like in the specification of this application and in the above drawings, if any, are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0207] It should be understood that in this application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including single or multiple combinations of any combination. For example, at least one of a, b or c, can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0208] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0209] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0210] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0211] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store program codes.

[0212] For the step numbers in the above method embodiments, they are set only for the convenience of description and explanation, and do not make any limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

Claims

1. A method of monitoring the quality of the shaping of a component produced by additive manufacturing, characterized in that, The method comprises the following steps: controlling an additive manufacturing robot to perform an additive manufacturing experiment, collecting experimental molten pool images during the additive manufacturing experiment and preprocessing the experimental molten pool images to form a molten pool dataset; building a neural network model, training the neural network model through the molten pool dataset, obtaining a pre-trained neural network model, and optimizing the pre-trained neural network model to generate a forming quality detection model; controlling the additive manufacturing robot to perform additive manufacturing according to current process parameters, collecting a current molten pool image, and predicting the quality of the additive manufacturing workpiece through the current molten pool image and the forming quality detection model to obtain quality prediction information, wherein the quality prediction information includes process parameters corresponding to the additive manufacturing, position information and categories of the molten pool, and quality numbers of corresponding forming quality; wherein the optimization of the pre-trained neural network model to generate a forming quality detection model comprises: controlling the additive manufacturing robot to perform a plurality of additive manufacturing tests, collecting a plurality of coaxial molten pool images through a coaxial CCD camera and a plurality of off-axis high-speed camera images through an off-axis high-speed camera during the additive manufacturing tests; preprocessing the coaxial molten pool images and the off-axis molten pool images to form a test set and an optimization training set; training the pre-trained neural network model by taking the optimization training set as the input of the pre-trained neural network model and based on the weight file YOLOv5.pt of the pre-trained neural network model; evaluating the performance of the neural network model by using the test set, and outputting the forming quality detection model; wherein the preprocessing of the coaxial molten pool images and the off-axis molten pool images to form a test set and an optimization training set comprises: classifying the forming quality of a plurality of test additive workpieces obtained through the additive manufacturing tests according to the dilution rate to determine the quality number corresponding to each test additive workpiece; labeling the off-axis molten pool images to obtain label data corresponding to the off-axis molten pool images, and adding the quality number corresponding to the test additive workpiece to the corresponding label data; performing data augmentation on the labeled off-axis molten pool images, and classifying the augmented off-axis molten pool images into a test set and a to-be-optimized training set; extracting molten pool parameter information from the coaxial molten pool images, wherein the molten pool parameter information includes process parameters corresponding to the additive manufacturing test, the width, height and depth of the molten pool; adding the molten pool parameter information to the label data of the corresponding off-axis molten pool images in the to-be-optimized training set to form the optimization training set.

2. A method of monitoring the quality of a part formed by additive manufacturing according to claim 1, characterized in that, The method of controlling the additive manufacturing robot to perform an additive manufacturing experiment and collecting experimental molten pool images during the additive manufacturing experiment comprises: controlling the additive manufacturing robot to perform i additive manufacturing experiments, wherein the process parameters of each additive manufacturing experiment are different, and i additive workpieces are obtained; wherein the i-th additive workpiece corresponds to the i-th set of additive manufacturing experiments. When the additive manufacturing experiment is performed, a plurality of experimental molten pool images are collected by a paraxial high-speed camera to form an i th molten pool experimental data set; The i th molten pool experimental data set corresponds to the i th additive manufacturing experiment.

3. A method of monitoring the quality of a part formed by additive manufacturing according to claim 2, wherein, The preprocessing of the experimental molten pool images to form a molten pool data set comprises: According to the experimental time of the additive manufacturing experiment, the process of the additive manufacturing experiment is equally divided into ten progress intervals; The ten progress intervals are [0%, 10%), [10%, 20%), [20%, 30%), [30%, 40%), [40%, 50%), [50%, 60%), [60%, 70%), [70%, 80%), [80%, 90%), and [90%, 100%], respectively; In each of the molten pool experimental data sets, the experimental molten pool images with a progress of [0%, 10%), [10%, 20%), [70%, 80%), [80%, 90%), and [90%, 100%] are discarded, and a certain number of experimental molten pool images are randomly selected from the [20%, 30%) progress interval, the [30%, 40%) progress interval, the [40%, 50%) progress interval, the [50%, 60%) progress interval, and the [60%, 70%) progress interval, respectively; According to a wetting angle threshold, additive manufacturing parts with a wetting angle less than the wetting angle threshold are screened, and the screened additive manufacturing parts are classified according to a dilution rate to determine a quality number corresponding to each of the additive manufacturing parts; The wetting angle threshold is 80°; The randomly selected experimental molten pool images are subjected to label processing to obtain label data corresponding to the experimental molten pool images, the quality number is added to the label data of the corresponding experimental molten pool images, and the experimental molten pool images after label processing are subjected to data augmentation to form a molten pool data set.

4. A method of monitoring the quality of a part formed by additive manufacturing according to claim 3, wherein, The classification of the screened additive manufacturing parts according to the dilution rate to determine the quality number corresponding to each of the additive manufacturing parts comprises: The additive manufacturing parts with a dilution rate less than 10% are classified as number zero, the number zero is used to map that the dilution rate of the additive manufacturing parts is too small, and the quality level of the additive manufacturing parts is failed; The additive manufacturing parts with a dilution rate greater than or less than 10% and less than or equal to 15% are classified as number one, the number one is used to map that the dilution rate of the additive manufacturing parts is general, and the quality level of the additive manufacturing parts is passed; The additive manufacturing parts with a dilution rate greater than 15% and less than or equal to 25% are classified as number two, the number two is used to map that the dilution rate of the additive manufacturing parts is excellent, and the quality level of the additive manufacturing parts is excellent; The additive manufacturing parts with a dilution rate greater than 25% are classified as number three, the number three is used to map that the dilution rate of the additive manufacturing parts is too large, and the quality level of the additive manufacturing parts is failed.

5. A method of monitoring the quality of a part formed by additive manufacturing according to claim 4, wherein, The randomly selected experimental molten pool images are labeled to obtain label data corresponding to the experimental molten pool images, the quality number is added to the label data of the corresponding experimental molten pool images, and the experimental molten pool images after label processing are subjected to data augmentation, including: The target in the randomly selected experimental molten pool image is labeled by an image labeling tool, the target is a molten pool, the category and position information of the target are located, the label data of the experimental molten pool image is obtained, and the quality number is added to the label data of the corresponding experimental molten pool image; The experimental molten pool image is subjected to linear transformation and blur filtering processing; The processed experimental molten pool image is subjected to data augmentation to obtain a data augmented image, and the resolution of the data augmented image and the experimental molten pool image is adjusted so that the resolution of the data augmented image and the experimental molten pool image is the same.

6. A method of monitoring the quality of a part formed by additive manufacturing according to claim 1, wherein, The neural network model is built, the neural network model is trained by the molten pool data set, and a pre-trained neural network model is obtained, including: The molten pool data set is divided into a training set and a validation set in a 1:1 ratio; A neural network model is built based on YOLOv5, the training set and the validation set are used as inputs of the neural network model, the neural network model is pre-trained, and a pre-trained neural network model and a weight file YOLOv5.pt thereof are obtained.

7. An additive manufacturing part formation quality monitoring system, comprising: Including: An image acquisition unit including a paraxial high-speed camera and a coaxial CCD camera, used for acquiring experimental molten pool images, coaxial molten pool images and paraxial molten pool images; A data processing unit for preprocessing the experimental molten pool images, the coaxial molten pool images and the paraxial molten pool images to form a molten pool data set, a tuning training set and a test set; A pre-training unit for building a neural network model, training the neural network model by the molten pool data set, and obtaining a pre-trained neural network model; A tuning unit for tuning the pre-trained neural network model by the tuning training set, and testing the performance of the tuned neural network model by the test set to generate a forming quality detection model; A high-speed camera real-time monitoring system loaded with the forming quality detection model, used for acquiring a current molten pool image, detecting the quality of the current molten pool image by the forming quality detection model, and outputting quality prediction information predicted by the model, the quality prediction information including process parameters corresponding to the additive manufacturing, position information and category of the molten pool, and a quality number corresponding to the forming quality; Wherein, the pre-trained neural network model is tuned to generate a forming quality detection model, including: The additive manufacturing robot is controlled to perform multiple additive manufacturing tests, during which a plurality of coaxial molten pool images are acquired by a coaxial CCD camera and a plurality of paraxial molten pool images are acquired by a paraxial high-speed camera; The coaxial molten pool images and the paraxial molten pool images are preprocessed to form a test set and a tuning training set; The tuning training set is taken as an input of the pre-trained neural network model, and the pre-trained neural network model is trained on the basis of a weight file YOLOv5.pt of the pre-trained neural network model; The performance of the neural network model is evaluated by using the test set, and the forming quality detection model is output; The preprocessing of the coaxial molten pool image and the off-axis molten pool image to form a test set and a tuning training set comprises: According to the dilution rate, the test additive manufacturing parts are classified according to the forming quality, and the quality number corresponding to each test additive manufacturing part is determined; The off-axis molten pool image is labeled to obtain the label data corresponding to the off-axis molten pool image, and the quality number corresponding to the test additive manufacturing part is added to the corresponding label data; The off-axis molten pool image after label processing is subjected to data augmentation, and the augmented off-axis molten pool image is classified into a test set and a tuning training set; The molten pool parameter information in the coaxial molten pool image is extracted, and the molten pool parameter information includes the process parameters corresponding to the additive manufacturing test, the width, height and depth of the molten pool; The molten pool parameter information is added to the label data of the corresponding off-axis molten pool image in the tuning training set to form the tuning training set.

8. A storage medium having stored therein instructions executable by a processor, the instructions causing the processor to perform the method of any one of claims 1-7. The instructions executable by the processor, when executed by the processor, are used to perform the additive manufacturing part forming quality monitoring method of any one of claims 1-6.

Images