A method and device for predicting strain of laser directed energy deposition using online monitoring

Abstract

The present invention discloses a method and device for online monitoring of laser directed energy deposition strain prediction, the method comprising: acquiring an image data set during the manufacturing process of a processed part and performing data preprocessing to obtain a preprocessed image data set; introducing significant feature analysis to construct a multi-scale multi-modal fusion prediction model; performing laser directed energy deposition strain prediction on the preprocessed image data set based on the multi-scale multi-modal fusion prediction model to obtain a prediction result of the laser directed energy deposition strain value during the manufacturing process of the processed part. The device comprises: a laser DED manufacturing system and a manufacturing monitoring system. The embodiments of the present invention can improve the efficiency of laser directed energy deposition strain prediction and the accuracy of laser directed energy deposition strain prediction. As an online monitoring method and device for laser directed energy deposition strain prediction, the present invention can be widely used in the field of laser directed energy deposition technology.

Description

Technical Field

[0001] The present invention relates to the technical field of laser directed energy deposition, and in particular to an online monitoring laser directed energy deposition strain prediction method and device. Background Art

[0002] Laser directed energy deposition (DED) technology is an advanced additive manufacturing process, but it is a non-equilibrium processing technology with rapid cooling rates and high thermal gradients. Numerous complex physical and chemical changes occur in a very short period of time, making it difficult to ensure the consistency of the processed parts. After the laser beam scans, the material near the molten pool is subjected to repeated thermal cycles. The thermal gradients and thermal cycles during the manufacturing process will lead to the accumulation of internal stress, further causing residual stress and strain in the part. Currently, strain prediction for the laser directed energy deposition process is mainly based on finite element analysis, which requires a large amount of simplification of the actual physical process and requires a huge amount of calculation and time. In addition, traditional stress and strain measurement methods such as drilling technology, X-ray technology, and neutron diffraction technology are non-in-situ measurements and cannot detect the strain data of this weld heat-affected zone. In other words, it is impossible to quickly and accurately predict the strain of the laser DED process. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide an online monitoring laser directed energy deposition strain prediction method and device, which can improve the efficiency and accuracy of laser directed energy deposition strain prediction.

[0004] The first technical solution adopted by the present invention is: an online monitoring laser directed energy deposition strain prediction method, comprising the following steps:

[0005] Acquire an image data set during the manufacturing process of a machined part and perform data preprocessing to obtain a preprocessed image data set;

[0006] Introducing significant feature analysis to build a multi-scale and multi-modal fusion prediction model;

[0007] The laser directed energy deposition strain prediction is performed on the preprocessed image dataset based on the multi-scale and multi-modal fusion prediction model, and the prediction results of the laser directed energy deposition strain value in the manufacturing process of the processed parts are obtained.

[0008] Furthermore, the step of acquiring an image dataset during the manufacturing process of a processed part and performing data preprocessing to obtain a preprocessed image dataset specifically includes:

[0009] Acquire an image dataset during the manufacturing process of a machined part, the image dataset comprising a molten pool image, a thermal imaging image, and a manufacturing sidewall image;

[0010] According to the aspect ratio of the molten pool, the molten pool image is preprocessed to obtain a pseudo-color image of the molten pool;

[0011] Perform matching calculation between adjacent images of the manufactured sidewall image to obtain the strain thermodynamic map;

[0012] Combine the pseudo-color image of the melt pool, the thermal image, and the strain thermogram, and perform spatiotemporal alignment and sliding window segmentation in sequence to obtain the segmented pseudo-color image of the melt pool, the segmented thermal image, and the segmented strain thermogram.

[0013] Get parameter vector;

[0014] The input data set is constructed by combining the pseudo-color image of the divided melt pool, the thermal imaging image after division, and the parameter vector;

[0015] The divided strain thermodynamic map is used as label data and combined with the input dataset to obtain the preprocessed image dataset.

[0016] Furthermore, the step of performing image preprocessing on the molten pool image according to the aspect ratio of the molten pool to obtain a pseudo-color image of the molten pool specifically includes:

[0017] Determine the four corner points of the molten pool area based on the molten pool image;

[0018] The four corner points of the molten pool area are calculated using the perspective transformation function of OpenCV to obtain the homography matrix;

[0019] Performing perspective transformation correction on the melt pool image according to the homography matrix to obtain a corrected melt pool image;

[0020] The corrected molten pool image is filtered to remove random noise points to obtain a filtered molten pool image;

[0021] The filtered melt pool image is subjected to binarization and threshold segmentation in sequence to obtain the melt pool contour;

[0022] The aspect ratio information of the melt pool contour is obtained through the minimum enclosing rectangle and normalized to obtain a pseudo-color image of the melt pool.

[0023] Furthermore, the step of performing matching calculation between adjacent images of the manufacturing sidewall image to obtain the strain thermodynamic map specifically includes:

[0024] The manufacturing sidewall images acquired at adjacent moments are processed by pixel feature point correlation matching using a sub-pixel matching algorithm to obtain the manufacturing sidewall images after correlation matching;

[0025] The full-field strain calculation is performed on the associated matched manufacturing sidewall image according to the full-field three-dimensional spatial coordinates to obtain the strain thermodynamic map.

[0026] Furthermore, the parameter vector specifically includes laser power, scanning speed, powder feeding rate, layer thickness, scanning strategy, gas flow rate, beam diameter and defocus, wherein:

[0027] The laser power represents the laser emission power set by the laser;

[0028] The scanning speed represents the displacement speed of the cladding head;

[0029] The powder feeding rate refers to the powder flow rate emitted by the powder feeder and collected by the cladding head;

[0030] The layer thickness indicates the height of the cladding head after printing a layer during the manufacturing process of the processed parts;

[0031] The scanning strategy represents the scanning rules preset during the manufacturing process of the machined parts;

[0032] The gas flow rate indicates the flow rate of the protective gas;

[0033] The beam diameter refers to the diameter of the laser beam after passing through the focusing lens and the collimating lens.

[0034] The defocus amount indicates the distance that the focus deviates from the processed part.

[0035] Furthermore, the multi-scale multimodal fusion prediction model includes a multi-scale multimodal feature extraction module, a significant feature analysis module and a strain value prediction module. The output end of the multi-scale multimodal feature extraction module is connected to the input end of the significant feature analysis module, and the output end of the significant feature analysis module is connected to the input end of the strain value prediction module, wherein:

[0036] The multi-scale multi-modal feature extraction module includes a shallow feature extraction module, a convolution operation module and a fusion module, wherein the convolution operation module includes a two-dimensional convolution layer, a batch normalization and a linear rectification unit;

[0037] The significant feature analysis module includes a t-distributed random neighborhood embedding layer and a principal component analysis layer;

[0038] The strain value prediction module includes a support vector machine.

[0039] Furthermore, the step of performing laser directed energy deposition strain prediction on the preprocessed image dataset based on the multi-scale multi-modal fusion prediction model to obtain the laser directed energy deposition strain value prediction result during the manufacturing process of the processed parts specifically includes:

[0040] Input the preprocessed image dataset into the multi-scale multimodal fusion prediction model;

[0041] The multi-scale multi-modal feature extraction module based on the multi-scale multi-modal fusion prediction model performs shallow feature extraction, downsampling convolution operation and feature fusion processing on the pre-processed image dataset to obtain the fused image features;

[0042] Based on the salient feature analysis module of the multi-scale multi-modal fusion prediction model, the fused image features are subjected to data dimensionality reduction processing to obtain the reduced-dimensional image features;

[0043] Based on the strain value prediction module of the multi-scale and multi-modal fusion prediction model, the reduced-dimensional image features are classified and the laser directed energy deposition strain is predicted to obtain the laser directed energy deposition strain value prediction results during the manufacturing process of processed parts.

[0044] The second technical solution adopted by the present invention is: an online monitoring laser directed energy deposition strain prediction device, including a laser DED manufacturing system and a manufacturing monitoring system, wherein:

[0045] The laser DED manufacturing system is used to provide a parameter vector;

[0046] The manufacturing monitoring system is used to obtain image data sets during the manufacturing process of machined parts.

[0047] Furthermore, the laser DED manufacturing system includes a robot, a laser, a water cooler, a powder feeding system and a cladding head, wherein:

[0048] The robot is used to provide the displacement required in the manufacturing process of the machined parts;

[0049] The laser is used to provide laser emission power;

[0050] The water cooler is used to cool the laser;

[0051] The powder feeding system is used to eject metal powder through the cladding head to provide raw materials for the manufacturing process of machined parts;

[0052] The cladding head is used to gather metal powder at one point.

[0053] Furthermore, the manufacturing monitoring system includes a molten pool industrial camera, a thermal imager and a strain vision sensor, wherein:

[0054] The molten pool industrial camera is used to obtain molten pool images;

[0055] The thermal imager is used to obtain thermal imaging images;

[0056] The strain vision sensor is used to obtain a manufacturing sidewall image.

[0057] The beneficial effects of the method and system of the present invention are as follows: the present invention obtains an image data set in the process of machining parts manufacturing and performs data preprocessing to obtain a preprocessed image data set, and by applying known information such as equipment, machining parameters, and machining process, the information is more fully utilized, and the accuracy of strain prediction is improved without adding other monitoring tools, and then significant feature analysis is introduced to construct a multi-scale multi-modal fusion prediction model, and finally, based on the multi-scale multi-modal fusion prediction model, laser directed energy deposition strain prediction is performed on the preprocessed image data set to obtain the laser directed energy deposition strain value prediction result in the process of machining parts manufacturing. The multi-scale multi-modal fusion framework integrates and analyzes the local details and global context of multi-source data, which can effectively improve the feature extraction accuracy and the prediction ability of the model, and improve the prediction accuracy through significant feature analysis, and inputs the pseudo-color map of the aspect ratio of the molten pool in the local window as the image feature into the deep learning model to extract features, and the process parameters and thermal imaging are combined to extract features. Figure 1 Taking the same into consideration, it provides rich information for strain prediction, greatly reduces the time of strain prediction, and improves the efficiency of strain prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of the steps of an online monitoring laser directed energy deposition strain prediction method of the present invention;

[0059] Figure 2 This is a flowchart of the steps for strain prediction of laser directed energy deposition provided by an embodiment of the present invention;

[0060] Figure 3 is a structural diagram of a laser DED manufacturing system provided by a specific embodiment of the present invention;

[0061] Figure 4 It is a structural schematic diagram of an online monitoring laser directed energy deposition strain prediction device of the present invention;

[0062] Figure 5 Schematic diagram of the structure of a multi-scale multi-modal fusion prediction model constructed in an embodiment of the present invention;

[0063] Figure 6 Schematic diagram of four different laser scanning modes provided by an embodiment of the present invention.

[0064] Figure numerals: 1. Robot; 2. L-shaped light baffle; 3. Rangefinder bracket; 4. Melt pool industrial camera; 5. Active light source; 6. Macro lens; 7. Cladding head; 8. Thin-wall manufacturing; 9. Thermal imager; 10. Mini tripod; 11. Vise; 12. Cladding platform; 13. Cross bracket; 14. Strain vision sensor; 15. Measuring head; 16. Blue light LED. DETAILED DESCRIPTION

[0065] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0066] First of all, it should be noted that laser directed energy deposition (DED) technology is an advanced additive manufacturing process that uses a high-energy-density heat source to melt metal powder and deposit it layer by layer. It has a wide range of material applicability, excellent mechanical properties, fast manufacturing speed, high design flexibility, and high efficiency in part repair and modification. DED technology can achieve the manufacture of complex geometries and is applicable to a variety of industrial fields, including aerospace, automotive, and medical. It also supports the processing of multiple materials and composite materials, providing advantages that traditional manufacturing methods cannot match. However, it is a non-equilibrium processing technology with a rapid cooling rate and high thermal gradient. Many complex physical and chemical changes will occur in a very short period of time, making it difficult to ensure the consistency of the processed parts. After the laser beam scans, the material near the molten pool will be subjected to repeated thermal cycles. The thermal gradient and thermal cycles during the manufacturing process will lead to the accumulation of internal stress, further causing residual stress and strain inside the part.

[0067] Currently, strain prediction for laser directed energy deposition processes is mainly based on finite element analysis, which requires a lot of simplification of the real physical process and requires huge computational effort and time.

[0068] Based on this, Figure 2 As shown, the embodiment of the present invention obtains a molten pool image, a thermal imaging image and an image of the manufacturing sidewall during the manufacturing process, obtains the molten pool length and molten pool width according to the molten pool image, processes the molten pool aspect ratio into a molten pool aspect ratio pseudo-color image, matches adjacent images according to the manufacturing sidewall image, calculates a strain thermodynamic map, aligns the molten pool aspect ratio pseudo-color map, thermal imaging map and strain thermodynamic map and performs sliding window division, uses the parameter vector in the manufacturing process, the molten pool aspect ratio pseudo-color map 32×32 and the thermal imaging map 32×32 as input data, labels them with the strain mean within 32×32 of the corresponding position, and divides the processed data into a training set and a test set in proportion; a deep convolutional neural network is constructed, and the strain value is predicted by the constructed multi-scale multi-modal fusion framework; the method can quickly and accurately predict the strain value, greatly saving the strain time required by the simulation method.

[0069] Reference Figure 1 The present invention provides an online monitoring method for strain prediction of laser directed energy deposition, which comprises the following steps:

[0070] S100, acquiring an image dataset during the manufacturing process of a processed part and performing data preprocessing to obtain a preprocessed image dataset;

[0071] S110, acquiring an image dataset during the manufacturing process of a machined part, the image dataset including a molten pool image, a thermal imaging image, and a manufacturing sidewall image;

[0072] Specifically, the acquisition of thermal images is first explained. The temperature information of the manufacturing process is collected by a thermal imager placed on the manufacturing side wall. The acquisition frequency of the thermal imager is 60Hz, and the acquisition temperature range is 0℃-2000℃. Secondly, the acquisition of the molten pool image is explained. The molten pool image is obtained by a paraxially mounted molten pool industrial camera. The built-in active vision and macro lens are used to ensure image quality and obtain rich molten pool information. The molten pool industrial camera has an acquisition frequency of 60Hz and a resolution of 1280×1024. Finally, the acquisition of the manufacturing side wall image is explained. The manufacturing side wall image obtains a high-definition image through two measuring heads.

[0073] S120, performing image preprocessing on the melt pool image according to the aspect ratio of the melt pool to obtain a pseudo-color image of the melt pool;

[0074] Specifically, the four corner points of the melt pool area are determined according to the melt pool image; the four corner points of the melt pool area are calculated by the perspective transformation function of OpenCV to obtain the homography matrix; the melt pool image is perspective transformed and corrected according to the homography matrix to obtain the corrected melt pool image; the corrected melt pool image is filtered to remove random noise points to obtain the filtered melt pool image; the filtered melt pool image is binarized and threshold segmented in turn to obtain the melt pool contour; the aspect ratio information of the melt pool contour is obtained by the minimum circumscribed rectangle and normalized to obtain a pseudo-color image of the melt pool.

[0075] In this embodiment, due to the 30° tilt angle between the camera and the molten pool, the captured image has perspective distortion, especially in the measurement of the length and area of ​​the molten pool. The image is corrected to a top-down perspective through perspective transformation to eliminate the influence of the angle. Specifically, the following steps are included: first, the four corner points of the molten pool area are selected from the distorted image as reference points, and then the homography matrix is ​​calculated using the perspective transformation function of OpenCV based on the actual positions of these points. Finally, the original image is corrected to a top-down perspective through perspective transformation to eliminate the geometric distortion caused by the angle. Denoising, enhancement and binarization are performed on the corrected image, and the molten pool contour is extracted using threshold segmentation. The width and length of the molten pool are obtained through the minimum circumscribed rectangle. The aspect ratio of the molten pool is calculated, and then the aspect ratio information of the molten pool is normalized. The corresponding pseudo-color image is generated through pseudo-color mapping.

[0076] S130, performing matching calculation between adjacent images of the manufacturing sidewall image to obtain a strain thermodynamic map;

[0077] Specifically, the manufacturing sidewall images acquired at adjacent moments are processed with pixel feature point association matching through a sub-pixel matching algorithm to obtain the associated-matched manufacturing sidewall image; the full-field strain of the associated-matched manufacturing sidewall image is calculated according to the full-field three-dimensional spatial coordinates to obtain a strain thermodynamic map.

[0078] Furthermore, it should be noted that the full-field three-dimensional spatial coordinates are calculated by the measuring head's built-in software after the two measuring heads are calibrated, which is equivalent to the three-dimensional coordinates automatically calculated by the binocular camera after calibration using the calibration plate.

[0079] In this embodiment, the collected high-definition manufacturing sidewall images are imported into the algorithm frame by frame, and the pixel points on the two adjacent images are associated through the image matching algorithm. The corresponding position on the subsequent image is found for the pixel points on the previous image, and then the full-field three-dimensional spatial coordinates obtained previously are used to calculate the full-field strain.

[0080] A sub-pixel matching algorithm is used to match feature points of manufacturing sidewall images acquired at adjacent moments. The algorithm calculates the similarity of each sub-region using a sub-region similarity function to obtain a matching result. The translation and deformation components of the reference sub-region are estimated based on the matching structure. These parameter values ​​are then used as initial values ​​for iteration to calculate the Lagrangian strain value. The strain value of the pixel points within the ROI region of the manufacturing layer wall in the manufacturing sidewall image is calculated.

[0081] Among them, the calculation expression of the sub-region similarity function is:

[0082]

[0083] In the above formula, C LS It represents the sub-region similarity value obtained after the reverse Newton iteration when each sub-region is matched, M represents the sub-region radius matched by the two-dimensional digital image correlation method, i represents the horizontal image pixel coordinate, j represents the vertical image pixel coordinate, f(x i ,y i ) represents the gray value distribution function of the reference image, represents the average grayscale value of the sub-area of ​​the reference image, represents the average grayscale value of the deformed image sub-area, g(x' i , y' i ) represents the grayscale value distribution function of the deformed image, Δ(f 2 ) represents the grayscale value variance of the reference image sub-area, Δ(g 2 ) represents the grayscale value variance of the deformed image.

[0084] S140, combining the molten pool pseudo-color image, the thermal imaging image, and the strain thermogram and sequentially performing spatiotemporal alignment and sliding window segmentation processing to obtain a segmented molten pool pseudo-color image, a segmented thermal imaging image, and a segmented strain thermogram;

[0085] Specifically, the obtained pseudo-color map of the melt pool aspect ratio, thermal imaging map, and strain thermodynamic map are aligned in time and space, and the corresponding coordinate positions and time of the three images are aligned. Then, a 32×32 sliding window is divided. The divided pseudo-color map of the melt pool aspect ratio 32×32, the thermal imaging map 32×32 and the parameter vector are used as input data, and the strain mean within the corresponding 32×32 is used as the label. Then, the labeled data is divided into training data and test data. The data information collected by multiple sensors is aligned in time and position, and the information in the same window is used as the feature, which provides rich features for accurate strain prediction.

[0086] S150, obtaining a parameter vector;

[0087] Specifically, the parameter vector includes laser power, scanning speed, powder feeding rate, layer thickness, scanning strategy, gas flow rate, beam diameter and defocus, among which, laser power represents the laser emission power set by the laser; scanning speed represents the displacement speed of the cladding head; powder feeding rate represents the powder flow rate emitted by the powder feeder and collected by the cladding head; layer thickness represents the lifting height of the cladding head after printing a layer during the manufacturing process of the processed parts; scanning strategy represents the preset scanning rules during the manufacturing process of the processed parts; gas flow rate represents the flow rate of the shielding gas; beam diameter represents the diameter reached by the laser beam after passing through the focusing mirror and the collimating mirror; defocus represents the distance that the focus deviates from the processed part.

[0088] Furthermore, it is important to specify that laser power refers to the laser emission power set by the laser; scanning speed refers to the displacement speed of the cladding head; powder feed rate refers to the powder flow rate emitted by the powder feeder and collected by the cladding head; layer thickness refers to the height at which the cladding head is lifted after printing a layer during the manufacturing process; scanning strategy refers to several preset scanning speeds during the manufacturing process; gas flow refers to the flow rate of shielding gas; beam diameter refers to the diameter of the laser beam after passing through the focusing and collimating lenses; and defocus refers to the distance the focal point deviates from the workpiece, which is mainly used to change the power density radiated to the workpiece surface. A focal plane located above the workpiece is considered positive defocus, while a focal plane located above the workpiece is considered negative defocus.

[0089] like Figure 6 As shown, the scanning strategy in the embodiment of the present invention is explained, wherein, Figure 6(a) shows the inter-layer dwell S-line scanning. The laser is turned off 10 mm before the robot is about to lift. The laser remains off when the layer is raised. After lifting, the laser is turned on after moving 10 mm in the opposite direction. This ensures a certain dwell time between layers, thereby reducing heat accumulation and ensuring that no bulges are generated on both sides of the thin-walled parts. Figure 6 (b) represents S-shaped scanning, where the robot end scans layer by layer in an S-shaped manner without stopping between layers. This type of scanning can print more efficiently and save printing time. Figure 6 (c) represents unidirectional scanning, where the robot scans in a single direction, starting from one side and ending at the other side each time. This scanning method can allow for longer inter-layer cooling time, further reducing heat accumulation, but is more time-consuming. Figure 6 (d) shows double-layer S-shaped scanning, where the adjacent two layers are scanned in a unidirectional manner, and the two layers are scanned in an S-shaped manner, which can combine the advantages of both to ensure a certain efficiency while reducing heat accumulation.

[0090] S160, combining the divided molten pool pseudo-color image, the divided thermal imaging image, and the parameter vector to construct an input data set;

[0091] S170 , using the divided strain thermodynamic map as label data and combining it with the input data set to obtain a preprocessed image data set.

[0092] S200, introduce significant feature analysis and build a multi-scale multi-modal fusion prediction model;

[0093] Specifically, if Figure 5 As shown, the multi-scale multimodal fusion prediction model includes a multi-scale multimodal feature extraction module, a significant feature analysis module and a strain value prediction module. The output end of the multi-scale multimodal feature extraction module is connected to the input end of the significant feature analysis module, and the output end of the significant feature analysis module is connected to the input end of the strain value prediction module. The multi-scale multimodal feature extraction module includes a shallow feature extraction module, a convolution operation module and a fusion module. The convolution operation module includes a two-dimensional convolution layer, a batch normalization and a linear rectification unit; the significant feature analysis module includes a t-distributed random neighborhood embedding layer and a principal component analysis layer; the strain value prediction module includes a support vector machine.

[0094] S300, based on the multi-scale multi-modal fusion prediction model, the laser directed energy deposition strain is predicted for the preprocessed image data set to obtain the prediction result of the laser directed energy deposition strain value in the manufacturing process of the processed parts.

[0095] S310, inputting the preprocessed image dataset into a multi-scale multi-modal fusion prediction model;

[0096] S320, a multi-scale multi-modal feature extraction module based on a multi-scale multi-modal fusion prediction model, sequentially performing shallow feature extraction, downsampling convolution operation, and feature fusion processing on the preprocessed image dataset to obtain fused image features;

[0097] Specifically, the pseudo-color image of the melt pool aspect ratio and the thermal image are first extracted into a 16×16×32 feature map through shallow feature extraction, where the shallow feature extraction first uses 32 convolution kernels, each with a size of 3×3 and a stride of 2, and then uses 64 convolution kernels, each with a size of 1×1 and a stride of 1; then, through a series of convolution operations, the shallow extracted feature map is downsampled to 8×8×64 and 4×4×128 in sequence. In this process, jump connections are used to retain feature information, and the convolution operations of each downsampling layer are implemented in the form of "Conv2d+BN+ReLU", that is, two-dimensional convolution layer, batch normalization and linear rectification unit; after feature extraction, the extracted feature map is fused with the parameter vector through the fusion module, and the fusion module uses a multi-head attention mechanism to dynamically select and combine features.

[0098] S330, performing data dimensionality reduction processing on the fused image features based on the salient feature analysis module of the multi-scale multi-modal fusion prediction model to obtain image features after dimensionality reduction;

[0099] Specifically, the fused features are processed by t-SNE (t-distributed stochastic neighbor embedding layer) and PCA (principal component analysis layer) for dimensionality reduction to improve the efficiency and visualization effect of subsequent analysis.

[0100] S340, based on the strain value prediction module of the multi-scale multi-modal fusion prediction model, classifies the image features after dimensionality reduction and predicts the laser directed energy deposition strain to obtain the laser directed energy deposition strain value prediction results in the manufacturing process of the processed parts.

[0101] Specifically, the significant features after dimensionality reduction are input into SVM (support vector machine) for classification and analysis, and finally the prediction results of strain values ​​are output.

[0102] Reference Figure 4 , an online monitoring laser directed energy deposition strain prediction device, including a laser DED manufacturing system and a manufacturing monitoring system, wherein:

[0103] The laser DED manufacturing system is used to provide the parameter vector;

[0104] Specifically, the laser DED manufacturing system includes a robot, a laser, a water cooler, a powder feeding system and a cladding head. The robot is used to provide the displacement required in the manufacturing process of the processed parts; the laser is used to provide the laser emission power; the water cooler is used to cool the laser; the powder feeding system is used to eject metal powder through the cladding head to provide raw materials for the manufacturing process of the processed parts; and the cladding head is used to gather the metal powder at one point.

[0105] In this embodiment, if Figure 3 As shown, robot 1 provides the displacement required for manufacturing and can ensure the precise displacement of the actuator end. The laser emits a high-energy laser beam with a laser power range of 0 to 20,000 W, providing the energy required for the manufacturing process. The water chiller cools the laser. The powder feeder sprays metal powder through the cladding head to provide the required raw materials for manufacturing. The cladding head is installed at the end of the robot and is the actuator for printing and manufacturing. It can not only gather the metal powder to one point, but also accurately output the laser beam to the powder gathering point to complete the specific manufacturing process.

[0106] Manufacturing monitoring systems are used to acquire image datasets during the manufacturing process of machined parts.

[0107] Specifically, the manufacturing monitoring system includes a molten pool industrial camera, a thermal imager and a strain vision sensor, wherein the molten pool industrial camera is used to obtain molten pool images; the thermal imager is used to obtain thermal imaging images; and the strain vision sensor is used to obtain manufacturing sidewall images.

[0108] In this embodiment, if Figure 4 As shown, the manufacturing monitoring system mainly consists of three sensors, namely a molten pool industrial camera 4, a thermal imager 9 and a strain vision sensor 14; the molten pool industrial camera is mounted on a paragon bracket 3 and moves synchronously with the cladding head 7. A macro lens 6 is mounted on it and an active light source 5 is provided inside it to provide illumination; the thermal imager 9 is mounted on a micro tripod 10 and placed together on the cladding platform 12, located on the left side of the manufactured part and perpendicular to the side wall of the manufactured part; the strain vision sensor 14 is mounted with two measuring heads 15 for collecting images of the side wall of the manufactured part, and the strain vision sensor is mounted on a cross bracket 13, with a blue light LED 16 placed next to it to provide sufficient light source for strain image acquisition. An L-shaped light baffle 2 is set next to the manufactured part to reduce the influence of the laser on image acquisition.

[0109] The molten pool industrial camera 4 is mounted on the paraxial bracket 3 and moves synchronously with the cladding head 7. A macro lens 6 is installed on it to filter and reduce the interference of the laser on the image, and an active light source 5 is inside it to provide the lighting required for image acquisition. The molten pool industrial camera is at a 30° angle to the cladding head, with an acquisition frame rate of 60Hz and a resolution of 1280×1024, which can capture images of the molten pool during the manufacturing process.

[0110] The thermal imager 9 is mounted on a miniature tripod 10 and placed together on a cladding platform 12. A vise 11 is provided under the cladding platform, located on the left side of the manufactured part and perpendicular to the side wall of the manufactured part. The thermal imager has a temperature measurement range of 0 to 2000°C and a sampling frequency of 60Hz. It can collect the temperature of the part during the manufacturing process and the temperature information within the ROI area.

[0111] The strain vision sensor 14 is equipped with two measuring heads 15 for collecting images of the side walls of the manufactured parts. The strain vision sensor is installed on the manufactured thin wall 8 on the cross bracket 13, and a blue light LED 16 is placed next to it to provide sufficient light for strain image acquisition. An L-shaped light baffle 2 is set next to the manufactured part to reduce the impact of the laser on image acquisition.

[0112] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0113] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for predicting strain of laser directed energy deposition by online monitoring, characterized in that: The following steps are involved: Acquire an image dataset during the manufacturing process of a machined part, the image dataset comprising a molten pool image, a thermal imaging image, and a manufacturing sidewall image; Determine the four corner points of the molten pool area based on the molten pool image; The four corner points of the molten pool area are calculated using the perspective transformation function of OpenCV to obtain the homography matrix; Performing perspective transformation correction on the melt pool image according to the homography matrix to obtain a corrected melt pool image; The corrected molten pool image is filtered to remove random noise points to obtain a filtered molten pool image; The filtered melt pool image is subjected to binarization and threshold segmentation in sequence to obtain the melt pool contour; The aspect ratio information of the melt pool outline is obtained through the minimum enclosing rectangle and normalized to obtain a pseudo-color image of the melt pool; The manufacturing sidewall images acquired at adjacent moments are processed by pixel feature point correlation matching using a sub-pixel matching algorithm to obtain the manufacturing sidewall images after correlation matching; The full-field strain calculation is performed on the associated matched manufacturing sidewall image according to the full-field three-dimensional spatial coordinates to obtain the strain thermodynamic map; Combine the pseudo-color image of the melt pool, the thermal image, and the strain thermogram, and perform spatiotemporal alignment and sliding window segmentation in sequence to obtain the segmented pseudo-color image of the melt pool, the segmented thermal image, and the segmented strain thermogram. Get parameter vector; The input data set is constructed by combining the pseudo-color image of the divided melt pool, the thermal imaging image after division, and the parameter vector; The divided strain thermodynamic map is used as label data and combined with the input dataset to obtain the preprocessed image dataset; Introducing significant feature analysis to build a multi-scale and multi-modal fusion prediction model; The laser directed energy deposition strain prediction is performed on the preprocessed image dataset based on the multi-scale and multi-modal fusion prediction model, and the prediction results of the laser directed energy deposition strain value in the manufacturing process of the processed parts are obtained.

2. The method for predicting strain of laser directed energy deposition by online monitoring according to claim 1, characterized in that: The parameter vector specifically includes laser power, scanning speed, powder feeding rate, layer thickness, scanning strategy, gas flow rate, beam diameter and defocus, wherein: The laser power represents the laser emission power set by the laser; The scanning speed represents the displacement speed of the cladding head; The powder feeding rate refers to the powder flow rate emitted by the powder feeder and collected by the cladding head; The layer thickness indicates the height of the cladding head after printing a layer during the manufacturing process of the processed parts; The scanning strategy represents the scanning rules preset during the manufacturing process of the machined parts; The gas flow rate indicates the flow rate of the protective gas; The beam diameter refers to the diameter of the laser beam after passing through the focusing lens and the collimating lens. The defocus amount indicates the distance that the focus deviates from the processed part.

3. The method for predicting strain of laser directed energy deposition by online monitoring according to claim 2, characterized in that: The multi-scale multi-modal fusion prediction model includes a multi-scale multi-modal feature extraction module, a significant feature analysis module and a strain value prediction module. The output end of the multi-scale multi-modal feature extraction module is connected to the input end of the significant feature analysis module, and the output end of the significant feature analysis module is connected to the input end of the strain value prediction module, wherein: The multi-scale multi-modal feature extraction module includes a shallow feature extraction module, a convolution operation module and a fusion module, wherein the convolution operation module includes a two-dimensional convolution layer, a batch normalization and a linear rectification unit; The significant feature analysis module includes a t-distributed random neighborhood embedding layer and a principal component analysis layer; The strain value prediction module includes a support vector machine.

4. The method for predicting strain of laser directed energy deposition by online monitoring according to claim 3, characterized in that: The step of performing laser directed energy deposition strain prediction on the preprocessed image data set based on the multi-scale multi-modal fusion prediction model to obtain the laser directed energy deposition strain value prediction result during the manufacturing process of the processed parts specifically includes: Input the preprocessed image dataset into the multi-scale multimodal fusion prediction model; The multi-scale multi-modal feature extraction module based on the multi-scale multi-modal fusion prediction model performs shallow feature extraction, downsampling convolution operation and feature fusion processing on the pre-processed image dataset to obtain the fused image features; Based on the salient feature analysis module of the multi-scale multi-modal fusion prediction model, the fused image features are subjected to data dimensionality reduction processing to obtain the reduced-dimensional image features; Based on the strain value prediction module of the multi-scale and multi-modal fusion prediction model, the reduced-dimensional image features are classified and the laser directed energy deposition strain is predicted to obtain the laser directed energy deposition strain value prediction results during the manufacturing process of processed parts.

5. An online monitoring laser directed energy deposition strain prediction device, applied to the online monitoring laser directed energy deposition strain prediction method according to claim 1, characterized in that: It includes laser DED manufacturing system and manufacturing monitoring system, including: The laser DED manufacturing system is used to provide a parameter vector; The manufacturing monitoring system is used to obtain image data sets during the manufacturing process of machined parts.

6. The on-line monitoring laser directed energy deposition strain prediction device according to claim 5, characterized in that: The laser DED manufacturing system includes a robot, a laser, a water cooler, a powder feeding system and a cladding head, wherein: The robot is used to provide the displacement required in the manufacturing process of the machined parts; The laser is used to provide laser emission power; The water cooler is used to cool the laser; The powder feeding system is used to eject metal powder through the cladding head to provide raw materials for the manufacturing process of machined parts; The cladding head is used to gather metal powder at one point.

7. The on-line monitoring laser directed energy deposition strain prediction device according to claim 5, characterized in that: The manufacturing monitoring system includes a molten pool industrial camera, a thermal imager, and a strain vision sensor, wherein: The molten pool industrial camera is used to obtain molten pool images; The thermal imager is used to obtain thermal imaging images; The strain vision sensor is used to obtain a manufacturing sidewall image.

Images