Methods and systems for quality monitoring and defect location in selective laser melting process

Abstract

This invention discloses a method and system for quality monitoring and defect localization in a selected area laser melting process. The invention involves real-time acquisition of molten pool images during the selected area laser melting process; extraction of molten pool contours from images belonging to the same printing layer, calculating the molten pool area and the center point of the minimum bounding rectangle of the molten pool contour; creation of a blank image of the same size as the molten pool images, and marking the center point of the minimum bounding rectangle of each molten pool contour at the corresponding position in the blank image; grid division of the blank image, filling the marked grid with the corresponding molten pool area value to generate a grid matrix; segmentation of the grid matrix, converting each segment into a grayscale sub-image; and inputting the grayscale sub-images into a trained neural network model to obtain quality classification results.

Description

Technical Field

[0001] This invention relates to the field of laser printing technology, and in particular to a method and system for quality monitoring of selective laser melting process. Background Technology

[0002] Additive manufacturing technology is constantly changing traditional production methods with its flexible and customized production and digital manufacturing models. Powder bed laser selective melting (LPBF) is considered the most promising mainstream technology because it can use a wide range of metal powder materials and manufacture lightweight metal structural parts with complex structures and composite materials without the need for binders. It is widely used in aerospace, military, medical device manufacturing, vehicle and general metal parts manufacturing.

[0003] Currently, LPBF technology is still immature in terms of manufacturing processes, equipment control, and non-destructive testing. Defects may appear in the manufactured metal workpieces, significantly affecting the strength, surface roughness, and other quality aspects of the metal structure, thus reducing the yield rate. Research on improving yield rate mainly focuses on three aspects: 1) real-time monitoring of the workpiece during processing; 2) real-time monitoring and feedback control of the molten pool morphology during processing; and 3) automatic real-time monitoring of powder spreading quality. Among these, real-time monitoring and feedback control of the molten pool morphology is of greatest interest. This involves using a high-speed camera to monitor the metal powder additive manufacturing process online in real time, processing images, extracting feature information, analyzing results, and providing feedback control to the processing. This effectively improves processing quality, enabling printing quality monitoring and defect location during the laser selective melting process, laying the technical foundation for closed-loop feedback control of quality defects in the forming process.

[0004] The document with publication number CN 113077423 A discloses a laser selective melting (SLM) molten pool image analysis system based on convolutional neural networks. This system can classify and identify molten pool images, capture the size, quantity, and dynamic behavior changes of the molten pool and sputtering contours, and evaluate and analyze molten pool images during SLM. However, this method directly uses molten pool images for neural network classification. Since SLM printing speed is high, the number of molten pool images obtained is often massive. This system cannot handle the complex processing of a large number of molten pool images while simultaneously providing real-time monitoring, and it cannot achieve defect localization in the defect analysis of the molten pool images. Summary of the Invention

[0005] Purpose of the invention: This invention addresses the problems existing in the prior art by providing a method and system for quality monitoring of the selective laser melting process.

[0006] Technical solution: The quality monitoring method for the selective laser melting process described in this invention includes the following steps:

[0007] (1) Real-time acquisition of images of the molten pool during the selected area laser melting process;

[0008] (2) Extract the molten pool contour for each molten pool image belonging to the same printing layer, and calculate the molten pool area and the center point of the minimum bounding rectangle of the molten pool contour.

[0009] (3) Create a blank image of the same size as the molten pool image, and mark the center point of the minimum bounding rectangle of each molten pool outline at the corresponding position in the blank image;

[0010] (4) Divide the blank image into a grid, fill the grid where the mark is located with the corresponding molten pool area value, and generate a grid matrix;

[0011] (5) Divide the grid matrix into blocks, and convert each block into a grayscale sub-image;

[0012] (6) Input the grayscale sub-image into the trained neural network model to obtain the quality classification result.

[0013] Furthermore, in step (1), a high-speed camera is used for paraxial monitoring to acquire images of the molten pool.

[0014] Furthermore, step (2) specifically includes:

[0015] (2.1) The molten pool image is sequentially processed by image grayscale conversion, bilateral filtering, binarization thresholding and opening operation to obtain the molten pool contour;

[0016] (2.2) Calculate the area of ​​the molten pool based on the molten pool profile;

[0017] (2.3) Draw the minimum bounding rectangle of the molten pool outline and calculate the center point of the minimum bounding rectangle.

[0018] Furthermore, step (4) specifically includes:

[0019] (4.1) Divide the blank image into a grid, and the number of grids after division is equivalent to the number of frames of the molten pool image belonging to the same printing layer;

[0020] (4.2) Fill the grid where the mark is located with the corresponding molten pool area value. If the same grid contains a mark, fill the grid with the average value of the molten pool area corresponding to multiple marks. If the grid does not contain a mark, fill the grid with the average value of the molten pool area of ​​the surrounding 8 grids, thereby generating a grid matrix.

[0021] Furthermore, step (5) specifically includes:

[0022] (5.1) Normalize the grid matrix, that is, map the numbers in the grid matrix to the range [0, 255]. The specific formula is as follows:

[0023]

[0024] Where X is the number in the grid of the grid matrix before normalization, X norm X is the number in the normalized grid. max X is the maximum value in the grid matrix before normalization. min It is the minimum value in the grid matrix before normalization;

[0025] (5.2) Divide the normalized grid matrix into multiple sub-grid matrices and convert each sub-grid matrix into a grayscale image.

[0026] Furthermore, the neural network model described in step (6) is an improved LeNet5 network, specifically comprising a first convolutional layer, a first batch of normalization operations, a first pooling layer, a second convolutional layer, a second batch of normalization operations, a second pooling layer, a global average pooling layer, and a softmax function connected in sequence. When training the neural network model, grayscale images obtained by processing molten pool images acquired under different process conditions are used as training samples.

[0027] The defect location method in the selective laser melting process described in this invention obtains quality classification results based on the above-mentioned quality monitoring method, and takes the location of the grayscale sub-image where the quality classification result does not meet the threshold as the location of the defect.

[0028] The quality monitoring system for the selective laser melting process described in this invention includes:

[0029] The acquisition module is used to acquire images of the molten pool in real time during the selected area laser melting process;

[0030] The melt pool image processing module is used to extract the melt pool contour from each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum bounding rectangle of the melt pool contour.

[0031] The image marking module is used to create a blank image of the same size as the molten pool image and mark the center point of the minimum bounding rectangle of each molten pool outline at the corresponding position in the blank image;

[0032] The grid matrix generation module is used to divide the blank image into grids, fill the grid where the marker is located with the corresponding molten pool area value, and generate a grid matrix.

[0033] The grayscale image generation module is used to divide the grid matrix into blocks, and each block is converted into a grayscale sub-image;

[0034] The classification module is used to input grayscale sub-images into a trained neural network model to obtain the quality classification result for each grayscale sub-image.

[0035] Furthermore, the mesh matrix generation module specifically includes:

[0036] A grid division unit is used to divide the blank image into grids, and the number of grids after division is equivalent to the number of frames of the melt pool image belonging to the same printing layer;

[0037] Numerical filling units are used to fill the grid containing the corresponding molten pool area value into the grid containing the marker. If the same grid contains a marker, the average value of the molten pool area corresponding to multiple markers is filled into the grid. If the grid does not contain a marker, the average value of the molten pool area of ​​the surrounding 8 grids is filled into the grid, thereby generating a grid matrix.

[0038] The defect location system for the selective laser melting process described in this invention includes the aforementioned quality monitoring system and a defect location module. The defect location module is used to identify the location of the defect as the position of the grayscale sub-image whose quality classification result does not meet the threshold.

[0039] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0040] 1. This invention extracts the area and position of the molten pool outline as molten pool features from the molten pool image acquired during laser melting. Based on the molten pool features, a small number of grayscale images are formed from the massive molten pool images generated during the printing process. Then, classification is performed based on the small number of grayscale images, which reduces the scale of massive data, improves data processing speed, and reduces the time for image processing and classification. This enables real-time quality monitoring during the manufacturing process.

[0041] 2. The grayscale image used for classification in this invention corresponds to the position of the molten pool in the molten pool image. Therefore, after classification based on the grayscale image, the printing quality of the corresponding area of ​​the grayscale image can be clearly known, thereby realizing the location of defects and laying the technical foundation for closed-loop feedback control of quality defects in the forming process.

[0042] 3. This invention uses an improved LeNet5 network to achieve quality classification. The improved LeNet5 network introduces batch normalization after each convolutional layer, which can improve the network's generalization ability. It also introduces global average pooling to replace fully connected layers, which can reduce parameter calculation and model training time.

[0043] 4. This invention uses deep learning methods to establish the relationship between the quality level of the printing layer and the printing process signals. It does not require the inspectors to have rich prior knowledge and avoids the subjective errors caused by manual feature selection. It improves efficiency and the accuracy of classification and recognition is also better. Attached Figure Description

[0044] Figure 1This is a flowchart illustrating the quality monitoring method for the selective laser melting process provided by the present invention.

[0045] Figure 2 This is a schematic diagram of the structure of the selective laser melting monitoring platform in a specific embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the molten pool image processing flow in a specific embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the dataset in a specific embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of metallographic characterization results and label fabrication in a specific embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Example 1

[0051] The embodiments of the present invention will be described in detail below.

[0052] This embodiment provides a method for quality monitoring of the selected area laser melting process, such as... Figure 1 As shown, it includes the following steps:

[0053] (1) Real-time acquisition of images of the molten pool during the selected area laser melting process.

[0054] For example, see Figure 2 As shown, the selective laser melting monitoring platform based on a high-speed camera adopts a paraxial monitoring method. A high-speed camera is mounted outside the cavity of the selective laser melting printer, and the acquired image of the melt pool completely covers the area to be monitored.

[0055] Taking the laser melting process using 316L stainless steel powder as the experimental material as an example, the printing speed was kept constant at 800 mm / s, and the printed sample size was a metal block of 36 mm × 12 mm × 1 mm. The frame rate of the high-speed camera was 4500 fps.

[0056] (2) Extract the melt pool contour for each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum bounding rectangle of the melt pool contour.

[0057] like Figure 3As shown, this step specifically includes:

[0058] (2.1) The molten pool image is sequentially processed by image grayscale conversion, bilateral filtering, binarization thresholding, and opening operation to obtain the molten pool contour. In specific implementation, the molten pool image can be processed based on OpenCV by sequentially calling the functions cv2.COLOR BGR2GRAY, cv2.bilateralFilter, cv2.THRESH BINARY, and cv2.morphologyEx. It is understood that the above processing can also be implemented by calculation or programming.

[0059] (2.2) Calculate the area of ​​the molten pool based on the molten pool contour; in practice, the cv2.findContours function can be called to calculate the area of ​​the molten pool, or the calculation can be implemented by calculation method, programming method, etc.

[0060] (2.3) Draw the minimum bounding rectangle of the molten pool outline and calculate the center point of the minimum bounding rectangle. In specific implementation, the minimum bounding rectangle of the molten pool can be drawn by calling cv2.boundingRect, and the center point of the rectangle can be obtained by calculating the average of the coordinates of the upper left corner and the lower right corner of the rectangle. Alternatively, the calculation can be implemented by calculation or programming.

[0061] (3) Create a blank image of the same size as the molten pool image, and mark the center point of the minimum bounding rectangle of each molten pool outline at the corresponding position in the blank image.

[0062] Continuing with the previous example, a blank image of 36mm × 12mm can be created, such as... Figure 3 As shown, the center point of the minimum bounding rectangle of the molten pool contour is marked in the blank image at the same position as the molten pool image, as follows. Figure 3 Point P is shown. In practice, the cv2.circle() function can be called to draw a mark on the blank image.

[0063] (4) Divide the blank image into grids, fill the grids where the marks are located with the corresponding molten pool area values, and generate a grid matrix.

[0064] This step specifically includes:

[0065] (4.1) Divide the blank image into a grid, with the number of grids corresponding to the number of frames of the molten pool image belonging to the same printing layer. Continuing the previous example, if a printing layer of size 36mm × 12mm is scanned at a printing speed of 800mm / s, with an average time of 6.5s, then the number of molten pool images obtainable for each printing layer is approximately 29250 frames. Therefore, each 4mm × 4mm area of ​​the blank image can be divided into 32 × 32 grids, that is, 36mm × 12mm is divided into 288 × 96 grids, for a total of 27648 grids. Figure 3 As shown;

[0066] (4.2) Fill the grid containing the marker with the corresponding molten pool area value. If the same grid contains a marker, fill in the average molten pool area of ​​multiple markers within that grid. If the grid does not contain a marker, fill in the average molten pool area of ​​the surrounding 8 grids within that grid, thus generating a grid matrix. Figure 4 As shown.

[0067] (5) Divide the grid matrix into blocks, and convert each block into a grayscale sub-image.

[0068] This step specifically includes:

[0069] (5.1) Normalize the grid matrix, that is, map the numbers in the grid matrix to the range [0, 255]. The specific formula is as follows:

[0070]

[0071] Where X is the number in the grid of the grid matrix before normalization, X norm X is the number in the normalized grid. max X is the maximum value in the grid matrix before normalization. min It is the minimum value in the grid matrix before normalization;

[0072] (5.2) Divide the normalized grid matrix into multiple sub-grid matrices, and convert each sub-grid matrix into a grayscale image. Continuing the previous example, the grid matrix can be divided into multiple 32×32 sub-grid matrices, and each sub-grid matrix can be converted into a 32×32 pixel (4mm×4mm) grayscale image, such as... Figure 4 As shown.

[0073] (6) Input the grayscale sub-image into the trained neural network model to obtain the quality classification result.

[0074] In practice, the neural network model is an improved LeNet5 network, which consists of a first convolutional layer, a first batch normalization (BN) operation, a first pooling layer, a second convolutional layer, a second batch normalization operation, a second pooling layer, a global average pooling (GAP) layer, and a softmax function, all connected sequentially. The improved LeNet5 network achieves adaptive feature extraction through two alternating convolutional and pooling layers. Batch normalization is introduced after each convolutional layer to improve the network's generalization ability. Global average pooling is introduced instead of fully connected layers to reduce parameter computation and model training time. Finally, classification is achieved through the softmax function.

[0075] The improved version of this invention has a lower computational cost compared to the traditional LeNet5 network.

[0076] When training the neural network model, grayscale images obtained by processing molten pool images acquired under different process conditions can be used as training samples. Continuing the previous example, as... Figure 5 As shown, the printing layer under different process conditions can be divided into a 9×3 printing area (4mm×4mm). Based on the porosity of the metallographic image of the printing layer, each printing area is graded in quality, for example, with three levels (good, medium, and poor). Alternatively, other quality systems with multiple levels can be established. Finally, a corresponding quality grade label is generated for the grayscale image of each printing area, which is used for neural network model training. The training method uses existing common training methods and will not be elaborated further.

[0077] This embodiment also provides a defect location method in the selected area laser melting process. The method obtains the quality classification result based on the above quality monitoring method, and takes the position of the grayscale sub-image where the quality classification result does not meet the threshold as the location of the defect.

[0078] Example 2

[0079] This embodiment provides a quality monitoring system for a selected area laser melting process. The system can be implemented using software and / or hardware, and can be configured in a terminal device, including:

[0080] The acquisition module is used to acquire images of the molten pool in real time during the selected area laser melting process;

[0081] The melt pool image processing module is used to extract the melt pool contour from each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum bounding rectangle of the melt pool contour.

[0082] The image marking module is used to create a blank image of the same size as the molten pool image and mark the center point of the minimum bounding rectangle of each molten pool outline at the corresponding position in the blank image;

[0083] The grid matrix generation module is used to divide the blank image into grids, fill the grid where the marker is located with the corresponding molten pool area value, and generate a grid matrix.

[0084] The grayscale image generation module is used to divide the grid matrix into blocks, and each block is converted into a grayscale sub-image;

[0085] The classification module is used to input grayscale sub-images into a trained neural network model to obtain the quality classification result for each grayscale sub-image.

[0086] In specific implementation, the molten pool image processing module includes:

[0087] The molten pool contour extraction unit is used to sequentially perform image grayscale conversion, bilateral filtering, binarization thresholding and opening operations on the molten pool image to obtain the molten pool contour.

[0088] Area calculation unit, used to calculate the area of ​​the molten pool based on the molten pool outline;

[0089] The center point calculation unit is used to draw the minimum bounding rectangle of the molten pool outline and calculate the center point of the minimum bounding rectangle.

[0090] In practical implementation, the mesh matrix generation module specifically includes:

[0091] A grid division unit is used to divide the blank image into grids, and the number of grids after division is equivalent to the number of frames of the melt pool image belonging to the same printing layer;

[0092] The molten pool area filling unit is used to fill the grid containing the mark with the corresponding molten pool area value. If the same grid contains a mark, the average value of the molten pool area corresponding to multiple marks is filled into the grid. If the grid does not contain a mark, the average value of the molten pool area of ​​the surrounding 8 grids is filled into the grid, thereby generating a grid matrix.

[0093] In practical implementation, the grayscale image generation module specifically includes:

[0094] The normalization unit is used to normalize the grid matrix, that is, to map the numbers in the grid matrix to the range [0, 255]. The specific formula is as follows:

[0095]

[0096] Where X is the number in the grid of the grid matrix before normalization, X norm X is the number in the normalized grid.max X is the maximum value in the grid matrix before normalization. min It is the minimum value in the grid matrix before normalization;

[0097] The grayscale conversion unit is used to divide the normalized grid matrix into multiple sub-grid matrices and convert each sub-grid matrix into a grayscale image.

[0098] In specific implementation, the neural network model is an improved LeNet5 network, specifically comprising a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a batch normalization layer, and a global average pooling layer connected in sequence. During training of the neural network model, grayscale images obtained by processing molten pool images acquired under different process conditions are used as training samples.

[0099] This embodiment also provides a defect location system for a selected area laser melting process, including the above-mentioned quality monitoring system and a defect location module. The defect location module is used to take the location of the grayscale sub-image where the quality classification result does not meet the threshold as the location of the defect.

[0100] The system provided in this embodiment of the invention can be used to execute the method provided in Embodiment 1 of the invention, and has the corresponding functions and beneficial effects of executing the method.

[0101] It is worth noting that in the embodiments of the above-mentioned determining device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0102] The embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art will clearly understand that each implementation can be achieved using software plus necessary general-purpose hardware platforms, or it can be implemented solely through hardware, as long as the function or purpose can be achieved.

[0103] Software code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

Claims

1. A method for quality monitoring in a selective laser melting process, characterized in that, Includes the following steps: (1) Real-time acquisition of images of the molten pool during the selected area laser melting process; (2) Extract the molten pool contour for each molten pool image belonging to the same printing layer, and calculate the molten pool area and the center point of the minimum bounding rectangle of the molten pool contour. (3) Create a blank image of the same size as the molten pool image, and mark the center point of the minimum bounding rectangle of each molten pool outline at the corresponding position in the blank image; (4) Divide the blank image into grids, fill the grids where the markers are located with the corresponding molten pool area values, and generate a grid matrix; (5) Divide the grid matrix into blocks, and convert each block into a grayscale sub-image; (6) Input the grayscale sub-image into the trained neural network model to obtain the quality classification result. The neural network model is an improved LeNet5 network, which specifically includes a first convolutional layer, a first batch of normalization operations, a first pooling layer, a second convolutional layer, a second batch of normalization operations, a second pooling layer, a global average pooling layer, and a softmax function connected in sequence.

2. The quality monitoring method for selective laser melting process according to claim 1, characterized in that, In step (1), a high-speed camera is used for paraxial monitoring to acquire images of the molten pool.

3. The quality monitoring method for selective laser melting process according to claim 1, characterized in that, Step (2) specifically includes: (2.1) The molten pool image is sequentially processed by image grayscale conversion, bilateral filtering, binarization thresholding and opening operation to obtain the molten pool contour; (2.2) Calculate the area of ​​the molten pool based on its outline; (2.3) Draw the minimum bounding rectangle of the molten pool outline and calculate the center point of the minimum bounding rectangle.

4. The quality monitoring method for the selective laser melting process according to claim 1, characterized in that, Step (4) specifically includes: (4.1) Divide the blank image into a grid, and the number of grids after division is equivalent to the number of frames of the molten pool image belonging to the same printing layer; (4.2) Fill the grid where the mark is located with the corresponding molten pool area value. If the same grid contains a mark, fill the grid with the average value of the molten pool area corresponding to multiple marks. If the grid does not contain a mark, fill the grid with the average value of the molten pool area of ​​the surrounding 8 grids, thereby generating a grid matrix.

5. The quality monitoring method for selective laser melting process according to claim 1, characterized in that, Step (5) specifically includes: (5.1) Normalize the grid matrix, that is, map the numbers in the grid matrix to the range [0, 255]. The specific formula is as follows: Where X is the number in the grid of the grid matrix before normalization, X norm X is the number in the normalized grid. max X is the maximum value in the grid matrix before normalization. min It is the minimum value in the grid matrix before normalization; (5.2) Divide the normalized grid matrix into multiple sub-grid matrices and convert each sub-grid matrix into a grayscale image.

6. The quality monitoring method for selective laser melting process according to claim 1, characterized in that, When training the neural network model, grayscale images obtained by processing molten pool images collected under different process conditions are used as training samples.

7. A defect localization method in a selective laser melting process, characterized in that: The quality monitoring method according to any one of claims 1-5 obtains quality classification results, and the location corresponding to the grayscale sub-image where the quality classification result does not meet the threshold is taken as the location of the defect.

8. A quality monitoring system for a selective laser melting process, characterized in that, include: The acquisition module is used to acquire images of the molten pool in real time during the selected area laser melting process; The melt pool image processing module is used to extract the melt pool contour from each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum bounding rectangle of the melt pool contour. The image marking module is used to create a blank image of the same size as the molten pool image and mark the center point of the minimum bounding rectangle of each molten pool outline at the corresponding position in the blank image; The grid matrix generation module is used to divide the blank image into grids, fill the grid where the marker is located with the corresponding molten pool area value, and generate a grid matrix. The grayscale image generation module is used to divide the grid matrix into blocks, and each block is converted into a grayscale sub-image; The classification module is used to input grayscale sub-images into a trained neural network model to obtain the quality classification result of each grayscale sub-image. The neural network model is an improved LeNet5 network, specifically including a first convolutional layer, a first batch of normalization operations, a first pooling layer, a second convolutional layer, a second batch of normalization operations, a second pooling layer, a global average pooling layer, and a softmax function connected in sequence.

9. A defect location system for a selective laser melting process, characterized in that, The system includes the quality monitoring system and defect location module as described in claim 8, wherein the defect location module is used to take the location of the grayscale sub-image where the quality classification result does not meet the threshold as the location of the defect.

Images