Method and system for quality monitoring and defect location in selective area laser melting processes

Abstract

The present invention discloses a quality monitoring and defect location method and system for the laser melting process of selected areas. The present invention collects images of the molten pool in real time during the laser melting process of selected areas, then extracts the molten pool profile for each molten pool image belonging to the same printing layer, calculates the area of ​​the molten pool and the center point of the smallest bounding rectangle of the molten pool profile, creates a blank image of the same size as the molten pool image, and marks the positions on the blank image corresponding to the center points of the smallest bounding rectangle of each molten pool profile. The blank image is meshed, and the corresponding molten pool area values ​​are entered into the cells where the marks are located to generate a cell matrix. The cell matrix is ​​divided into blocks, and each block is converted into a grayscale sub-image. The grayscale sub-image is input into a trained neural network model to obtain a quality classification result.

Description

[Technical Field]

[0001] The present invention relates to the field of laser printing technology, and more particularly to a method and system for monitoring the quality of a selective area laser fusing process. [Background technology]

[0002] Additive manufacturing technology is constantly changing traditional production methods due to its flexible, customized production and digitalized manufacturing model. Powder bed laser selective area melting (LPBF) is considered the most promising mainstream technology because it can use a wide range of metal powder materials to manufacture lightweight metal structural components with complex structures without adhesives, and is widely used in the fields of aerospace, military, medical equipment manufacturing, vehicles, and general metal parts manufacturing.

[0003] Currently, LPBF technology is not yet fully mature in terms of manufacturing process, equipment control, and non-destructive testing, which can result in defects in the manufactured metal parts. This significantly impacts the strength, surface finish, and other quality aspects of metal structural parts, reducing the yield rate. Research into improving yield rates mainly focuses on three aspects: 1) real-time monitoring of the part during processing, 2) real-time monitoring and feedback control of the morphological features of the molten pool during processing, and 3) automated real-time monitoring of powder deposition quality. Real-time monitoring and feedback control of the morphological features of the molten pool has received the most attention. Using high-speed cameras to monitor the metal powder additive manufacturing process online in real time, real-time processing of images, extracting feature information, and analyzing the results for feedback control of the processing process can effectively improve processing quality. This can enable monitoring of the printing quality and defect location during the laser-selected melting process, laying the technological foundation for closed-loop feedback control of quality defects during the forming process.

[0004] Patent publication number CN113077423A discloses a convolutional neural network-based image analysis system for weld pools in laser selective area melting, which can classify and identify weld pool images, capture the size, number, and dynamic behavior changes of weld pools and sputtering profiles, and evaluate and analyze weld pool images during the laser selective area melting process. However, this method directly uses neural network classification on weld pool images, while the printing speed of selective area laser melting is fast and the number of weld pool images obtained is usually enormous. Therefore, the system cannot monitor and process the large number of weld pool images in real time, and defect analysis of weld pool images cannot realize defect location. Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE INVENTION In view of the problems existing in the prior art, the present invention aims to provide a method and system for monitoring the quality of a selective area laser melting process. [Means for solving the problem]

[0006] The technical solution is as follows: The quality monitoring method for the selective area laser melting process described in the present invention is as follows: (1) acquiring real-time images of the molten pool during the laser melting process of the selected area; (2) extracting a profile of the molten pool from the image of each molten pool belonging to the same printing layer, and calculating the area of ​​the molten pool and the center point of the minimum bounding rectangle of the molten pool profile; (3) creating a blank image having the same dimensions as the image of the molten pool, and marking a position on the blank image corresponding to the center point of the minimum bounding rectangle of each molten pool profile; (4) meshing the blank image and filling the corresponding molten pool area values ​​into the cells where the marks are located to generate a cell matrix; (5) dividing the cell matrix into blocks and converting each block into a grayscale sub-image; (6) inputting the grayscale sub-images into the trained neural network model to obtain quality classification results.

[0007] Furthermore, in step (1), an image of the molten pool is acquired by monitoring the rangefinder of the high-speed camera.

[0008] Furthermore, step (2) specifically includes: (2.1) grayscaling the image of the molten pool, bilateral filtering, binarization thresholding, and opening processing are sequentially performed to obtain a profile of the molten pool; (2.2) calculating the area of ​​the molten pool based on the molten pool profile; (2.3) Drawing a minimum enclosing rectangle of the weld pool profile and calculating the center point of the minimum enclosing rectangle.

[0009] Furthermore, step (4) specifically includes: (4.1) The blank image is meshed, and the number of cells after meshing is equivalent to the number of frames of the image of the molten pool belonging to the same printing layer; (4.2) Entering the corresponding weld pool area value into the cell where the mark is located, and if the same cell contains marks, entering the average value of the weld pool areas corresponding to multiple marks into the cell, and if the cell does not contain marks, entering the average value of the weld pool areas of the surrounding eight cells into the cell, thereby generating a cell matrix.

[0010] Furthermore, step (5) specifically includes: (5.1) Normalize the cell matrix, that is, map the values ​​in the cell matrix between [0, 255], and the specific formula is as follows:

number

[0011] Furthermore, in step (6), the neural network model is an improved LeNet5 network, specifically including a first convolutional layer, a first batch normalization operation, a first pooling layer, a second convolutional layer, a second batch normalization operation, a second pooling layer, a global average pooling layer, and a softmax function, which are connected in sequence. When training the neural network model, grayscale maps obtained by processing images of molten pools taken under different process conditions are used as training samples.

[0012] The defect location method for the selective area laser melting process described in the present invention obtains the quality classification result by the above-mentioned quality monitoring method, and determines the defect location as the location corresponding to the grayscale sub-image whose quality classification result does not meet the threshold.

[0013] The quality monitoring system for the selective area laser melting process described in the present invention comprises: an acquisition module for acquiring images of the molten pool in real time during the laser melting process of the selected area; a molten pool image processing module for extracting a molten pool profile from an image of each molten pool belonging to the same printing layer and calculating the area of ​​the molten pool and the center point of the minimum bounding rectangle of the molten pool profile; an image marking module for creating a blank image having the same dimensions as the image of the molten pool and marking a corresponding position on the blank image of a center point of a minimum enclosing rectangle of each molten pool profile; a cell matrix generating module for meshing the blank image, inputting the cell where the mark is located into the corresponding molten pool area value, and generating a cell matrix; a grayscale image generation module for dividing the cell matrix into blocks and converting each block into a grayscale sub-image; and a classification module for inputting the grayscale sub-images into the trained neural network model to obtain a classification result of the quality of each grayscale sub-image.

[0014] Furthermore, the cell matrix generation module specifically includes: a meshing unit for meshing the blank image, the number of cells after meshing being equal to the number of frames of the image of the molten pool belonging to the same printing layer; and a value entry unit for entering the corresponding weld pool area value into the cell where the mark is located, and if the same cell contains a mark, entering the average value of the weld pool areas corresponding to multiple markings into the cell, and if the cell does not contain a mark, entering the average value of the weld pool areas of the surrounding eight cells into the cell, thereby generating a cell matrix.

[0015] The defect location system for the selective area laser melting process described in the present invention includes the above-mentioned quality monitoring system and a defect location module, which is used to determine the defect location as the location corresponding to the grayscale sub-image whose quality classification result does not meet the threshold. [Effects of the Invention]

[0016] The present invention has the following significant advantages over the prior art:

[0017] 1. The present invention extracts the area and position of the molten pool profile from the image of the molten pool collected during the laser melting process to identify the molten pool characteristics. Based on the molten pool characteristics, the image of the massive molten pool generated during the printing process is converted into a small number of grayscale images. Classification is then performed based on the small number of grayscale images, thereby reducing the size of the massive data, improving the data processing speed, and reducing the time required for image processing and classification, thereby enabling real-time quality monitoring during the manufacturing process.

[0018] 2. In the present invention, the grayscale map used for classification corresponds to the molten pool position in the molten pool image. Therefore, after classification based on the grayscale map, the printing quality of the position of the area corresponding to the grayscale map can be clearly grasped, which realizes defect location and lays a technical foundation for closed-loop feedback control of quality defects in the forming process.

[0019] 3. The present invention adopts an improved LeNet5 network to achieve quality classification. The improved LeNet5 network introduces a batch normalization operation after each convolutional layer to improve the generalization ability of the network. Global average pooling is introduced to replace the global connection layer, reducing the amount of parameter calculation and the model training time.

[0020] 4. The present invention uses deep learning methods to create a connection between the printing layer quality level and the printing process signals, eliminating the need for extensive prior knowledge on the part of the inspector. This avoids subjective errors caused by manually selecting features, improving efficiency while also improving classification accuracy. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a flow diagram of a method for quality monitoring of a selective area laser melting process according to the present invention; [Figure 2] 1 is a structural schematic diagram of a selective area laser melting monitoring platform according to a specific embodiment of the present invention; [Figure 3]1 is a schematic diagram of an image processing flow of a molten pool in a specific embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram of a data set according to a specific embodiment of the present invention. [Figure 5] 1 is a schematic diagram of a metal structure representation result and label creation in a specific embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0022] The following clearly and completely describes the technical solutions in the embodiments of the present invention, in conjunction with the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only some of the embodiments of the present invention, not all of the embodiments, and any other embodiments that can be obtained by those skilled in the art based on the embodiments of the present invention without any creative efforts are all within the scope of protection of the present invention. [Example]

[0023] Examples of the present invention will be described in detail below.

[0024] This embodiment provides a quality monitoring method for the selective area laser melting process, which includes the following steps, as shown in FIG. (1) Real-time images of the molten pool during the laser melting process of the selected area are acquired. For example, as shown in Figure 2, the selected area laser melting monitoring platform based on high-speed cameras adopts rangefinder monitoring, where one high-speed camera is installed outside the chamber of the selected area laser melting printing machine, and the captured image of the molten pool completely covers the area to be monitored. The selective laser melting process was performed using 316L stainless steel powder as an example. The printing speed was set to 800 mm / s, and the printing sample was a metal block with dimensions of 36 mm x 12 mm x 1 mm. The frame rate of the high-speed camera was 4500 fps. (2) The profile of the molten pool is extracted from the image of each molten pool belonging to the same printing layer, and the area of ​​the molten pool and the center point of the minimum bounding rectangle of the molten pool profile are calculated.

[0025] As shown in FIG. 3, the steps specifically include: (2.1) To obtain a profile of a molten pool by sequentially performing grayscaling, bilateral filtering, binarization thresholding, and opening on the image of the molten pool. In particular, the image of the molten pool may be processed based on OpenCV, and the above processes may be realized by sequentially calling the functions cv2.COLOR_BGR2GRAY, cv2.bilateralFilter, cv2.THRESH_BINARY, and cv2.morphologyEx. As can be understood, the above processes may also be realized by adopting a calculation method, a programming method, etc. (2.2) Calculating the area of ​​the molten pool based on the profile of the molten pool. In particular, the area of ​​the molten pool can be calculated by calling the cv2.findContours function, or by using a calculation method, programming method, etc.; (2.3) Drawing the minimum bounding rectangle of the weld pool profile and calculating the center point of the minimum bounding rectangle. In a specific implementation, the minimum bounding rectangle of the weld pool can be drawn by calling cv2.boundingRect, and the center point of the rectangle can be obtained by calculating the average coordinates of the upper left corner and the lower right corner of the rectangle. The calculation can also be achieved by a calculation method, a programming method, etc.

[0026] (3) Create a blank image of the same dimensions as the molten pool image, and mark the position on the blank image that corresponds to the center point of the smallest bounding rectangle of each molten pool profile.

[0027] Following the above example, a blank image of 36 mm × 12 mm may be created, and a mark may be placed on the blank image at the center point of the smallest bounding rectangle of the weld pool profile, as shown in Figure 3, at the same position as the weld pool image, for example, point P in Figure 3. In specific implementation, a mark may be drawn on the blank image by calling the cv2.circle() function.

[0028] (4) The blank image is meshed, and the area value of the corresponding molten pool is input into the cell where the mark is located to generate a cell matrix.

[0029] Specifically, this step includes: (4.1) The blank image is meshed, and the number of cells after meshing is equivalent to the number of frames of the molten pool image belonging to the same printing layer. Following the above example, if a printing layer with a dimension of 36mm x 12mm is scanned at a printing speed of 800mm / s and the average usage time is 6.5s, the number of molten pool images obtained for each printing layer is approximately 29,250 frames. Each 4mm x 4mm area of ​​the blank image can be divided into 32 x 32 cells, that is, 36mm x 12mm can be divided into 288 x 96 cells, for a total of 27,648 cells (as shown in Figure 3). (4.2) Enter the corresponding weld pool area value into the cell where the mark is located, and if the same cell contains marks, enter the average value of the weld pool areas corresponding to multiple marks into the cell; if the cell does not contain marks, enter the average value of the weld pool areas of the surrounding eight cells into the cell, thereby generating a cell matrix (shown in Figure 4).

[0030] (5) Block the cell matrix and convert each block into one grayscale sub-image. Specifically, this step includes: (5.1) Normalize the cell matrix, that is, map the values ​​in the cell matrix between [0, 255], and the specific formula is as follows:

number

[0031] (6) The grayscale subimages are input into the trained neural network model to obtain quality classification results.

[0032] In a specific implementation, the neural network model is an improved LeNet5 network, specifically including a first convolutional layer, a first batch normalization (BN) operation, a first pooling layer, a second convolutional layer, a second batch normalization, a second pooling layer, a global average pooling (GAP) layer, and a softmax function, which are connected in sequence. The improved LeNet5 network achieves adaptive feature extraction using two reciprocal convolutional layers and pooling layers, and introduces a batch normalization operation after each convolutional layer to improve the generalization ability of the network. Global average pooling is introduced to replace the global connection layer, reducing the amount of parameter calculation and model training time. Finally, classification is achieved using a softmax function.

[0033] When improved by the present invention, the computational complexity is lower compared to the conventional LeNet5 network.

[0034] When training the neural network model, grayscale maps obtained by processing images of the molten pool taken under different process conditions can be used as training samples. Following the example above, as shown in Figure 5, the printed layers under different process conditions can be divided into 9x3 print areas (4mm x 4mm). Each print area can be graded based on the size of the porosity in the metallographic image of the printed layer, for example, into three ranks (good quality, medium quality, poor quality). Other multiple-rank quality systems can also be used for easy understanding. Finally, a corresponding quality rank label is generated for each grayscale map corresponding to each print area, which is used to train the neural network model. The training method is a commonly used conventional training method and will not be further described here.

[0035] This embodiment further provides a defect location method for a selective area laser melting process, which obtains a quality classification result by the above-mentioned quality monitoring method, and determines the location of the defect as the location corresponding to the grayscale sub-image whose quality classification result does not meet the threshold. [Example]

[0036] This embodiment provides a quality monitoring system for the laser melting process of the selected area, which may be realized by adopting a software and / or hardware method, and the system may be located in a terminal device, and the system: an acquisition module for acquiring images of the molten pool in real time during the laser melting process of the selected area; a molten pool image processing module for extracting a molten pool profile from an image of each molten pool belonging to the same printing layer and calculating the area of ​​the molten pool and the center point of the minimum bounding rectangle of the molten pool profile; an image marking module for creating a blank image having the same dimensions as the image of the molten pool and marking a position on the blank image corresponding to a center point of a minimum enclosing rectangle of each molten pool profile; a cell matrix generating module for meshing the blank image, inputting the corresponding molten pool area values ​​into the cells where the marks are located, and generating a cell matrix; a grayscale image generation module for dividing the cell matrix into blocks and converting each block into a grayscale sub-image; and a classification module for inputting the grayscale sub-images into the trained neural network model to obtain a classification result of the quality of each grayscale sub-image.

[0037] In a specific implementation, the molten pool image processing module: a molten pool profile extraction unit for sequentially performing grayscaling, bilateral filtering, binarization thresholding, and opening on the molten pool image to obtain a molten pool profile; an area calculation unit for calculating an area of ​​the molten pool based on the profile of the molten pool; and a center point calculation unit for drawing a minimum enclosing rectangle of the profile of the molten pool and calculating a center point of the minimum enclosing rectangle.

[0038] In a specific implementation, the cell matrix generation module specifically includes: a meshing unit for meshing the blank image, the number of cells after meshing being equal to the number of frames of the image of the molten pool belonging to the same printing layer; and a weld pool area entry unit for entering the corresponding weld pool area value in the cell where the mark is located, and if the same cell contains marks, entering the average value of the weld pool areas corresponding to multiple marks in the cell, and if the cell does not contain a mark, entering the average value of the weld pool areas of the surrounding eight cells in the cell, thereby generating a cell matrix.

[0039] In a specific implementation, the grayscale image generation module specifically includes: A normalization unit for performing normalization on the cell matrix, i.e., mapping the numerical values ​​in the cell matrix to a range between [0, 255], the specific formula of which is as follows:

number

[0040] In a specific implementation, the neural network model is an improved LeNet5 network, specifically including a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a batch normalization layer, and a global average pooling layer, which are connected in sequence. When training the neural network model, grayscale maps obtained by processing images of the molten pool taken under different process conditions are used as training samples.

[0041] This embodiment further provides a defect location system for a selective area laser melting process, which includes the above-mentioned quality monitoring system and a defect location module, and the defect location module is used to determine the location of the defect as the location corresponding to the grayscale sub-image whose quality classification result does not meet the threshold.

[0042] The system according to the embodiment of the present invention may be used to perform the method according to the first embodiment of the present invention, and has corresponding functions and beneficial effects for performing the method.

[0043] In the above embodiment of the determination device, each unit and module included is simply classified according to functional theory, and is not limited to the above classification, as long as it can realize the corresponding function, and the specific names of each functional unit are merely for the purpose of making it easier to distinguish them from each other, and are not intended to limit the scope of protection of the present invention.

[0044] The above-described embodiments are merely illustrative, and modules described as separate components may or may not be physically separated, and components represented as modules may or may not be physical modules, and may be co-located or distributed across multiple network modules. Depending on actual needs, some or all of the modules may be selected to achieve the objectives of the solutions of the present embodiments. As will be apparent to those skilled in the art, each embodiment may be realized by combining software with a required general-purpose hardware platform, or may be realized solely by hardware, as long as the function or action can be achieved.

[0045] Software code for carrying out the operations of the present invention may be written in one or more programming languages ​​or combinations thereof, including objective programming languages ​​such as Java, Smalltalk, C++, and further including conventional procedural programming languages ​​such as "C" or similar programming languages.

Claims

1. (1) acquiring real-time images of a molten pool during laser melting of a selected area; (2) extracting a profile of the molten pool from the image of each molten pool belonging to the same printing layer, and calculating the area of ​​the molten pool and the center point of the minimum bounding rectangle of the molten pool profile; (3) creating a blank image having the same dimensions as the image of the molten pool, and marking a position on each blank image corresponding to the center point of the smallest enclosing rectangle of the profile of the molten pool; (4) meshing the blank image and filling the corresponding molten pool area values ​​into the cells where the marks are located to generate a cell matrix; (5) dividing the cell matrix into blocks and converting each block into a grayscale sub-image; (6) inputting the grayscale sub-images into a trained neural network model to obtain quality classification results.

2. 2. The method for monitoring the quality of a selected area laser melting process as claimed in claim 1, wherein in step (1), the image of the molten pool is acquired by monitoring the rangefinder of a high-speed camera.

3. Specifically, step (2) is (2.1) grayscaling the image of the molten pool, bilateral filtering, binarization thresholding, and opening processing are sequentially performed to obtain a profile of the molten pool; (2.2) calculating the area of ​​the molten pool based on the molten pool profile; 2.3) drawing a minimum enclosing rectangle of the profile of the molten pool and calculating the center point of the minimum enclosing rectangle.

4. Specifically, step (4) is (4.1) The blank image is meshed, and the number of cells after meshing is equivalent to the number of frames of the image of the molten pool belonging to the same printing layer; (4.2) The quality monitoring method for the laser melting process of the selected area as claimed in claim 1, characterized in that it also includes: entering the corresponding numerical value of the area of ​​the molten pool into the cell where the mark is located; if the same cell contains a mark, entering the average value of the areas of the molten pool corresponding to multiple marks into the cell; if the cell does not contain a mark, entering the average value of the areas of the molten pool of the surrounding eight cells into the cell, thereby generating a cell matrix.

5. Specifically, step (5) is (5.1) Normalize the cell matrix, that is, map the values ​​in the cell matrix between [0, 255], and the specific formula is as follows: [Equation 4] where X is the numerical value in the cell in the cell matrix before normalization, and X ｎｏｒｍ is the value in the cell after normalization, and X ｍａｘ is the maximum value in the cell matrix before normalization, and X ｍｉｎ is the minimum value in the cell matrix before normalization, and (5.2) Dividing the normalized cell matrix into a plurality of sub-cell matrices and converting each sub-cell matrix into a single grayscale image.

6. 2. The quality monitoring method for the laser melting process of the selected area according to claim 1, wherein in step (6), the neural network model is an improved LeNet5 network, specifically including, in sequence, a first convolutional layer, a first batch normalization operation, a first pooling layer, a second convolutional layer, a second batch normalization operation, a second pooling layer, a global average pooling layer, and a softmax function.

7. 2. The method for monitoring the quality of a selected area during laser melting as claimed in claim 1, wherein when training the neural network model, grayscale maps obtained by processing images of the molten pool taken under different process conditions are used as training samples.

8. A method for locating defects in a laser melting process of a selected area, characterized in that a quality classification result is obtained by the quality monitoring method described in any one of claims 1 to 6, and the position corresponding to the grayscale sub-image where the quality classification result does not satisfy a threshold is determined to be the defect position.

9. an acquisition module for acquiring images of the molten pool in real time during the laser melting process of the selected area; a molten pool image processing module for extracting a molten pool profile from an image of each molten pool belonging to the same printing layer and calculating the area of ​​the molten pool and the center point of the minimum bounding rectangle of the molten pool profile; an image marking module for creating a blank image having the same dimensions as the image of the molten pool and marking a position on the blank image corresponding to a center point of a minimum enclosing rectangle of each molten pool profile; a cell matrix generating module for meshing the blank image, inputting the corresponding molten pool area values ​​into the cells where the marks are located, and generating a cell matrix; a grayscale image generation module for dividing the cell matrix into blocks and converting each block into a grayscale sub-image; a classification module for inputting the grayscale sub-images into a trained neural network model to obtain a classification result of the quality of each grayscale sub-image.

10. 10. A system for locating defects in a selected area using a laser melting process, comprising the quality monitoring system of claim 9 and a defect location module, wherein the defect location module is used to determine the location of the defect as corresponding to a grayscale sub-image whose quality classification result does not satisfy a threshold.

Images