Quality monitoring method and system for selective laser melting process, and defect locating method and system for same

Abstract

Disclosed in the present invention are a quality monitoring method and system for a selective laser melting process, and a defect locating method and system for same. The quality monitoring method in the present invention comprises: collecting melt pool images in a selective laser melting process in real time; then performing melt pool contour extraction on the melt pool images that belong to the same printing layer, and calculating melt pool areas and center points of melt-pool-contour minimum bounding rectangles; creating a blank image having the same size as the melt pool images, and marking, at a corresponding position in the blank image, the center point of each melt-pool-contour minimum bounding rectangle; gridding the blank image, and filling grid cells, in which marks are located, with values of the melt pool areas of corresponding melt pools, so as to generate a grid matrix; dividing the grid matrix into blocks, and converting each block into a grayscale sub-image; and inputting the grayscale sub-images into a trained neural network model, so as to obtain a quality classification result.

Description

Quality monitoring and defect location method and system for selective laser melting process Technical Field

[0001] The present invention relates to the field of laser printing technology, and in particular to a quality monitoring method and system for a selective laser melting process. Background Art

[0002] Additive manufacturing technology, with its flexible, customized production and digital manufacturing models, is constantly transforming traditional production methods. Powder bed laser selective melting (LPBF) is considered the most promising mainstream technology, as it can utilize a wide range of metal powder materials and produce lightweight metal structures made of complex and composite materials without the use of binders. It is widely used in aerospace, military, medical device manufacturing, vehicle manufacturing, and general metal parts manufacturing.

[0003] Currently, LPBF technology is still immature in terms of manufacturing processes, equipment control, and nondestructive testing. Defects can occur in manufactured metal workpieces, significantly impacting the strength, surface roughness, and other aspects of the metal structure, reducing the yield rate. Research to improve yield rates focuses on three key areas: 1) real-time monitoring of the workpiece during machining; 2) real-time monitoring and feedback control of the melt pool morphology during machining; and 3) automated, real-time monitoring of powder spreading quality. Of these, real-time monitoring and feedback control of the melt pool morphology has garnered the most attention. This approach utilizes high-speed cameras to monitor the metal powder additive manufacturing process online, process images, extract feature information, analyze results, and provide feedback control over the process. This approach effectively improves machining quality, enabling print quality monitoring and defect location during the selective laser melting process, laying the technical foundation for closed-loop feedback control of quality defects during the forming process.

[0004] Publication number CN 113077423 A discloses a convolutional neural network-based melt pool image analysis system for selective laser melting. This system is capable of classifying and identifying melt pool images, capturing the outline, size, quantity, and dynamic behavior of the melt pool and spatter, and evaluating and analyzing melt pool images during the selective laser melting process. However, this method directly uses the melt pool images for neural network classification. Selective laser melting prints at high speeds, and the resulting melt pool images are often massive. This system cannot perform complex processing on a large number of melt pool images while simultaneously ensuring real-time monitoring. Furthermore, defect analysis of the melt pool images cannot accurately locate defects.

[0005] Summary of the Invention

[0006] Purpose of the invention: To address the problems existing in the prior art, the present invention provides a quality monitoring method and system for the selective laser melting process.

[0007] Technical solution: The quality monitoring method of the selective laser melting process described in the present invention comprises the following steps:

[0008] (1) Real-time acquisition of molten pool images during selective laser melting;

[0009] (2) Extract the melt pool contours of each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum enclosing rectangle of the melt pool contour;

[0010] (3) creating a blank image of the same size as the melt pool image, and marking the center point of the minimum enclosing rectangle of each melt pool outline at the corresponding position in the blank image;

[0011] (4) dividing the blank image into grids, and filling the grids where the marks are located with the values ​​of the molten pool area corresponding to the molten pool to generate a grid matrix;

[0012] (5) Dividing the grid matrix into blocks, and converting each block into a grayscale sub-image;

[0013] (6) Input the grayscale sub-image into the trained neural network model to obtain the quality classification result.

[0014] Furthermore, in step (1), a high-speed camera is used for off-axis monitoring to realize the acquisition of the molten pool image.

[0015] Furthermore, step (2) specifically includes:

[0016] (2.1) The melt pool image is subjected to image grayscale conversion, bilateral filtering, binarization threshold processing, and opening operation processing in sequence to obtain the melt pool contour;

[0017] (2.2) Calculate the molten pool area based on the molten pool contour;

[0018] (2.3) Draw the minimum enclosing rectangle of the molten pool outline and calculate the center point of the minimum enclosing rectangle.

[0019] Furthermore, step (4) specifically includes:

[0020] (4.1) dividing the blank image into grids, wherein the number of grids after division is equal to the number of frames of the melt pool image belonging to the same printing layer;

[0021] (4.2) Fill the grid where the mark is located with the melt pool area value of the corresponding melt pool. If the same grid contains a mark, fill the grid with the average value of the melt pool areas corresponding to multiple marks. If the grid does not contain a mark, fill the grid with the average value of the melt pool areas of the surrounding 8 grids, thereby generating a grid matrix.

[0022] Furthermore, step (5) specifically includes:

[0023] (5.1) Normalize the grid matrix, that is, map the numbers in the grid matrix to [0, 255]. The specific formula is:

[0024] Where X is the number of grid cells in the grid matrix before normalization, X norm is the number in the normalized grid, X max is the maximum value in the grid matrix before normalization, X min is the minimum value in the grid matrix before normalization;

[0025] (5.2) The normalized grid matrix is ​​divided into multiple sub-grid matrices, and each sub-grid matrix is ​​converted into a grayscale image.

[0026] Furthermore, the neural network model in step (6) is an improved LeNet5 network, specifically including a first convolution layer, a first batch of normalization operations, a first pooling layer, a second convolution layer, a second batch of normalization operations, a second pooling layer, a global average pooling layer, and a softmax function connected in sequence. When training the neural network model, grayscale images obtained by processing the molten pool images collected under different process conditions are used as training samples.

[0027] The defect location method for the selective laser melting process described in the present invention obtains a quality classification result according to the above-mentioned quality monitoring method, and uses the position corresponding to the grayscale sub-image whose quality classification result does not meet the threshold as the defect location.

[0028] The quality monitoring system for the selective laser melting process of the present invention comprises:

[0029] An acquisition module is used to acquire the molten pool image in real time during the selective laser melting process;

[0030] The melt pool image processing module is used to extract the melt pool contours of each melt pool image belonging to the same printing layer, calculate the melt pool area and the center point of the minimum enclosing rectangle of the melt pool contour;

[0031] An image marking module is used to create a blank image of the same size as the melt pool image, and mark the center point of the minimum enclosing rectangle of each melt pool outline at a corresponding position in the blank image;

[0032] A grid matrix generation module is used to divide the blank image into grids, fill the grid where the mark is located with the molten pool area value of the corresponding molten pool, and generate a grid matrix;

[0033] A grayscale image generation module, configured to divide the grid matrix into blocks and convert each block into a grayscale sub-image;

[0034] The classification module is used to input the grayscale sub-image into the trained neural network model to obtain the quality classification result of each grayscale sub-image.

[0035] Furthermore, the grid matrix generation module specifically includes:

[0036] A grid division unit, configured to divide the blank image into grids, wherein the number of grids after division is equivalent to the number of frames of the melt pool image belonging to the same printing layer;

[0037] The numerical filling unit is used to fill the grid where the mark is located with the melt pool area value of the corresponding melt pool. If the same grid contains a mark, the average value of the melt pool areas corresponding to multiple marks is filled in the grid. If the grid does not contain a mark, the average value of the melt pool areas of the surrounding 8 grids is filled in the grid to generate a grid matrix.

[0038] The defect localization system for the selective laser melting process of the present invention comprises the above-mentioned quality monitoring system and a defect localization module, wherein the defect localization module is configured to use the position corresponding to the grayscale sub-image whose quality classification result does not meet the threshold as the defect location.

[0039] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0040] 1. The present invention extracts the area and position of the melt pool contour from the melt pool images collected during the laser melting process as melt pool features. Based on the melt pool features, the massive melt pool images generated during the printing process are converted into a small number of grayscale images. Classification is then performed based on these small number of grayscale images. This reduces the scale of massive data, improves data processing speed, and reduces the time for image processing and classification, thus enabling real-time quality monitoring during the manufacturing process.

[0041] 2. The grayscale image used for classification in the present invention corresponds to the molten pool position in the molten pool image. Therefore, after classification based on the grayscale image, the print quality of the corresponding area of ​​the grayscale image can be clearly determined, thereby locating defects and laying a technical foundation for closed-loop feedback control of quality defects in the forming process.

[0042] 3. The present invention uses an improved LeNet5 network to implement quality classification. The improved LeNet5 network introduces batch normalization operation after each convolutional layer to improve the network generalization ability, and introduces global average pooling instead of the fully connected layer to reduce parameter calculation and model training time.

[0043] 4. The present invention uses deep learning methods to establish the connection between the quality level of the printed layer and the printing process signal. It does not require the inspection personnel to have rich prior knowledge and avoids the subjective errors caused by manual feature selection. While improving efficiency, the accuracy of classification and recognition is also better. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] FIG1 is a schematic flow chart of a quality monitoring method for a selective laser melting process provided by the present invention;

[0045] FIG2 is a schematic structural diagram of a selective laser melting monitoring platform in a specific embodiment of the present invention;

[0046] FIG3 is a schematic diagram of a melt pool image processing process according to a specific embodiment of the present invention;

[0047] FIG4 is a schematic diagram of a data set in a specific embodiment of the present invention;

[0048] FIG5 is a schematic diagram of metallographic characterization results and label production in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] Example 1

[0051] The embodiments of the present invention are described in detail below.

[0052] This embodiment provides a method for monitoring the quality of a selective laser melting process, as shown in FIG1 , including the following steps:

[0053] (1) Real-time acquisition of molten pool images during selective laser melting.

[0054] For example, as shown in FIG2 , the selective laser melting monitoring platform based on a high-speed camera adopts paraxial monitoring. A high-speed camera is set up outside the cavity of the selective laser melting printer, and the collected molten pool image completely includes the area to be monitored.

[0055] For example, the laser melting process of 316L stainless steel powder was used as the experimental material. The printing speed was set to a constant 800 mm / s, and the printed sample size was 36 mm × 12 mm × 1 mm. The frame rate of the high-speed camera was 4500 fps.

[0056] (2) Extract the melt pool contours of each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum enclosing rectangle of the melt pool contour.

[0057] As shown in Figure 3, this step specifically includes:

[0058] (2.1) The melt pool image is subjected to image grayscale conversion, bilateral filtering, binarization threshold processing, and opening operation processing in sequence to obtain the melt pool contour; in a specific implementation, the melt pool image can be processed based on OpenCV, and the above processing is implemented by calling cv2.COLOR_BGR2GRAY, cv2.bilateralFilter, cv2.THRESH_BINARY, and cv2.morphologyEx functions in sequence. It can be understood that the above processing can also be implemented by calculation, programming, etc.;

[0059] (2.2) Calculate the area of ​​the molten pool according to the molten pool contour; in specific implementation, the cv2.findContours function can be called to calculate the area of ​​the molten pool, or the calculation method or programming method can be used to implement the calculation;

[0060] (2.3) Draw the minimum enclosing rectangle of the molten pool outline and calculate the center point of the minimum enclosing rectangle; in specific implementation, the minimum enclosing rectangle of the molten pool can be drawn by calling cv2.boundingRect, and the center point of the rectangle can be obtained by calculating the average value of the coordinates of the upper left corner and the lower right corner of the rectangle. The calculation can also be implemented by calculation, programming, etc.

[0061] (3) Create a blank image of the same size as the melt pool image, and mark the center point of the minimum enclosing rectangle of each melt pool outline at the corresponding position in the blank image.

[0062] Continuing with the previous example, we can create a 36mm×12mm blank image, as shown in Figure 3. The center point of the minimum enclosing rectangle of the melt pool outline is marked in the blank image at the same position as the melt pool image, as shown by point P in Figure 3. In practice, we can call the cv2.circle() function to draw a mark on the blank image.

[0063] (4) Divide the blank image into grids, fill the grids where the marks are located with the values ​​of the molten pool area of ​​the corresponding molten pool, and generate a grid matrix.

[0064] This step specifically includes:

[0065] (4.1) The blank image is divided into grids, and the number of grids after division is equivalent to the number of frames of the melt pool image belonging to the same print layer. Continuing with the above example, if a print layer of size 36 mm × 12 mm is scanned at a print speed of 800 mm / s, and the average scanning time is 6.5 seconds, the number of melt pool images that can be obtained for each print layer is approximately 29,250 frames. Therefore, each 4 mm × 4 mm area of ​​the blank image can be divided into 32 × 32 grids, that is, 36 mm × 12 mm is divided into 288 × 96 grids, for a total of 27,648 grids, as shown in Figure 3.

[0066] (4.2) Fill the grid where the mark is located with the melt pool area value of the corresponding melt pool. If the same grid contains a mark, fill the grid with the average value of the melt pool areas corresponding to multiple marks. If the grid does not contain a mark, fill the grid with the average value of the melt pool areas of the surrounding 8 grids, thereby generating a grid matrix, as shown in Figure 4.

[0067] (5) Divide the grid matrix into blocks, and convert each block into a grayscale sub-image.

[0068] This step specifically includes:

[0069] (5.1) Normalize the grid matrix, that is, map the numbers in the grid matrix to [0, 255]. The specific formula is:

[0070] Where X is the number of grid cells in the grid matrix before normalization, X norm is the number in the normalized grid, X max is the maximum value in the grid matrix before normalization, X min is the minimum value in the grid matrix before normalization;

[0071] (5.2) Divide the normalized grid matrix into multiple sub-grid matrices and convert each sub-grid matrix into a grayscale image. Continuing with the previous example, the grid matrix can be divided into multiple 32×32 sub-grid matrices and each sub-grid matrix can be converted into a 32×32 pixel (4mm×4mm) grayscale image, as shown in Figure 4.

[0072] (6) Input the grayscale sub-image into the trained neural network model to obtain the quality classification result.

[0073] In specific implementation, the neural network model is a modified LeNet5 network, specifically consisting of a first convolutional layer, a batch normalization operation (BN), a first pooling layer, a second convolutional layer, a second batch normalization operation, a second pooling layer, a global average pooling (GAP) layer, and a softmax function. The modified LeNet5 network achieves adaptive feature extraction through two alternating convolutional and pooling layers. Batch normalization is introduced after each convolutional layer to improve network generalization capabilities. Global average pooling is introduced instead of fully connected layers to reduce parameter calculation and model training time. Finally, classification is achieved through the softmax function.

[0074] The improved present invention has lower computational complexity than the traditional LeNet5 network.

[0075] When training the neural network model, the grayscale images obtained by processing the molten pool images collected under different process conditions can be used as training samples. Continuing with the previous example, as shown in FIG5 , the printed layer under different process conditions can be divided into 9×3 printing areas (4mm×4mm), and the quality of each printed area is graded according to the porosity of the metallographic image of the printed layer. For example, three levels (good quality, medium quality, and poor quality) can be set. It is understandable that other multiple-level quality systems can also be divided. Finally, corresponding quality grade labels are formed for the grayscale images corresponding to each printed area for use in neural network model training. The training method adopts the existing commonly used training method and will not be repeated here.

[0076] This embodiment also provides a defect location method for a selective laser melting process, which obtains a quality classification result according to the above-mentioned quality monitoring method, and uses the position corresponding to the grayscale sub-image whose quality classification result does not meet the threshold as the defect location.

[0077] Example 2

[0078] This embodiment provides a quality monitoring system for a selective laser melting process. The system can be implemented using software and / or hardware. The system can be configured in a terminal device and includes:

[0079] An acquisition module is used to acquire the molten pool image in real time during the selective laser melting process;

[0080] The melt pool image processing module is used to extract the melt pool contours of each melt pool image belonging to the same printing layer, calculate the melt pool area and the center point of the minimum enclosing rectangle of the melt pool contour;

[0081] An image marking module is used to create a blank image of the same size as the melt pool image, and mark the center point of the minimum enclosing rectangle of each melt pool outline at a corresponding position in the blank image;

[0082] A grid matrix generation module is used to divide the blank image into grids, fill the grid where the mark is located with the molten pool area value of the corresponding molten pool, and generate a grid matrix;

[0083] A grayscale image generation module, configured to divide the grid matrix into blocks and convert each block into a grayscale sub-image;

[0084] The classification module is used to input the grayscale sub-image into the trained neural network model to obtain the quality classification result of each grayscale sub-image.

[0085] In a specific implementation, the molten pool image processing module includes:

[0086] The molten pool contour extraction unit is used to perform image grayscale, bilateral filtering, binarization threshold processing and opening operation processing on the molten pool image in sequence to obtain the molten pool contour;

[0087] An area calculation unit, used to calculate the molten pool area based on the molten pool contour;

[0088] The center point calculation unit is used to draw the minimum enclosing rectangle of the molten pool outline and calculate the center point of the minimum enclosing rectangle.

[0089] In specific implementation, the grid matrix generation module specifically includes:

[0090] A grid division unit, configured to divide the blank image into grids, wherein the number of grids after division is equivalent to the number of frames of the melt pool image belonging to the same printing layer;

[0091] The melt pool area filling unit is used to fill the grid where the mark is located with the melt pool area value of the corresponding melt pool. If the same grid contains a mark, the average value of the melt pool areas corresponding to multiple marks is filled in the grid. If the grid does not contain a mark, the average value of the melt pool areas of the surrounding 8 grids is filled in the grid to generate a grid matrix.

[0092] In specific implementation, the grayscale image generation module specifically includes:

[0093] The normalization unit is used to normalize the grid matrix, that is, to map the numbers in the grid matrix to between [0, 255]. The specific formula is:

[0094] Where X is the number of grid cells in the grid matrix before normalization, X norm is the number in the normalized grid, X max is the maximum value in the grid matrix before normalization, X min is the minimum value in the grid matrix before normalization;

[0095] The grayscale conversion unit is used to divide the normalized grid matrix into multiple sub-grid matrices and convert each sub-grid matrix into a grayscale image.

[0096] In a specific implementation, the neural network model is a modified LeNet5 network, specifically comprising a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a batch normalization layer, and a global average pooling layer connected in sequence. When training the neural network model, grayscale images obtained by processing melt pool images collected under different process conditions are used as training samples.

[0097] This embodiment also provides a defect localization system for a selective laser melting process, comprising the above-mentioned quality monitoring system and a defect localization module, wherein the defect localization module is configured to use the position corresponding to the grayscale sub-image whose quality classification result does not meet the threshold as the defect location.

[0098] The system provided in the embodiment of the present invention can be used to execute the method provided in the first embodiment of the present invention, and has the corresponding functions and beneficial effects of executing the method.

[0099] It is worth noting that in the embodiment of the above-mentioned determination device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0100] The embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art will readily appreciate that each embodiment may be implemented using software plus a necessary general-purpose hardware platform, or may be implemented solely through hardware, as long as the functionality or effect can be achieved.

[0101] Software code for carrying out the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages.

Claims

1. A quality monitoring method for a selective laser melting process, characterized in that: The following steps are involved: (1) Real-time acquisition of molten pool images during selective laser melting; (2) Extract the melt pool contours of each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum enclosing rectangle of the melt pool contour; (3) creating a blank image of the same size as the molten pool image, and marking the center point of the minimum enclosing rectangle of each molten pool outline at the corresponding position in the blank image; (4) dividing the blank image into grids, filling the grids where the marks are located with the molten pool area values ​​of the corresponding molten pools, and generating a grid matrix; (5) dividing the grid matrix into blocks, and converting each block into a grayscale sub-image; (6) Input the grayscale sub-image into the trained neural network model to obtain the quality classification result.

2. The quality monitoring method of the selective laser melting process according to claim 1, characterized in that: In step (1), a high-speed camera is used for side-axis monitoring to realize molten pool image acquisition.

3. The quality monitoring method of the selective laser melting process according to claim 1, characterized in that: Step (2) specifically includes: (2.1) The molten pool image is subjected to image grayscale, bilateral filtering, binarization threshold processing and opening operation processing in sequence to obtain the molten pool contour; (2.2) Calculate the molten pool area based on the molten pool contour; (2.3) Draw the minimum enclosing rectangle of the molten pool outline and calculate the center point of the minimum enclosing rectangle.

4. The quality monitoring method of the selective laser melting process according to claim 1, characterized in that: Step (4) specifically includes: (4.1) dividing the blank image into grids, wherein the number of grids after division is equal to the number of frames of the molten pool image belonging to the same printing layer; (4.2) Fill the grid where the mark is located with the molten pool area value of the corresponding molten pool. If the same grid contains a mark, fill the grid with the average value of the molten pool areas corresponding to multiple marks. If the grid does not contain a mark, fill the grid with the average value of the molten pool areas of the surrounding 8 grids, thereby generating a grid matrix.

5. The quality monitoring method of the selective laser melting process according to claim 1, characterized in that: Step (5) specifically includes: (5.1) The grid matrix is ​​normalized, that is, the numbers in the grid matrix are mapped to between [0, 255]. The specific formula is: Where X is the number of cells in the grid matrix before normalization, X norm is the number of normalized grid cells, X max is the maximum value in the grid matrix before normalization, X min is the minimum value in the grid matrix before normalization; (5.2) The normalized grid matrix is ​​divided into multiple sub-grid matrices, and each sub-grid matrix is ​​converted into a grayscale image.

6. The quality monitoring method of the selective laser melting process according to claim 1, characterized in that: The neural network model described in step (6) is an improved LeNet5 network, which specifically includes a first convolutional layer, a first batch of normalization operations, a first pooling layer, a second convolutional layer, a second batch of normalization operations, a second pooling layer, a global average pooling layer, and a softmax function connected in sequence.

7. The quality monitoring method of the selective laser melting process according to claim 1, characterized in that: When training the neural network model, the grayscale images obtained by processing the molten pool images collected under different process conditions are used as training samples.

8. A defect location method in a selective laser melting process, characterized in that: The quality classification result is obtained according to the quality monitoring method according to any one of claims 1 to 6, and the position corresponding to the grayscale sub-image whose quality classification result does not meet the threshold is used as the defect location.

9. A quality monitoring system for a selective laser melting process, characterized in that: include: An acquisition module is used to acquire the molten pool image in real time during the selective laser melting process; The melt pool image processing module is used to extract the melt pool contour of each melt pool image belonging to the same printing layer, and calculate the melt pool area and the center point of the minimum enclosing rectangle of the melt pool contour; An image marking module is used to create a blank image of the same size as the molten pool image, and mark the center point of the minimum enclosing rectangle of each molten pool outline at a corresponding position in the blank image; A grid matrix generation module, used for dividing the blank image into grids, filling the grid where the mark is located with the molten pool area value of the corresponding molten pool, and generating a grid matrix; A grayscale image generation module, used for dividing the grid matrix into blocks, and converting each block into a grayscale sub-image; The classification module is used to input the grayscale sub-image into the trained neural network model to obtain the quality classification result of each grayscale sub-image.

10. A defect location system for a selective laser melting process, characterized in that: It comprises the quality monitoring system and defect location module as described in claim 9, wherein the defect location module is used to take the position corresponding to the gray sub-image whose quality classification result does not meet the threshold as the defect location.

Images