Visual inspection and control method and system for powder bed melting manufacturing process

Abstract

The invention discloses a method and a system for visual inspection and control of a powder bed melting manufacturing process, and relates to the technical field of visual defect identification of a powder bed melting technology.The method comprises the following steps: obtaining slices of a part patch model, extracting geometric features of each layer, and obtaining a part patch model; different areas of each layer are divided into a forming risk area, an easy-to-generate defect area and a common area; defining defect severity thresholds and defect area thresholds of different regions, and defining judgment rules of the different regions; a powder bed image in the printing process is obtained and preprocessed, and an enhanced image is output; extracting defect features and geometric features in the enhanced image, identifying defect positions, and calculating defect areas; when the defect area and / or the defect severity are / is larger than a threshold value, a corresponding feedback strategy is triggered, and the method has the advantages that through multi-level collaborative optimization, the core problems that in the prior art, the small defect detection capacity is insufficient, the complex part misjudgment rate is high, and feedback lags are comprehensively solved.

Description

Technical Field

[0001] This invention relates to the technical field of visual defect identification in metal powder bed melting technology, and particularly to a method and system for visual inspection and control in the powder bed melting manufacturing process. Background Technology

[0002] Laser powder bed fusion (L-PBF) has wide applications in aerospace, weaponry, and other fields. However, the complex physicochemical reactions between the laser and the material can induce various abnormal defects during the printing process, significantly affecting manufacturing stability. Among these, macroscopic defects are particularly harmful; deformation and fracture of parts can not only destroy the integrity of components but may also cause permanent damage to equipment.

[0003] Current research has proposed using computer vision to detect different features during the printing process and identify abnormal defects. However, current detection strategies suffer from insufficient accuracy or high false positive rates for complex structures; furthermore, their ability to detect small defects is limited, such as millimeter-level pores and cracks generated by laser powder bed melting processes, which pose a potential threat to the strength and sealing of precision or pressure-resistant components. Traditional Faster R-CNN, due to limitations in feature extraction scale and receptive field, struggles to capture subtle defect features, resulting in a false positive rate exceeding 20% ​​in actual detection. Summary of the Invention

[0004] The main objective of this invention is to propose a method and system for visual inspection and control in the powder bed melting manufacturing process, aiming to solve the technical problem of identifying subtle feature defects.

[0005] To achieve the above objectives, the present invention proposes a method for visual inspection and control of a powder bed melting manufacturing process, comprising:

[0006] Obtain slices of the component surface model, extract the geometric features of each layer, and divide the different regions of each layer into forming risk areas, defect-prone areas, and normal areas;

[0007] Define the defect severity threshold and defect area threshold for different regions, and define the judgment rules for different regions;

[0008] Acquire powder bed images during the printing process, perform preprocessing, and output enhanced images;

[0009] Extract defect features and geometric features from the enhanced image, identify defect locations, and calculate defect areas;

[0010] Determine whether the defect area and / or defect severity are greater than a threshold. If the determination is yes, trigger the corresponding feedback strategy.

[0011] In one embodiment, the different regions of each layer are divided into forming risk areas, defect-prone areas, and normal areas;

[0012] The forming risk zone includes the area adjacent to the lower surface;

[0013] The areas prone to defects include areas with large-area complex curved surfaces at small angles, areas with large-area single-sided overhangs, and areas with large internal holes.

[0014] The ordinary area refers to the remaining additive manufacturing area that has not been assigned to the forming risk area and the defect-prone area.

[0015] In one embodiment, the preprocessing includes: camera calibration, color enhancement algorithm, and guided filtering.

[0016] In one embodiment, different regions are assigned detection weights and defect detection thresholds. The detection weights are used for attention enhancement in the subsequent feature extraction stage, and the detection thresholds are used for confidence screening of defect classification.

[0017] In one embodiment, the camera calibration employs a calibration method to calculate the camera intrinsic parameter matrix and distortion coefficient vector, which are used to eliminate radial and tangential distortion of the lens; the acquired image undergoes perspective transformation and is mapped to a standard rectangle so that the frame of the corrected image coincides with the edge of the substrate;

[0018] The color enhancement algorithm uses the multi-scale Retinex band color recovery algorithm (MSRCR) to correct the uneven illumination and decompose the corrected image at multiple scales before outputting the image.

[0019] The guided filtering process uses the original image as a guide to perform guided filtering on the image, enhancing the edge contrast and details of defective areas and suppressing noise.

[0020] In one embodiment, extracting defect features and geometric features from the enhanced image specifically includes:

[0021] ResNet50 is used as the basic backbone network to extract multi-scale convolutional features. Feature maps are extracted from layers C3, C4, and C5 of ResNet50, and the feature dimensions are unified.

[0022] A Feature Pyramid Network (FPN) is constructed, which connects the feature maps of the ResNet layers from top to bottom and laterally to generate multi-scale feature representations for detecting defects of different sizes.

[0023] The extracted features are divided into a defect-sensitive branch and a geometric deformation branch. The defect branch is used to extract millimeter-level defect features, and the geometric branch is used to extract uneven powder spreading deformation features.

[0024] In the defect-sensitive branch, the Laplacian operator is applied to enhance edge details and improve the identification of small-sized defects; in the geometric deformation branch, deformable convolution and shape feature pooling are applied to identify and capture geometric deformations caused by uneven powder spreading or structural warping.

[0025] In one embodiment, identifying the defect location and calculating the defect area specifically includes:

[0026] The extracted feature maps are processed using the Region Proposal Network (RPN) of Mask R-CNN to generate a set of candidate region boxes;

[0027] The candidate region boxes are aligned using the RoIAlign layer to obtain a fixed-size feature map for subsequent defect classification, location regression, and instance segmentation.

[0028] The candidate boxes are classified by the classification head to determine the defect category and output the defect confidence C. The location of the defect is obtained by refining the coordinates of the candidate boxes by the regression head.

[0029] Based on the classification and regression results, candidate region boxes with confidence scores higher than a preset threshold are selected as defect regions; mask branching is applied to the defect regions for instance segmentation to generate binary masks.

[0030] Based on the binary mask, the pixel area of ​​each defect is calculated, and the pixel area is converted into physical area A according to the camera calibration parameters. Combined with the region weight, the defect severity index S = w × C is generated.

[0031] In one embodiment, the step of triggering a corresponding feedback strategy when the defect area and / or defect severity exceed a threshold specifically includes:

[0032] Based on the forming risk area (D1), the defect-prone area (D2), and the normal area (D3), and combined with historical defect data, simulation data, process parameters, etc., a regionally differentiated defect judgment rule is established.

[0033] For D1, D2, and D3, set dynamic defect confidence thresholds T and severity thresholds S respectively. stop and area threshold A stop ,

[0034] Based on the severity, confidence level, and area of ​​the defects in the detection results, corresponding feedback strategies are triggered, including continuing printing, continuing printing with an alarm, emergency shutdown, or termination of printing.

[0035] In one embodiment, the step of triggering a corresponding feedback strategy based on the defect severity attribute, confidence level, and area in the detection results includes continuing printing, continuing printing with an alarm, emergency shutdown, or printing termination, specifically including:

[0036] Defect severity index S = w × C; A is the defect area A; A stop For each region, a preset area threshold, S stop Set the shutdown threshold. When the defect severity index S ≥ S stop Defect area A≥A stop The corresponding decision is triggered at that time;

[0037] Execute a hierarchical response, D1 if S≥S stop or A≥A stop1 Immediately trigger shutdown; D2 If S≥S stop or A≥A stop1 Trigger an alarm (for small-area defects) or immediately shut down the machine; D3 If S≥S stop And A≥A stop1 This triggers a local rescan (for small-area defects) or an operational alarm (for large-area defects). If S≥S, D3 stop or A≥A stop1 This will trigger an alarm or continue printing.

[0038] To achieve the above objectives, this application also provides a visual inspection and control system for powder bed melting manufacturing process, including: a region division module: acquiring slices of the part surface model, extracting geometric features of each layer, and dividing different regions of each layer into forming risk areas, defect-prone areas, and ordinary areas;

[0039] Threshold management module: Defines the defect severity threshold and defect area threshold for different regions, and defines the judgment rules for different regions;

[0040] Image processing module: Acquires powder bed images during printing, performs preprocessing, and outputs enhanced images;

[0041] Feature extraction and defect determination module: Extracts defect features and geometric features from the enhanced image, identifies defect locations, and calculates defect areas;

[0042] Dynamic feedback control module: Determines whether the defect area and / or defect severity are greater than a threshold. If the determination is yes, the corresponding feedback strategy is triggered.

[0043] The technical solution of this invention achieves precise positioning of weak parts of the structure by slicing the surface model of the component into layers and extracting geometric features, and dynamically dividing different regions (forming risk area, defect-prone area and ordinary area) according to forming risk. This effectively focuses detection resources, significantly improves the defect identification accuracy of complex structural parts, and avoids the misjudgment problem caused by global uniform detection in traditional methods.

[0044] Secondly, by acquiring powder bed images in real time and preprocessing them to generate enhanced images, and combining the dual extraction of defect features and geometric features, interference such as uneven lighting and material reflection is overcome, enhancing the ability to capture minute defects such as micron-level pores and cracks, and solving the problem of missed detection caused by the limitation of feature extraction scale in traditional vision models.

[0045] Furthermore, by defining defect severity and area thresholds based on regional differences and formulating targeted judgment rules, intelligent classification and assessment of defect severity is achieved, avoiding the lag in response of a single threshold to defects in high-risk areas and ensuring the structural integrity of precision high-voltage devices.

[0046] Finally, by using dynamic threshold determination of defect area and severity to trigger real-time feedback strategies, a "detection-evaluation-control" closed loop is formed. This allows for proactive adjustment of process parameters or shutdown intervention before part deformation or fracture, preventing equipment damage, reducing material waste, and significantly improving the adaptability and stability of the manufacturing process.

[0047] This solution addresses core issues in existing technologies, such as insufficient small defect detection capabilities, high misjudgment rates for complex parts, and delayed feedback, through multi-level collaborative optimization. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating an embodiment of a method and system for visual inspection and control in a powder bed melting manufacturing process provided by the present invention;

[0050] Figure 2 This is a schematic diagram of part pretreatment in one embodiment of a method and system for visual inspection and control in a powder bed melting manufacturing process provided by the present invention;

[0051] Figure 3 A flowchart of camera calibration and guide filtering processing is provided in one embodiment of a method and system for visual inspection and control of powder bed melting manufacturing process provided by the present invention.

[0052] Figure 4 A flowchart of defect recognition based on Mask R-CNN is provided in one embodiment of a method and system for visual inspection and control in a powder bed melting manufacturing process provided by the present invention.

[0053] Figure 5A flowchart illustrating the feedback control of defects and anomalies in additive manufacturing in an embodiment of a method and system for visual inspection and control of powder bed melting manufacturing provided by the present invention.

[0054] Figure 6 This is a schematic diagram of a physical defect in an embodiment of a method and system for visual inspection and control of a powder bed melting manufacturing process provided by the present invention.

[0055] Figure 7 for Figure 6 An enlarged schematic diagram of part A in the middle.

[0056] Figure 8 for Figure 6 Enlarged diagram of part B.

[0057] Figure 9 This is a schematic diagram of warping during powder spreading in an embodiment of a method and system for visual inspection and control of powder bed melting manufacturing process provided by the present invention.

[0058] Figure 10 To Figure 9 A schematic diagram illustrating the identification of warping locations and the calculation of defect areas during powder application.

[0059] Figure 11 To Figure 10 An enlarged schematic diagram of part C.

[0060] Figure 12 This is a schematic diagram of printing defects during the printing process in one embodiment of a method and system for visual inspection and control of powder bed melting manufacturing provided by the present invention.

[0061] Figure 13 To Figure 12 A schematic diagram illustrating defect location identification and defect area calculation during printing.

[0062] Figure 14 To Figure 13 An enlarged schematic diagram of part D.

[0063] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0065] It should be noted that if directional indicators (such as up, down, left, right, front, back, etc.) are involved in the embodiments of this invention, these directional indicators are only used to explain the relative positional relationships and movement of the components in a specific posture. If the specific posture changes, the directional indicators will also change accordingly. Unless otherwise explicitly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. For those skilled in the art, the specific meaning of the above terms in this application can be understood according to the specific circumstances.

[0066] Furthermore, if the embodiments of the present invention involve descriptions using terms such as "first," "second," etc., these descriptions are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Furthermore, the use of "and / or" or "and / or" throughout the text includes three parallel options; for example, "A and / or B" includes option A, option B, or options where both A and B are satisfied simultaneously. Furthermore, the technical solutions of the various embodiments can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0067] This invention proposes a method for visual inspection and control of a powder bed melting manufacturing process, comprising:

[0068] Obtain slices of the component surface model, extract the geometric features of each layer, and divide the different regions of each layer into forming risk areas, defect-prone areas, and normal areas;

[0069] Define the defect severity threshold and defect area threshold for different regions, and define the judgment rules for different regions;

[0070] Acquire powder bed images during the printing process, perform preprocessing, and output enhanced images;

[0071] Extract defect features and geometric features from the enhanced image, identify defect locations, and calculate defect areas;

[0072] Determine whether the defect area and / or defect severity are greater than a threshold. If the determination is yes, trigger the corresponding feedback strategy.

[0073] In one embodiment, the different regions of each layer are divided into forming risk areas, defect-prone areas, and normal areas;

[0074] The forming risk zone includes the area adjacent to the lower surface;

[0075] The areas prone to defects include areas with large-area complex curved surfaces at small angles, areas with large-area single-sided overhangs, and areas with large internal holes.

[0076] The ordinary area refers to the remaining additive manufacturing area that has not been assigned to the forming risk area and the defect-prone area.

[0077] In one embodiment, the preprocessing includes: camera calibration, color enhancement algorithm, and guided filtering.

[0078] In one embodiment, different regions are assigned detection weights and defect detection thresholds. The detection weights are used for attention enhancement in the subsequent feature extraction stage, and the detection thresholds are used for confidence screening of defect classification.

[0079] In one embodiment, the camera calibration employs a calibration method to calculate the camera intrinsic parameter matrix and distortion coefficient vector, which are used to eliminate radial and tangential distortion of the lens; the acquired image undergoes perspective transformation and is mapped to a standard rectangle so that the frame of the corrected image coincides with the edge of the substrate;

[0080] The color enhancement algorithm uses the multi-scale Retinex band color recovery algorithm (MSRCR) to correct the uneven illumination and decompose the corrected image at multiple scales before outputting the image.

[0081] The guided filtering process uses the original image as a guide to perform guided filtering on the image, enhancing the edge contrast and details of defective areas and suppressing noise.

[0082] In one embodiment, extracting defect features and geometric features from the enhanced image specifically includes:

[0083] ResNet50 is used as the basic backbone network to extract multi-scale convolutional features. Feature maps are extracted from layers C3, C4, and C5 of ResNet50, and the feature dimensions are unified.

[0084] A Feature Pyramid Network (FPN) is constructed, which connects the feature maps of the ResNet layers from top to bottom and laterally to generate multi-scale feature representations for detecting defects of different sizes.

[0085] The extracted features are divided into a defect-sensitive branch and a geometric deformation branch. The defect branch is used to extract millimeter-level defect features, and the geometric branch is used to extract uneven powder spreading deformation features.

[0086] In the defect-sensitive branch, the Laplacian operator is applied to enhance edge details and improve the identification of small-sized defects; in the geometric deformation branch, deformable convolution and shape feature pooling are applied to identify and capture geometric deformations caused by uneven powder spreading or structural warping.

[0087] In one embodiment, identifying the defect location and calculating the defect area specifically includes:

[0088] The extracted feature maps are processed using the Region Proposal Network (RPN) of Mask R-CNN to generate a set of candidate region boxes;

[0089] The candidate region boxes are aligned using the RoIAlign layer to obtain a fixed-size feature map for subsequent defect classification, location regression, and instance segmentation.

[0090] The candidate boxes are classified by the classification head to determine the defect category and output the defect confidence C. The location of the defect is obtained by refining the coordinates of the candidate boxes by the regression head.

[0091] Based on the classification and regression results, candidate region boxes with confidence scores higher than a preset threshold are selected as defect regions; mask branching is applied to the defect regions for instance segmentation to generate binary masks.

[0092] Based on the binary mask, the pixel area of ​​each defect is calculated, and the pixel area is converted into physical area A according to the camera calibration parameters. Combined with the region weight, the defect severity index S = w × C is generated.

[0093] In one embodiment, the step of triggering a corresponding feedback strategy when the defect area and / or defect severity exceed a threshold specifically includes:

[0094] Based on the forming risk area (D1), the defect-prone area (D2), and the normal area (D3), and combined with historical defect data, simulation data, process parameters, etc., a regionally differentiated defect judgment rule is established.

[0095] For D1, D2, and D3, set dynamic defect confidence thresholds T and severity thresholds S respectively. stop and area threshold A stop ,

[0096] Based on the severity, confidence level, and area of ​​the defects in the detection results, corresponding feedback strategies are triggered, including continuing printing, continuing printing with an alarm, emergency shutdown, or termination of printing.

[0097] In one embodiment, the step of triggering a corresponding feedback strategy based on the defect severity attribute, confidence level, and area in the detection results includes continuing printing, continuing printing with an alarm, emergency shutdown, or printing termination, specifically including:

[0098] Defect severity index S = w × C; A is the defect area A; A stop For each region, a preset area threshold, S stop Set the shutdown threshold. When the defect severity index S ≥ S stop Defect area A≥A stop The corresponding decision is triggered at that time;

[0099] Execute a hierarchical response, D1 if S≥S stop or A≥A stop1 Immediately trigger shutdown; D2 If S≥S stop or A≥A stop1 Trigger an alarm (for small-area defects) or immediately shut down the machine; D3 If S≥S stop And A≥A stop1 This triggers a local rescan (for small-area defects) or an operational alarm (for large-area defects). If S≥S, D3 stop or A≥A stop1 This will trigger an alarm or continue printing.

[0100] To achieve the above objectives, this application also provides a visual inspection and control system for powder bed melting manufacturing process, including: a region division module: acquiring slices of the part surface model, extracting geometric features of each layer, and dividing different regions of each layer into forming risk areas, defect-prone areas, and ordinary areas;

[0101] Threshold management module: Defines the defect severity threshold and defect area threshold for different regions, and defines the judgment rules for different regions;

[0102] Image processing module: Acquires powder bed images during printing, performs preprocessing, and outputs enhanced images;

[0103] Feature extraction and defect determination module: Extracts defect features and geometric features from the enhanced image, identifies defect locations, and calculates defect areas;

[0104] Dynamic feedback control module: Determines whether the defect area and / or defect severity are greater than a threshold. If the determination is yes, the corresponding feedback strategy is triggered.

[0105] The technical solution of this application will be further described below with reference to preferred embodiments:

[0106] By layering and slicing the component surface model and extracting geometric features, and dynamically dividing different regions (forming risk area, defect-prone area and ordinary area) according to forming risk, the system achieves accurate positioning of weak parts in the structure, effectively focuses inspection resources, significantly improves the defect identification accuracy of complex structural parts, and avoids the misjudgment problem caused by global uniform detection in traditional methods.

[0107] Secondly, by acquiring powder bed images in real time and preprocessing them to generate enhanced images, and combining the dual extraction of defect features and geometric features, interference such as uneven lighting and material reflection is overcome, enhancing the ability to capture minute defects such as micron-level pores and cracks, and solving the problem of missed detection caused by the limitation of feature extraction scale in traditional vision models.

[0108] Furthermore, by defining defect severity and area thresholds based on regional differences and formulating targeted judgment rules, intelligent classification and assessment of defect severity is achieved, avoiding the lag in response of a single threshold to defects in high-risk areas and ensuring the structural integrity of precision high-voltage devices.

[0109] Finally, by using dynamic threshold determination of defect area and severity to trigger real-time feedback strategies, a "detection-evaluation-control" closed loop is formed. This allows for proactive adjustment of process parameters or shutdown intervention before part deformation or fracture, preventing equipment damage, reducing material waste, and significantly improving the adaptability and stability of the manufacturing process.

[0110] This solution addresses core issues in existing technologies, such as insufficient small defect detection capabilities, high misjudgment rates for complex parts, and delayed feedback, through multi-level collaborative optimization.

[0111] The following description, using a preferred embodiment, illustrates the content related to the above embodiments:

[0112] A method and system for visual inspection and control of powder bed melting manufacturing process, including hardware equipment and software system:

[0113] The hardware includes an industrial high-resolution camera, a computer, and signal lights. The industrial camera is installed inside the powder bed melt forming chamber to collect images of the powder bed during the additive manufacturing process. The industrial camera, which acquires image information, is connected to a computer located outside the forming chamber via wired Ethernet and gigabit fiber optic cable.

[0114] The software system includes automatic judgment of each region in the pre-imported part slice model, automatic acquisition and enhancement of powder bed images, extraction of defects and geometric features, defect detection and judgment in the processed image, defect detection feedback, and indicator light status indication. It is used to achieve online detection and control of defects and anomalies during laser powder bed fusion additive manufacturing. The indicator lights are connected to the computer via a serial port. The software system judges the severity of the anomaly, thereby generating a detection feedback signal to control whether the equipment continues the printing task and to control the indicator lights to display green or red light.

[0115] It includes the following steps:

[0116] S1 imports the component surface model into the slicing software, extracts geometric features layer by layer, and divides each layer into forming risk areas, defect-prone structural areas, and ordinary areas.

[0117] S2 acquires powder bed images and preprocesses them during the printing process, performing camera calibration, MSRCR (Multi-scale Retina Enhancement Algorithm with Color Restoration) illumination correction, and directional filtering in sequence.

[0118] S3: Extract defect features and geometric features from the image processed by S2 using an optimized backbone network;

[0119] S4: The Mask R-CNN region proposal network is used to generate candidate regions. After classification and regression operations, the defect locations are identified. Then, the defect regions are segmented by mask branch and the area is calculated.

[0120] S5: Establish regional differentiation judgment rules and set dynamic thresholds for different regions to achieve diversified feedback control strategies.

[0121] Further, step S1 includes the following steps:

[0122] S11 imports the part surface model into slicing software (such as Magics), generates a file with the actual printing layer thickness (30~100μm), extracts the geometric features of each layer simultaneously, identifies forming risk areas (large area small angle complex curved surface, large area single side overhang, large inner hole and other feature areas), marks and generates binary mask M1.

[0123] S12 imports the slice file based on the same S11 and generates a file, identifies areas prone to defects, which are defined as formable but of poor quality (such as the upper wall of the flow channel at an angle of 20°, overhangs with vertical walls on both sides, formable but with poor quality on the adjacent lower surface, etc.), marks and generates a binary mask M2.

[0124] S13 removes M1 marked in S11 and M2 marked in S12, divides the remaining additive manufacturing area into ordinary areas, marks them, and generates a binary mask M3; finally, a three-layer area division is formed: forming risk area (D1), defect-prone area (D2), and ordinary area (D3), satisfying D1∪D2∪D3=Ω and

[0125] S14 assigns differentiated detection parameters to the divided regions, with different parameters set for different part models. For example: Region D1 has a detection weight w1 = 1.3–1.5 and a defect detection threshold T1 = 0.8–0.9; Region D2 has a detection weight w2 = 1.1–1.3 and a defect detection threshold T2 = 0.7–0.8; Region D3 has a detection weight w3 = 0.9–1.1 and a defect detection threshold T3 = 0.6–0.7. The detection weights are used for attention enhancement in the subsequent feature extraction stage, and the detection thresholds are used for confidence screening in defect classification. A defect severity index S = w·C (where w is the region weight and C is the defect confidence) is defined, and differentiated shutdown thresholds are set: Region D1 S... stop1 =0.95~1.1; D2 region S stop2 =0.8~0.95; D3 area S stop3 = 0.65~0.8; when S≥S stop The system triggers a corresponding decision when S ≥ the corresponding threshold. At the same time, the decision dynamically adjusts the threshold of each region based on historical defect data and process parameters.

[0126] Further, step S2 includes the following steps:

[0127] S21 uses Zhang's calibration method, which uses at least 10 checkerboard calibration plates at different angles to calibrate the industrial camera, calculates the camera intrinsic parameter matrix K and distortion coefficient vector D, and uses them to eliminate radial and tangential distortion of the lens.

[0128] During or after additive manufacturing, S22 uses a calibrated industrial camera to capture images of the powder bed or formed parts at a resolution of no less than 1980×1080. The exposure time is dynamically adjusted to 10-50ms according to the reflective properties of the powder material.

[0129] S23 performs perspective transformation on the acquired image based on camera calibration parameters. By identifying the coordinates of the four vertices of the substrate in the image, it solves the homography matrix H using the least squares method and maps it to a standard rectangle, so that the overlap error between the frame of the corrected image and the edge of the substrate does not exceed 5 pixels.

[0130] S24 uses the Multi-Scale Retinex Color Restoration Algorithm (MSRCR) to correct the uneven illumination of the rectified image. It decomposes the image into multiple scales using a Gaussian filter bank, calculates the reflection component R and the color restoration factor C, and finally outputs the image O = β·R·C (β is the brightness constant).

[0131] S25 performs guided filtering based on the original image, setting the window radius r to 5-10 pixels and the regularization parameter ε to 0.01-0.1. It calculates the output q = a·I + b (I is the original image) through local linear coefficients a and b, thereby enhancing the edge contrast and details of defective areas and suppressing noise.

[0132] Further, step S3 includes the following steps:

[0133] S31 uses ResNet50 as the basic backbone network to extract multi-scale convolutional features. The resolution and number of channels of each ResNet50 layer are as follows: C3 layer is 56×56 and 512, used to extract millimeter-level defect edge and detail features; C4 layer is 28×28 and 1024, used to extract medium-sized defect structural features; C5 layer is 14×14 and 2048, used to extract large-sized defect abstract features; at the same time, 1×1 convolution is applied to the feature maps of C3, C4 and C5 to reduce the dimensionality to 256 channels.

[0134] S32 constructs a Feature Pyramid Network (FPN), performs a 2x nearest neighbor upsampling on the C5 feature map, merges it with the C4 feature map to generate P4, then upsamples P4 and merges it with the C3 feature map to generate P3. The C4 and C3 feature maps are then reduced to 256 channels using a 1×1 convolution, and added to the upsampled feature map. Finally, a 3×3 convolution is applied to P3, P4, and P5 to eliminate the upsampling aliasing effect. The generated multi-scale feature maps are used to detect defects of different sizes.

[0135] S33 divides the multi-scale feature maps generated by FPN into a defect-sensitive branch (P3 feature map, extracting millimeter-level pores, etc.) and a geometric deformation branch (P4 and P5 feature maps, extracting uneven powder spreading, etc.). A Laplacian operator is applied to the P3 feature map to enhance the defect edge response; deformable convolutions are applied to the P4 and P5 feature maps, and offset parameters are learned to adjust the sampling position of the convolution kernel. Simultaneously, shape feature pooling is applied to the convolved feature maps to extract shape features. The weighted and enhanced feature maps are then input into S4, providing multi-scale, high-response feature data for defect detection.

[0136] Furthermore, S4, based on the Mask R-CNN framework, achieves accurate detection and quantization from feature maps to defect instances. The core technologies are as follows:

[0137] The S41 Region Proposal Network (RPN) generates candidate region boxes containing potential defects by sliding a 3×3 convolutional window across the feature map output from S3. The RPN dynamically adjusts the anchor point size for different defect thresholds in S1: For the high-risk region D1, small anchor points (16×16, 32×32 pixels) are prioritized to capture millimeter-level defects; for the defect-prone region D2, medium-sized anchor points (64×64, 128×128 pixels) are generated to detect moderate defects such as structural warping; for ordinary regions, large anchor points (256×256 pixels) are retained to detect large-area defects. Compared to the unoptimized model, this design improves candidate box generation efficiency by 20% and reduces the false negative rate by 15%.

[0138] To address the complexity of defect boundaries in powder bed images (such as edge blurring caused by metallic powder reflection), S42 employs RoIAlign instead of the traditional RoIPoolin. Candidate regions are divided into 7×7 or 14×14 grids, with the grid size dynamically selected based on the defect scale. Non-integer quantization sampling is performed on the four corner points of each grid (e.g., coordinates (x+0.25, y+0.25)) to avoid feature misalignment. Then, pooling is performed on the sampled feature values ​​of each grid to generate a fixed-size feature map.

[0139] The S43 classification head uses two fully connected layers (1024 dimensions) to extract features, outputs the defect category probability and defect confidence C through softmax, and filters out defect regions with confidence ≥ T based on the region detection threshold T set in S14. The regression head predicts the bounding box offsets (Δx, Δy, Δw, Δh) through linear layers to refine the candidate boxes generated by RPN: center point coordinate correction: x′=x+Δx×w, y′=y+Δy×h; width and height correction: w′=w×e Δw h′=h×e Δh ).

[0140] The S44 mask branch comprises three convolutional layers (3×3, 256 channels, padding 1) and two deconvolutional layers (upsampling factor 2, stride 2). All three convolutional layers expand the receptive field through dilated convolutions to capture defect contextual information (such as the region surrounding the defect). The two transposed convolutional layers progressively restore the feature map size to the input image resolution. The output feature map from the transposed convolution is compressed to a single channel by a 1×1 convolution, and a mask prediction map with a 0-1 probability distribution is generated using a sigmoid activation function, enhancing the distinction between defects and low-contrast backgrounds. Finally, morphological opening (kernel size 3×3) is applied to the mask to eliminate isolated noise and ensure connectivity in the defect region.

[0141] Based on the S45 and S2 camera calibration results (e.g., an industrial camera with a resolution of 1920×1080, corresponding to a physical field of view of 200mm×112.5mm), the physical area of ​​a single pixel is S. pixel=0.0117mm 2 After binarization and masking, the number of defective pixels N is counted, and the defect area A = N × S pixel Defects in high-risk areas are assigned a weight of 1.5 times, areas prone to defects are assigned a weight of 1.3 times, and ordinary areas are assigned a weight of 1 time, generating a severity index S = w × C, which is used for subsequent classification decisions.

[0142] Further, step S5 includes the following steps:

[0143] S51 receives the defect coordinate information output from the S4 defect detection module, and simultaneously obtains the region mask (M1-M3) generated in the S1 region division step. By matching the defect coordinates with the region mask, it determines the region (D1 / D2 / D3) to which the defect belongs, and outputs the defect region label L∈{D1,D2,D3}.

[0144] S52, based on the region label L obtained in S51, calls the threshold table pre-configured in S1 to obtain the detection threshold T for the corresponding region. L Severity threshold S stopL Area threshold A stopL Where T1>T2>T3, A stop1 <A stop2 <A stop3 S stop1 >S stop2 >S stop3 This allows the shutdown triggering conditions in high-risk areas to be simultaneously adjusted by both weight amplification and threshold raising, avoiding frequent false shutdowns; based on the detection weights allocated by S1 (different weights for different printed parts, e.g., D1:w1 = 1.3~1.5, D2:w2 = 1.1~1.3, D3:w3 = 0.9~1.1), combined with the defect severity index S = w·C (C is the defect confidence level),

[0145] S53 receives the defect confidence level C, area A, and severity index S output by S4, and calculates the defect triggering condition F(L) by combining it with the dynamic threshold parameter obtained from S52. For regions D1 and D2, the triggering condition is F(L) = [S ≥ S stop ]∨[A≥A stop For region D3, the trigger condition is F(L) = [S ≥ S]. stop ]∧[A≥A stop The output is a boolean value F, indicating whether a response is triggered.

[0146] Based on the trigger condition calculated by S53, S54 executes different response strategies: when trigger condition F is false, the system continues normal printing; if the defect belongs to area D1 and trigger condition F is true, the system immediately performs a shutdown operation and generates a corresponding defect report, including information such as defect location, type, and severity index; if the defect belongs to area D2 and trigger condition F is true, the defect area is further determined, and if A < 2.0 mm... 2 When A ≥ 2.0 mm, the system issues an alarm and automatically adjusts the scanning strategy, such as increasing the number of scans or adjusting scanning parameters; 2 When the defect occurs, a shutdown operation is performed and the area is marked; if the defect belongs to area D3 and trigger condition F is true, the defect area A is further determined. If A < 2.0 mm 2 Continue printing and marking the defect; when it reaches 2.0mm 2 ≤A<5.0mm 2 When the abscissa is ≥ 5.0 mm, the system issues an alarm, continues printing, and triggers operations such as toner respreading; 2 The system will then perform a shutdown operation.

[0147] The following section provides a definition of terms used in this application:

[0148] L-PBF: L-PBF (Laser Powder Bed Fusion) is a core technology in metal additive manufacturing (3D printing). Its core principle is to selectively melt layers of metal powder using a high-energy laser beam, stacking them layer by layer to form a dense, three-dimensional solid part.

[0149] Faster R-CNN: Faster R-CNN (Faster Region-based Convolutional Neural Network) is a landmark algorithm in the field of object detection. Its core is to achieve efficient end-to-end object detection through a Region Proposal Network (RPN), solving the speed bottleneck of traditional methods that rely on external region generation algorithms (such as Selective Search).

[0150] STL Model: STL stands for Stereolithography, which is the cornerstone file format for 3D printing. It defines the geometric surface using triangular meshes.

[0151] MSRCR: MSRCR stands for Multi-Scale Retinex with Color Restoration.

[0152] 1) Multi-Scale: Extracts details at different levels of an image using multiple Gaussian scales (usually 3, with scale parameters σ = 15, 80, 200).

[0153] 2) Retinex Theory: Based on Land's visual model, it posits that the color perceived by the human eye is determined by the light reflected from an object, and is independent of illumination (color constancy).

[0154] 3) Color Restoration: Introducing a color restoration factor (C) to correct color distortion caused by multi-scale enhancement, resulting in a more natural result.

[0155] 4) Through the synergy of multi-scale decomposition and color restoration factors, a balance between detail enhancement and color fidelity is achieved in complex lighting scenarios (medical, agricultural, underwater).

[0156] Mask R-CNN: Mask R-CNN stands for Mask Region-based Convolutional Neural Network. It eliminates quantization errors through RoIAlign and achieves pixel-level segmentation through parallel mask branches, enabling high-precision target contour extraction in fields such as medicine, industry, and remote sensing. Its core value lies in unifying the tasks of object detection and segmentation, propelling computer vision from "framework definition" to "shape understanding." In the future, combining attention mechanisms, lightweight design, and multimodal data, Mask R-CNN will continue to expand its application boundaries in real-time edge computing and complex scenarios.

[0157] O: Represents the final image matrix output by the algorithm, where each pixel value O(x,y) is constrained within the range [0,255], conforming to the standard 8-bit unsigned integer (uint8) image format requirements.

[0158] ResNet50: ResNet50 stands for ResidualNetwork 50-layer. It uses residual learning as its core and achieves stable training and high-performance feature extraction of deep networks through skip connections and bottleneck block design.

[0159] Feature Pyramid Network (FPN): FPN aims to address the challenge of multi-scale object detection in computer vision tasks. Its core idea is to construct a pyramid structure that combines high-resolution details and strong semantic information by fusing feature maps from different levels, thereby improving the model's ability to detect objects of different sizes (especially small objects).

[0160] Region Proposal Network (RPN): This network generates multi-scale candidate boxes through an anchoring mechanism and combines bi-branch prediction to achieve efficient target region selection, solving the problems of slow speed and poor compatibility of traditional methods. Its end-to-end design has driven innovation in object detection technology and has become a core component of two-stage detection models (such as Faster R-CNN and Mask R-CNN).

[0161] RoIPooling (Region of Interest Pooling) quantizes candidate regions (RoIs) of arbitrary size and pools them into fixed-size feature maps (e.g., 7×7) for object detection classification and regression.

[0162] RoI Align (Region of Interest Align) achieves sub-pixel alignment between candidate regions and feature maps by eliminating quantization errors and using bilinear interpolation, thus improving the accuracy of small targets and edges.

[0163] Softmax: The normalized exponential function, which maps a real vector to a probability distribution through exponential normalization.

[0164] Sigmoid: The sigmoid function (also known as the logistic function or normalized exponential function) is a non-linear activation function. Its core function is to map any real number to the (0,1) interval, generating a probabilistic output. In defect detection, it significantly improves the distinction between defect areas and low-contrast backgrounds by generating a 0-1 probability mask.

[0165] Perspective Transformation: This method utilizes the geometric condition that the perspective center (viewpoint), image point (projection point), and target point are collinear. It involves rotating the projection plane to disrupt the original projection ray beam while maintaining the original projection geometry.

[0166] It should be understood that the terms "one embodiment" or "one example" throughout the specification mean that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in one example" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to the invention.

[0167] In various embodiments of the present invention, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It is particularly important to note that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0169] The above description is merely an exemplary embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the technical concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for visual inspection and control in a powder bed melting manufacturing process, characterized in that, include: Obtain slices of the component surface model, extract the geometric features of each layer, and divide the different regions of each layer into forming risk areas, defect-prone areas, and normal areas; Define the defect severity threshold and defect area threshold for different regions, and define the judgment rules for different regions; Acquire powder bed images during the printing process, perform preprocessing, and output enhanced images; Extract defect features and geometric features from the enhanced image, identify defect locations, and calculate defect areas; Determine whether the defect area and / or defect severity are greater than a threshold. If the determination is yes, trigger the corresponding feedback strategy.

2. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 1, characterized in that: Each layer is divided into three areas: forming risk zone, defect-prone zone, and normal zone. The forming risk zone includes the area adjacent to the lower surface; The areas prone to defects include areas with large-area complex curved surfaces at small angles, areas with large-area single-sided overhangs, and areas with large internal holes. The ordinary area refers to the remaining additive manufacturing area that has not been assigned to the forming risk area and the defect-prone area.

3. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 1, characterized in that, The preprocessing includes: camera calibration, color enhancement algorithm, and guided filtering.

4. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 2, characterized in that: Different regions are assigned detection weights and defect detection thresholds. The detection weights are used for attention enhancement in the subsequent feature extraction stage, and the detection thresholds are used for confidence screening of defect classification.

5. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 3, characterized in that: The camera calibration adopts a calibration method to calculate the camera intrinsic parameter matrix and distortion coefficient vector, which are used to eliminate radial and tangential distortion of the lens; the acquired image is subjected to perspective transformation and mapped to a standard rectangle so that the frame of the corrected image coincides with the edge of the substrate. The color enhancement algorithm uses the multi-scale Retinex band color recovery algorithm (MSRCR) to correct the uneven illumination and decompose the corrected image at multiple scales before outputting the image. The guided filtering process uses the original image as a guide to perform guided filtering on the image, enhancing the edge contrast and details of defective areas and suppressing noise.

6. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 1, characterized in that: The extraction of defect features and geometric features from the enhanced image specifically includes: ResNet50 is used as the basic backbone network to extract multi-scale convolutional features. Feature maps are extracted from layers C3, C4, and C5 of ResNet50, and the feature dimensions are unified. A Feature Pyramid Network (FPN) is constructed, which connects the feature maps of the ResNet layers from top to bottom and laterally to generate multi-scale feature representations for detecting defects of different sizes. The extracted features are divided into a defect-sensitive branch and a geometric deformation branch. The defect branch is used to extract millimeter-level defect features, and the geometric branch is used to extract uneven powder spreading deformation features. In the defect-sensitive branch, the Laplacian operator is applied to enhance edge details and improve the identification of small-sized defects; in the geometric deformation branch, deformable convolution and shape feature pooling are applied to identify and capture geometric deformations caused by uneven powder spreading or structural warping.

7. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 1, characterized in that: The process of identifying the defect location and calculating the defect area specifically includes: The extracted feature maps are processed using the Region Proposal Network (RPN) of Mask R-CNN to generate a set of candidate region boxes; The candidate region boxes are aligned using the RoIAlign layer to obtain a fixed-size feature map for subsequent defect classification, location regression, and instance segmentation. The candidate boxes are classified by the classification head to determine the defect category and output the defect confidence C. The location of the defect is obtained by refining the coordinates of the candidate boxes by the regression head. Based on the classification and regression results, candidate region boxes with confidence scores higher than a preset threshold are selected as defect regions; mask branching is applied to the defect regions for instance segmentation to generate binary masks. Based on the binary mask, the pixel area of ​​each defect is calculated, and the pixel area is converted into physical area A according to the camera calibration parameters. Combined with the region weight, the defect severity index S = w × C is generated.

8. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 1, characterized in that: When the defect area and / or defect severity exceed a threshold, a corresponding feedback strategy is triggered, specifically including: Based on the forming risk area (D1), the defect-prone area (D2), and the normal area (D3), and combined with historical defect data, simulation data, process parameters, etc., a regionally differentiated defect judgment rule is established. For D1, D2, and D3, set dynamic defect confidence thresholds T and severity thresholds S respectively. stop and area threshold A stop , Based on the severity, confidence level, and area of ​​the defects in the detection results, corresponding feedback strategies are triggered, including continuing printing, continuing printing with an alarm, emergency shutdown, or termination of printing.

9. The method for visual inspection and control of a powder bed melting manufacturing process as described in claim 8, characterized in that, The system triggers corresponding feedback strategies based on the defect severity attribute, confidence level, and area in the detection results. These strategies include continuing printing, continuing printing with an alarm, emergency shutdown, or printing termination. Specifically, these include: Defect severity index S = w × C; A is the defect area A; A stop For each region, a preset area threshold, S stop Set the shutdown threshold. When the defect severity index S ≥ S stop Defect area A≥A stop The corresponding decision is triggered at that time; Execute a hierarchical response, D1 if S≥S stop or A≥A stop1 Immediately trigger shutdown; D2 If S≥S stop or A≥A stop1 Trigger an alarm (for small-area defects) or immediately shut down the machine; D3 If S≥S stop And A≥A stop1 This triggers a local rescan (for small-area defects) or an operational alarm (for large-area defects). If S≥S, D3 stop or A≥A stop1 This will trigger an alarm or continue printing.

10. A system for visual inspection and control of a powder bed melting manufacturing process, characterized in that, include: Region division module: Obtain slices of the component surface model, extract the geometric features of each layer, and divide the different regions of each layer into forming risk areas, defect-prone areas, and normal areas; Threshold management module: Defines the defect severity threshold and defect area threshold for different regions, and defines the judgment rules for different regions; Image processing module: Acquires powder bed images during printing, performs preprocessing, and outputs enhanced images; Feature extraction and defect determination module: Extracts defect features and geometric features from the enhanced image, identifies defect locations, and calculates defect areas; Dynamic feedback control module: Determines whether the defect area and / or defect severity are greater than a threshold. If the determination is yes, the corresponding feedback strategy is triggered.

Images