Rocket wallboard weld defect high-precision detection method based on visual guidance and geometric deambiguity

Abstract

The invention discloses a visual guidance and geometric deambiguity-based high-precision detection method for a weld defect of a rocket wallboard. The method comprises the following steps: acquiring a 2D image and a 3D point cloud of a weld of the rocket wallboard; a 2D global reconnaissance network is constructed, and a composite loss function is adopted for training; deploying a trained network to perform reasoning on the 2D image, and generating a guidance signal through connected domain analysis and noise filtering; based on the guidance signals and 3D point cloud quantitative analysis, macroscopic geometric indexes of the whole weld joint and microscopic geometric feature descriptors of the suspicious defect areas are analyzed; performing macroscopic-microcosmic hybrid decision on the defects, and outputting a weld defect decision list; and packaging the judgment list to obtain a structured digital quality archive report, and outputting the structured digital quality archive report to a downstream quality tracing system in a JSON format. While optical artifacts and geometric noise interference are processed, high efficiency and high robustness are kept, key defects can be captured through a structured report, and the welding seam quality can be accurately monitored.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and quality inspection technology, and in particular to a high-precision detection method for weld defects in rocket wall panels based on vision guidance and geometric deambiguation. Background Technology

[0002] In the field of modern aerospace manufacturing, rocket panels, as critical load-bearing structures, directly affect the safety and reliability of missions through the quality of their weld assembly. Traditional quality inspection methods are inefficient, prone to fatigue, highly subjective, and lack digital traceability capabilities. They have been gradually replaced by advanced non-destructive testing technologies based on machine vision. Deep learning-based automated inspection provides an efficient means for monitoring and controlling the quality of rocket panel welds.

[0003] Traditional automated inspection methods suffer from insufficient accuracy and susceptibility to interference when deployed under stringent aerospace manufacturing standards. They are unable to achieve high-precision quantitative judgment under complex weld morphology and variable imaging conditions (such as optical artifacts caused by metal reflection and difficulty in imaging micro-cracks).

[0004] In existing technologies, detection methods based on deep learning, such as 2D image-based schemes, emphasize high efficiency and color difference recognition, but have limitations such as insensitivity to geometric shapes and susceptibility to optical artifacts, and cannot meet the requirements of high-precision quantization. Schemes based on 3D point clouds have limited model capacity and lack contextual information, resulting in insufficient robustness and accuracy when dealing with geometric noise, which easily leads to false detections and false negatives.

[0005] To resolve this contradiction, existing technologies attempt to fuse 2D images and 3D point clouds. However, this knowledge transfer is usually limited to simple stitching at the data level. When the measurement results of 2D vision and 3D geometry conflict, the system lacks a robust adjudication mechanism to truly solve the root cause problem of "ambiguity" in a single modality. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention proposes a high-precision detection method for rocket wall panel weld defects based on visual guidance and geometric deambiguation. By introducing a serial analysis strategy that combines 2D global reconnaissance guidance with 3D local quantization, and utilizing its characteristics of using 3D geometric features as the primary key and performing geometric deambiguation on 2D visual results, this method aims to capture high-precision defect quantification results of aerospace-grade rocket wall panels when facing 2D optical artifacts and 3D geometric noise interference, so as to accurately detect the weld quality status.

[0007] To achieve the above-mentioned technical objectives, the present invention provides the following technical solution: A high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation, specifically including: By jointly calibrating and synchronously acquiring 2D color images and 3D point clouds of rocket wall panel welds, a precise mapping relationship from pixels to space is established, and a weld defect dataset is constructed. Construct a 2D global reconnaissance network that takes 2D color images as input and outputs pixel-level category predictions. A composite loss function is used for training to obtain a 2D global reconnaissance network with high robustness in defect detection. A pre-trained 2D global reconnaissance network is deployed to infer from real-time acquired 2D color images, obtain pixel-level category predictions, and then perform connected component analysis and noise filtering to generate guidance signals that provide guidance and basis for subsequent defect detection; the guidance signals include a global mask for welds and a list of suspected defects. The weld body point cloud and the base material point clouds on both sides of the weld body are extracted from the 3D point cloud based on the global mask of the weld. A zero-height reference plane is fitted based on the base material point clouds on both sides, and the overall macroscopic geometric indicators of the weld are quantitatively analyzed. Extract the local 3D point cloud of each suspected defect region in the suspected defect list, extract the key cross section from the local 3D point cloud using the maximum feature strategy, and quantize a set of micro-geometric feature descriptors of the suspected defect regions based on the key cross section and the zero-height reference plane. Based on the overall macroscopic geometric indicators of the weld and the microscopic geometric feature descriptors of the suspected defect areas, the weld defects are first macroscopically adjudicated, and then the 2D reconnaissance results are microscopically deambigued according to the AWS D17.1 Class A standard, and finally a weld defect adjudication list is output. Based on the weld defect adjudication list, a structured digital quality archive report is encapsulated and output to the downstream quality traceability system in JSON format.

[0008] Based on the above technical solution, the present invention has at least the following technical effects: The method proposed in this invention introduces a serial analysis strategy that combines 2D global reconnaissance guidance with 3D local quantization. This efficiently integrates the high recall capability of 2D reconnaissance networks with the high-precision geometric quantization capability of 3D sensing, enabling the final deployed detection system to deterministically handle interference from 2D optical artifacts and 3D geometric noise while maintaining high efficiency and robustness. This addresses the core contradiction between data misjudgment caused by optical ambiguity or geometric limitations and the requirement for high-precision adjudication in practical applications of single-modal detection methods. From a novel perspective different from existing technologies, it captures high-precision defect quantification results conforming to the AWS D17.1 Class A standard in environments with interference from 2D optical artifacts and 3D geometric noise, providing highly robust and traceable sensing data and support for downstream quality traceability systems and digital quality archives. Attached Figure Description

[0009] Figure 1 This is an overall flowchart of a high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation proposed in this invention. Figure 2 This is a schematic diagram of the system architecture of the method proposed in this invention. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1-2 The present invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0011] Although the steps in this invention are arranged by reference numerals, this is not intended to limit the order of the steps. Unless the order of the steps is explicitly stated or the execution of a step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items.

[0012] This invention proposes a high-precision detection method for rocket panel weld defects based on vision guidance and geometric deambiguation. Based on this method, this invention proposes a serial analysis strategy based on 2D visual global reconnaissance and 3D geometric local deambiguation, constructing a system such as... Figure 2 The guidance system, as shown, incorporates a highly robust 2D reconnaissance network, enabling high-precision quantification of the 3D analysis step under the guidance of a list of suspected defects. This strategy integrates the combined capabilities of high-recall 2D reconnaissance and high-precision quantification in 3D through a fusion adjudication engine, implicitly adjudicating 2D results with 3D geometric features as the primary key. This significantly improves the model's robustness to optical artifacts and geometric noise; reduces reliance on perfect imaging of a single modality; and integrates reconnaissance and quantification tasks, allowing a single system to simultaneously retain anti-interference capabilities and high-precision detail adjudication capabilities, thus enhancing the application robustness of the detection method. It provides a new perspective for rocket panel weld inspection, delivering robust detection results even in environments with interference from 2D optical artifacts and 3D geometric noise. It can capture critical defects through structured reports, enabling automated systems to accurately monitor weld quality.

[0013] like Figure 1-2 As shown, the method proposed in this invention specifically includes the following steps: S1. By simultaneously acquiring 2D color images and 3D point clouds of rocket wall panel welds through joint calibration, a precise mapping relationship from pixels to space is established. Construct a weld defect dataset; In a preferred embodiment, step S1 specifically comprises: At the rocket panel welding station, external hardware trigger signals synchronously control the exposure and acquisition of a 2D high-resolution color camera and a 3D high-precision sensor to scan the rocket panel welds. Authentic, defective rocket panel weld data is selected and annotated, and corresponding 2D color images are acquired while ensuring consistent timestamps. and 3D point cloud A weld defect dataset was constructed; and pre-defined mapping relationships were used. make sure Pixels in There are definite spatial correspondences between the spatial points in the equation, that is... , for pixel coordinates in for The spatial point coordinates, the mapping relationship The per-plane mapping function, obtained through target calibration, includes 2D camera intrinsics, 3D sensor laser plane equations, and hand-eye calibration extrinsic parameters for both. It is used to accurately map pixel coordinates to spatial coordinates; that is, it maps 2D camera pixel coordinates to spatial coordinates through this mapping relationship. (in For pixel column coordinates, Precisely convert pixel row coordinates to spatial coordinates in the 3D world coordinate system. The formula is expressed as: ; Based on the AWS D17.1 Class A standard (an acceptance standard for welded joints of critical aerospace structures published by the American Welding Society (AWS)), define the set of semantic categories of welds to be inspected. ; in, Indicates background, Indicates the main body of the weld; Indicates biting edge, Indicates stomata, Indicates a crack. This indicates that the weld is oxidized. Indicates splashing. Indicates weld bead, It indicates that it was burned through; exist Top Except All categories other than those mentioned above are annotated at the pixel level to generate semantic segmentation mask ground truth values ​​for subsequent training of 2D global reconnaissance network models. .

[0014] S2. Construct a 2D global reconnaissance network that takes 2D color images as input and outputs pixel-level category predictions. A composite loss function is used for training to obtain a 2D global reconnaissance network with high robustness in defect detection. In a preferred embodiment, step S2 specifically comprises: Considering that rocket panel weld defects (such as cracks) are both extremely small (low pixel ratio) and irregular in shape, while other defects (such as weld oxidation) manifest as large-area color and texture abnormalities, this embodiment uses a 2D global reconnaissance network. The U-Net codec structure is adopted, and a VGG-16 network pre-trained on the ImageNet dataset is used on the encoding path to extract multi-scale contextual features from the input 2D color image, which enables it to effectively identify large-area color difference defects. Upsampling and skip connections are performed along the decoding path. High-resolution spatial detail features extracted from shallow convolutional layers and low-resolution features with strong semantic expressiveness extracted from deep convolutional layers of the VGG-16 network are fused together through step-by-step upsampling and skip connections. This comprehensively utilizes contextual information at different scales and reconstructs pixel-level masks. Finally, the 2D global reconnaissance network outputs the original pixel-level class predictions. Its formula is expressed as: ; in, for The input image; for The output Logits tensor The total number of categories is taken in this embodiment. .

[0015] To address the extreme class imbalance problem in labeled data (i.e.) , , (for minute defects with extremely low pixel ratios), this method defines a composite loss. right Conduct supervised training, For Dice's loss and Focal loss Its formula is expressed as: ; in, and To balance the hyperparameters of the two losses; The formula is expressed as: ; in, For pixel index; For pixels Category The One-Hot truth value (0 or 1); for Output After being activated by Softmax, the pixels Category The predicted probability; The formula is expressed as: ; in, The total number of pixels in the image (i.e. ); For the model to pixels The probability of predicting correctly; For modulation factor, To focus the parameters, this embodiment will set them to enhance attention to difficult-to-distinguish samples; The category balancing factor can be dynamically determined by calculating the reciprocal of the total number of pixels of each category in the training set and normalizing it, so as to automatically balance the weights of rare defects and common categories.

[0016] In this embodiment, the Dice loss, by maximizing the overlap between the predicted result and the actual defect area, can significantly alleviate the extreme class imbalance problem caused by the "extremely small defect category area" that is common in weld detection, thereby enhancing the coverage of the actual defect area; the Focal loss, by reducing the weight of easily classified samples and enhancing the gradient contribution of difficult-to-classify samples, effectively avoids the impact of differences in the number of categories on model training, thereby improving the identification ability of difficult-to-detect defects; the composite loss function formed by the combination of the two improves both the recall ability and prediction accuracy of the defect area, and is a highly robust loss function suitable for weld defect identification.

[0017] 2D color images truth value As a 2D global reconnaissance network The input; by minimizing the composite loss function optimization All learnable parameters The optimization objective can be expressed as: ; in, To train the optimal model parameters after convergence; The network is trained until the training loss decreases and stabilizes within a preset threshold range, or until a preset maximum number of iterations is reached, thus completing the network training. The trained network is denoted as […]. .

[0018] S3. Deploy the trained 2D global reconnaissance network to infer from the real-time acquired 2D color images, obtain pixel-level category predictions, and then perform connected component analysis and noise filtering to generate guidance signals that provide guidance and basis for subsequent defect detection; the guidance signals include a global mask for welds and a list of suspected defects. In a preferred embodiment, step S3 specifically includes: The trained 2D global reconnaissance network is deployed on an online detection system, taking real-time acquired 2D color images as input and outputting raw pixel-level class predictions. The formula is expressed as: ; right Perform the argmax operation along the category dimension, that is, for each pixel position. Select its predicted values ​​across all categories. The category index corresponding to the largest item in the list yields a size of The original segmentation mask The formula is expressed as: ; It includes all the tiny, possibly noisy, predicted pixels; right Perform connected component analysis to eliminate isolated pixels caused by sensor noise or model misjudgment: pre-calibrate the minimum defect pixel area threshold based on the statistical distribution of noise samples. In this embodiment, the minimum defect pixel area threshold is pre-calibrated based on the statistical distribution of noise samples. Specifically: A representative noise sample set was collected, which included sensor background noise images without welded workpieces, as well as data collected using a 2D reconnaissance network. Artifact mask generated when performing inference on a known defect-free weld area ; Perform connected component analysis on all masks in the noise sample set to obtain the area of ​​all noise spots and generate a histogram of the statistical distribution of these areas; Will Set as the statistical distribution of noise spot area quantiles, among which High percentile level; preferably 95%–99%; verify Does it meet the requirements? ,in The expected pixel area of ​​the minimum acceptable defect as defined by the AWS D17.1 Class A standard at this imaging resolution, where η is a preset safety margin coefficient, and 0 < η < 1, preferably η = 0.5 to 0.8; if the above conditions are not met, the noise threshold calibration is deemed invalid. When the calibration is determined to be invalid, a recalibration process is executed, which includes at least one of the following methods: (a) Reacquire the noise sample set and recalculate under the current imaging parameters. ; (b) After adjusting the imaging illumination and / or exposure conditions, re-acquire the noise sample set under the new imaging conditions and recalculate. ; (c) Adjust the high quantile level while keeping the noise sample set unchanged. To redetermine ; Until satisfied The method completes the calibration by applying the constraints and using a calibration judgment based on the minimum defect pixel area threshold. This ensures that noise is filtered out and that real defects are not mistakenly deleted.

[0019] After that Perform 8-neighborhood connected component analysis on all non-background pixels to obtain the number of pixels containing the background pixels. A set of independent connected components Calculate its pixel area ,like If the connected component is determined to be a pseudo-defect caused by sensor noise or model isolation misjudgment, then... The category index of all pixels within the mask is reset to the background index, thus completing the removal of noisy connected components and obtaining the cleaned segmentation mask. ; from Extract the guidance signal, including the global mask of the weld and the list of suspected defects: A binary mask is formed by all pixels whose category index is the main body of the weld, which serves as the global mask for the weld. For all connected components of defects whose category indices are neither background nor weld body, the category indices of these connected components and their minimum bounding rectangle coordinates are combined to generate a structured list of suspected defects. Each element in the list of suspected defects contains the category prediction obtained from the 2D reconnaissance network. .

[0020] S4. Extract the main point cloud of the weld and the base material point clouds on both sides of the main weld from the 3D point cloud based on the global mask of the weld; fit the zero-height reference plane based on the base material point clouds on both sides, and quantitatively analyze the overall macroscopic geometric indicators of the weld. In a preferred embodiment, step S4 specifically comprises: According to the calibrated mapping relationship The pixels covered by the global mask of the weld are mapped to 3D space, and the corresponding part of the point cloud in the 3D point cloud is extracted to generate the main point cloud of the weld. The pixels in the background area, excluding the global mask of the weld, are mapped to 3D space. Partial point clouds corresponding to these spatial locations are extracted from the 3D point cloud to generate the original point clouds of the base material on both sides of the weld. ; Calculate the outer contour of the global mask for the weld and decompose it into the left contour. and right outline Define a neighborhood width that simultaneously avoids the heat-affected zone and ensures stable fitting. ,Will Shift to the left 1 pixel, generate the left parent material mask ;Will Translate to the right 1 pixel, generate the right parent material mask .

[0021] It should be noted that the neighborhood width in this embodiment satisfies the following two conditions: First, it is necessary to avoid the heat-affected zone (HAZ), i.e. Defined sampling start position (i.e. and The outward translation distance should be greater than the maximum expected width of the HAZ known in the welding process. This is to ensure that the fit to the zero-height reference plane is not affected by HAZ geometric deformation; Secondly, it is necessary to ensure the stability of the fit, that is... The width of the defined sampling region (i.e. the number of pixels translated) should be large enough to ensure that enough inliers are provided for the subsequent RANSAC algorithm; Therefore, in this embodiment, as a preferred implementation, It can be set to the number of spatial sampling points of a 3D sensor (line laser profilometer) in the Y direction (perpendicular to the weld direction). ( The pixel width corresponding to each sampling point is used to ensure that... and The region always contains a sufficient number of 3D data points to ensure the robustness of subsequent RANSAC plane fitting.

[0022] Considering that in actual welding conditions, the base material point clouds on both sides of the weld will inevitably be subject to geometric noise caused by defects such as spatter, these noise points, as "outsiders," will seriously interfere with the accuracy of traditional least squares plane fitting. To combat this noise, this embodiment uses the RANSAC algorithm to fit a reference plane on the base material point clouds on both sides of the weld. use from Extracting the original point cloud of the left parent material ,right The RANSAC plane fitting algorithm is applied to find the fitting plane that receives the most inlier support. Outliers are then removed to obtain the left reference plane. , The plane equation is ; Similarly, the original point cloud of the parent material on the right side is obtained. and the reference plane on the right , The plane equation is

[0023] Based on weld seam main point cloud Left reference plane Right reference plane The overall macroscopic geometric parameters of the weld are quantified; these parameters include: misalignment, excess height, and indentation. The specific quantification process is as follows: Calculate the left reference plane and the right reference plane The average vertical height difference between them is used as the misalignment amount. ; for One of the weld points , for Calculate the three-axis coordinates. to and arithmetic mean of signed vertical distances The formula is expressed as: ; It should be noted that in this embodiment... In reality, it is the average relative height of the weld point with respect to the left and right reference planes of the base material; Will As Zero-height reference distance; traversal For each weld point, count all zero-height reference distances. The largest positive value among them is taken as the residual height. The smallest negative value is taken as the indentation amount. ,Right now A zero-height reference plane is defined based on all the zero-height reference distances collected. ; Encapsulated , as a macroscopic geometric indicator for subsequent rulings.

[0024] S5. Extract the local 3D point cloud of each suspected defect region in the suspected defect list, extract the key cross section from the local 3D point cloud using the maximum feature strategy, and quantize a set of micro-geometric feature descriptors of the suspected defect regions based on the key cross section and the zero-height reference plane. In a preferred embodiment, step S5 specifically comprises: Iterate through each suspected defect region in the suspected defect list, for the first... A suspected defect area According to the mapping relationship It maps pixel regions on a 2D color image to 3D space, and extracts pixels from the 3D point cloud based on the extent of the mapped region in the 3D space. Corresponding local 3D point cloud ; For each extracted local 3D point cloud First, the maximum feature strategy is used to extract the key cross-sectional contour lines. In this embodiment, the maximum feature strategy specifically refers to: for The same calculation method as above is used. Calculate in the same way Each weld point relative height Search for the point with the maximum absolute relative height. The key section is defined as the point through which the key cross section passes. Furthermore, by extracting all 3D points from the unique cross-section perpendicular to the weld direction, the key cross-section contour line is obtained. .

[0025] Then examine the profile of this key section. Preprocessing is performed using a Savitzky-Golay filter to filter out impurities while preserving the contour shape. High-frequency noise in the medium; after filtering The zero-height reference plane Compare and calculate Compared to 3D geometric feature descriptor, ; That is, the first Microscopic geometric feature descriptor for a suspected defect region; in, Outline Compared to Maximum depth; Outline Compared to The maximum height; in this embodiment, to ensure the measurement consistency of steps S4 and S5, the maximum height will be... and Defined as all points on the contour line relative height The minimum and maximum values; Outline The curvature; Outline exist The cross-sectional area below; Outline The shape factor is a dimensionless parameter used to distinguish between sharp V-shaped grooves and gentle U-shaped grooves (mainly used to differentiate the geometric morphological differences inside the cross-section of concave defects to determine the sharpness or gentleness of the concave section). It is calculated... get; Encapsulate the microscopic analysis results for each Calculate 3D geometric feature descriptors And associate it with its corresponding 2D reconnaissance category; all The 3D geometric feature descriptors are combined and encapsulated into a structured list of micro-geometric feature descriptors. .

[0026] S6. Based on the overall macroscopic geometric indicators of the weld and the microscopic geometric feature descriptors of the suspected defect areas, the weld defects are first macroscopically adjudicated, and then the 2D reconnaissance results are microscopically deambigued according to the AWS D17.1 Class A standard, and finally the weld defect adjudication list is output. In a preferred embodiment, step S6 specifically includes: Define an acceptance criteria library, clarify the quantitative rules for weld defect fusion adjudication based on AWS D17.1 Class A standard, and formulate weld inspection acceptance criteria and threshold definition documents, including: Defect categories are divided into zero-tolerance categories and quantification categories. The zero-tolerance category includes: crack, weld beads, Weld oxidation, Burn-through, quantitative categories include Biting edge Stomata Splash; Set acceptable macro-geometry quantization thresholds for the quantization category, including: the maximum allowable depth of undercut. Maximum permissible pore diameter Maximum permissible height of splash and geometric tolerances used to determine flatness. ; The macroscopic geometric quantization threshold is derived from the corresponding clause of the AWS D17.1 Class A standard, and its specific value is automatically updated as the standard version changes; Define a shape factor threshold to distinguish between sharp V-shaped grooves and gentle U-shaped grooves. ; After defining the acceptance criteria library, the integration and adjudication process is carried out. The first step is to perform macroscopic defect adjudication, based on the overall macroscopic geometric indicators of the weld. The macroscopic geometric indicators are compared with the set macroscopic geometric quantification thresholds for the weld. If any macroscopic geometric indicator exceeds the set threshold, the entire weld is directly judged as unqualified, and no further microscopic defect judgment process is required. If all macroscopic indicators meet the requirements, the microscopic defect judgment process is executed, and the macroscopic judgment result is output. ; Perform serial adjudication of micro-defects, using a list of sub-descriptions based on micro-geometric features. For input, iterate through each suspected defect area in the list. The corresponding 3D geometric feature descriptor ; Again Primary key, 2D reconnaissance category To aid in the adjudication process, the specific defect category to which each suspected defect area belongs is determined, and a preliminary microscopic adjudication result is output. .

[0027] In this embodiment, the further... Primary key, 2D reconnaissance category To aid in the adjudication process, the specific defect category to which each suspected defect area belongs is determined, and a preliminary microscopic adjudication result is output. Specifically: First, based on geometric tolerances and , The size of the defects is used to classify them into three main categories: depressions, bulges, and flat surfaces. when At that time, it was judged as a dent-type defect (including cracks, undercuts, burn-through, and porosity). when At that time, it was judged as a protruding defect (including weld beads and spatter); when and When the defect is identified as a flat defect (including only weld oxidation), it is classified as a flat defect. Within each major category, the specific defect type is determined based on the 2D reconnaissance category. It should be noted that in this embodiment, the 2D reconnaissance category is only used as auxiliary information to refine the specific defect type within the same 3D geometric major category. Among them, concave defects are further subdivided into two geometric subcategories: V-shaped sharp grooves and U-shaped gentle grooves. The 2D candidate subset corresponding to V-shaped sharp grooves is {crack, undercut}, the 2D candidate subset corresponding to U-shaped gentle grooves is {burn-through, porosity}, the 2D candidate subset corresponding to protruding defects is {weld bead, spatter}, and the 2D candidate subset corresponding to flat defects is {weld oxidation}.

[0028] In this embodiment, after classifying the major categories of defect types, the possible scenarios for serially determining the defect type of each suspected defect region in the micro-defect analysis are as follows: (1) When it is judged to be a concave defect (i.e. ); Ⅰ. If shape factor If so, it will be further determined to be a V-shaped sharp groove; In 2D reconnaissance category As supplementary information, further decisions will be made if... for If a crack is detected, it is classified as a zero-tolerance defect, and a preliminary microscopic assessment result is output. : Determined to be a crack; if for If there is an edge bite, it is judged as a quantization defect, and a preliminary microscopic judgment result is output. It is determined to be an edge bite; II. If shape factor If so, it will be further determined to be a U-shaped gentle trough; like for If the defect is burn-through, it is considered a zero-tolerance defect, and a preliminary microscopic determination result is output. : Determined to be burn-through; if for (Porosity) is then identified as a quantitative defect, and a preliminary microscopic determination result is output. It is determined to be a pore; In this embodiment, since the 3D geometry category is determined by the maximum depth With maximum height The only certainty is that when the specific defect type given in the 2D ruling is inconsistent with the 3D geometric category or its geometric sub-category candidate subset (for example, it is judged as a dent defect, but the 2D ruling is different), the 2D ruling will not be consistent with the 3D geometric category or its geometric sub-category candidate subset. If (pores) are detected, the 2D decision result is determined to be a misjudgment or an optical artifact and is not used as a specific category basis. The 3D geometric category and its micro-geometric feature descriptor are used as the main key to complete the deambiguation and final refinement decision. Furthermore, in determining whether a defect belongs to (biting edge) or When dealing with (pores), there may be situations where the 2D decision result cannot be determined. This means that the probability difference between the 2D reconnaissance network's recognition of the bite and the pore is insufficient to reach the confidence threshold. In this case, the 3D geometric feature descriptor is used to further refine the judgment and obtain the final decision result.

[0029] (2) When it is judged to be a protrusion-type defect (i.e. ): Ⅰ. If for (Weld beads) are then classified as zero-tolerance defects, and a preliminary microscopic determination result is output. It was determined to be weld bead; II. If for (Splashing) indicates a quantization defect, and the preliminary microscopic assessment result is output as follows: It was determined to be a splash. III. Similarly, ambiguity can also arise with protruding defects, such as... (biting edge) / (pores) / (crack) / (Bur-through) indicates that the 2D assessment result classifies it as a dent defect, but this conflicts with the 3D assessment result classifying it as a protrusion defect (weld bead or spatter). Since the two are in conflict, a deambiguation operation is required: using the 3D geometric category result as the primary key, the dent category obtained from the 2D investigation is completely rejected, and the specific type is reassessed based on the 3D geometric feature descriptor of the suspected defect. The deambigued preliminary microscopic assessment result is then output. .

[0030] (3) When it is determined to be a flat defect (i.e.) and ): Ⅰ. If for (Weld oxidation) indicates that the 2D assessment result is a color difference, which is consistent with the flatness defect in the 3D assessment result. It is judged as a zero-tolerance defect, and the preliminary microscopic assessment result is output. The weld was determined to be oxidized. II. If for (biting edge) / (pores) / Non-flat defects such as cracks indicate that the 2D adjudication results suggest the defects are geometric, but the 3D adjudication results classify them as flat defects, resulting in a conflict. A deambiguation operation is required, determining the 2D adjudication results as optical artifacts, and outputting preliminary microscopic adjudication results. Ignore as dirt / shadow / artifact.

[0031] Summary of macro-level rulings and preliminary micro-level rulings Based on the defect categories determined by the preliminary microscopic assessment, all defects classified as zero-tolerance (i.e., cracks, weld oxidation, weld beads, and burn-through) are directly marked as rejected. For all defects classified as quantitative, a comparison is made... The various geometric features in the data are compared with the macroscopic geometric quantization threshold of the weld. Quantization category defects that exceed the threshold are marked as rejected, while quantization category defects that do not exceed the threshold are marked as accepted.

[0032] In this embodiment, the threshold determination for quantization defects is as follows: For defects identified as undercut, and Comparison: like If the defect is deemed to exceed the limit, the decision is rejected. like If so, the final ruling for the defect is that it is acceptable. For defects identified as porosity, the cross-sectional area is used as the initial basis. With maximum depth Calculating aperture using geometric approximation formulas (In this embodiment, it is assumed that the key cross-sectional profile of the porosity defect is approximated as a parabola; according to geometry, the area of ​​the parabolic groove...) ,depth and opening width Satisfying Relationships Therefore, the aperture (corresponding to) ) can be (correspond )and (correspond Approximate calculation yields: that is Then, the calculated aperture and For comparison: like If the defect is deemed to exceed the limit, the decision is rejected. like If so, the final ruling for the defect is that it is acceptable. For defects identified as splashing, and Comparison: like If the defect is deemed to exceed the limit, the decision is rejected. like If the defect is deemed acceptable, then the final ruling is that it is qualified.

[0033] All defect entries that are judged as either rejected or accepted are combined to generate a complete and unique weld defect decision list. The defect entry is a structured data unit that includes defect spatial location information, defect category, and final decision mark (accept / reject).

[0034] S7. Based on the weld defect adjudication list, encapsulate the structured digital quality archive report and output it to the downstream quality traceability system in JSON format; In a preferred embodiment, the structured digital quality archive report includes the following information: Metadata, including the timestamp of the current inspection and the unique identifier of the inspected workpiece; Macroeconomic analysis results, including macroeconomic geometric indicators and macroeconomic adjudication results; Weld Defect Judgment List ; Defect snapshots in the original data; the defect snapshot acquisition process is as follows: traversal For all defect entries deemed rejected (including defect spatial location information, defect category, and structured data units marked as accepted / rejected in the final decision), a data cropping operation is performed once for each suspected defect region marked as rejected, from the original 2D color image. The local 2D pixel region corresponding to the suspected defect area is cropped out from the original 3D point cloud to generate a 2D snapshot. In the process, a subset of the local 3D point cloud corresponding to the suspected defect area is extracted to obtain a 3D snapshot; the defect snapshot consists of a 2D snapshot and a 3D snapshot, serving as visual evidence of the suspected defect area.

[0035] This structured digital quality archive report, in JSON data stream format, can be output in real time to downstream quality traceability and management systems or weld digital archive databases via a standardized data interface, enabling full lifecycle traceability and archiving of the rocket panel weld quality.

[0036] In summary, this invention proposes a high-precision detection method for rocket panel weld defects based on visual guidance and geometric deambiguation. It introduces 2D global reconnaissance to provide guidance signals with high recall and utilizes a serial analysis strategy for high-precision geometric feature extraction using 3D local quantization. The fusion adjudication rule engine of this invention, combining 2D vision and 3D assemblies, uses 3D geometric features as the primary key to perform deterministic geometric deambiguation adjudication on the detected 2D visual categories, rejecting optical artifacts while resolving modal conflicts and refining geometric classification.

[0037] The method proposed in this invention ensures that, even in the face of interference from 2D optical artifacts and 3D sensor noise, the weld quality inspection work of aerospace-grade rocket panels can still capture high-precision defect quantification results that meet the AWS D17.1 Class A standard, providing highly robust and traceable sensing data and support for downstream quality traceability systems and digital quality archives.

[0038] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0039] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0040] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation, characterized in that, Specifically, the following steps are included: By jointly calibrating and synchronously acquiring 2D color images and 3D point clouds of rocket wall panel welds, a precise mapping relationship from pixels to space is established, and a weld defect dataset is constructed. Construct a 2D global reconnaissance network that takes 2D color images as input and outputs pixel-level category predictions. A composite loss function is used for training to obtain a 2D global reconnaissance network with high robustness in defect detection. A pre-trained 2D global reconnaissance network is deployed to infer from real-time acquired 2D color images, obtain pixel-level category predictions, and then perform connected component analysis and noise filtering to generate guidance signals that provide guidance and basis for subsequent defect detection; the guidance signals include a global mask for welds and a list of suspected defects. The weld body point cloud and the base material point clouds on both sides of the weld body are extracted from the 3D point cloud based on the global mask of the weld. A zero-height reference plane is fitted based on the base material point clouds on both sides, and the overall macroscopic geometric indicators of the weld are quantitatively analyzed. Extract the local 3D point cloud of each suspected defect region in the suspected defect list, extract the key cross section from the local 3D point cloud using the maximum feature strategy, and quantize a set of micro-geometric feature descriptors of the suspected defect regions based on the key cross section and the zero-height reference plane. Based on the overall macroscopic geometric indicators of the weld and the microscopic geometric feature descriptors of the suspected defect areas, the weld defects are first macroscopically adjudicated, and then the 2D reconnaissance results are microscopically deambigued according to the AWS D17.1 Class A standard, and finally a weld defect adjudication list is output. Based on the weld defect adjudication list, a structured digital quality archive report is encapsulated and output to the downstream quality traceability system in JSON format.

2. The high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 1, characterized in that, The process of jointly calibrating and synchronously acquiring 2D color images and 3D point clouds of rocket wall panel welds to establish a precise pixel-to-space mapping relationship and construct a weld defect dataset specifically involves: At the rocket panel welding station, external hardware trigger signals synchronously control the exposure and acquisition of a 2D high-resolution color camera and a 3D high-precision sensor to scan the rocket panel welds. Authentic, defective rocket panel weld data is selected and annotated, and corresponding 2D color images are acquired while ensuring consistent timestamps. and 3D point cloud A weld defect dataset was constructed; and pre-defined mapping relationships were used. make sure Pixels in There are definite spatial correspondences between the spatial points in the equation, that is... , for pixel coordinates in for The spatial point coordinates, the mapping relationship The per-plane mapping function obtained by calibrating the target includes 2D camera intrinsic parameters, 3D sensor laser plane equations, and hand-eye calibration extrinsic parameters of both, and is used to accurately map pixel coordinates to spatial coordinates; Define the set of semantic categories of welds to be inspected according to the AWS D17.1 Class A standard. ; in, Indicates background, Indicates the main body of the weld; Indicates biting edge, Indicates stomata, Indicates a crack. This indicates that the weld is oxidized. Indicates splashing. Indicates weld bead, It indicates that it was burned through; exist Top Except All categories other than those mentioned above are annotated at the pixel level to generate semantic segmentation mask ground truth values ​​for subsequent training of 2D global reconnaissance network models. .

3. The high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 1, characterized in that, The 2D global reconnaissance network that takes 2D color images as input and outputs pixel-level category predictions. A composite loss function is used for training to obtain a 2D global reconnaissance network with high robustness in defect detection. Specifically: 2D Global Reconnaissance Network This paper adopts a U-Net encoder-decoder architecture. In the encoding path, a VGG-16 network pre-trained on the ImageNet dataset is used to extract multi-scale contextual features from the input 2D color image. In the decoding path, upsampling and skip connections are performed. High-resolution spatial detail features extracted from shallow convolutional layers and low-resolution features with strong semantic expressiveness extracted from deep convolutional layers by the VGG-16 network are fused with these features through progressive upsampling and skip connections. This comprehensive utilization of contextual information at different scales allows for the reconstruction of pixel-level masks. Finally, the 2D global reconnaissance network outputs the original pixel-level class predictions. ; Define composite loss right Conduct supervised training, For Dice's loss and Focal loss The weighted sum; 2D color images truth value As a 2D global reconnaissance network The input; by minimizing the composite loss function optimization All learnable parameters This reduces the training loss and stabilizes it within a preset threshold range, or reaches the preset maximum number of iterations, thereby completing network training.

4. The high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 1, characterized in that, The pre-trained 2D global reconnaissance network performs inference on real-time acquired 2D color images, obtains pixel-level category predictions, and then performs connected component analysis and noise filtering to generate guidance signals that provide guidance and basis for subsequent defect detection. Specifically: The trained 2D global reconnaissance network is deployed on an online detection system, taking real-time acquired 2D color images as input and outputting raw pixel-level class predictions. , For one The Logits tensor, Image size, This represents the total number of semantic categories for weld seams. right Perform the argmax operation along the category dimension, that is, for each pixel position. Select its predicted values ​​across all categories. The category index corresponding to the largest item in the list yields a size of The original segmentation mask ; right Perform connected component analysis to eliminate isolated pixels caused by sensor noise or model misjudgment: pre-calibrate the minimum defect pixel area threshold based on the statistical distribution of noise samples. ,right Perform 8-neighborhood connected component analysis on all non-background pixels to obtain the number of pixels containing the background pixels. A set of independent connected components Calculate its pixel area ,like If the connected component is determined to be a pseudo-defect caused by sensor noise or model isolation misjudgment, then... The category index of all pixels within the mask is reset to the background index, thus completing the removal of noisy connected components and obtaining the cleaned segmentation mask. ; from Extract the guidance signal, including the global mask of the weld and the list of suspected defects: A binary mask is formed by all pixels whose category index is the main body of the weld, which serves as the global mask for the weld. For all connected components of defects whose category indices are neither background nor weld body, the category indices of these connected components and their minimum bounding rectangle coordinates are combined to generate a structured list of suspected defects. Each element in the list of suspected defects contains the category prediction obtained from the 2D reconnaissance network. .

5. A high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 4, characterized in that, The minimum defect pixel area threshold is pre-calibrated based on the statistical distribution of noise samples. Specifically: Collect a representative noise sample set, which includes sensor background noise images without welded workpieces, and artifact masks generated when inferring known defect-free weld areas using a 2D reconnaissance network. Perform connected component analysis on all masks in the noise sample set to obtain the area of ​​all noise spots and generate a histogram of the statistical distribution of these areas; Will Set as the statistical distribution of noise spot area quantiles, among which High quantile level; verify Does it meet the requirements? ,in The expected pixel area of ​​the minimum acceptable defect as defined by the AWS D17.1 Class A standard at this imaging resolution, where η is a preset safety margin coefficient and 0 < η < 1; if the above conditions are not met, the noise threshold calibration is deemed invalid. When the calibration is determined to be invalid, a recalibration process is executed, which includes at least one of the following methods: (a) Reacquire the noise sample set and recalculate under the current imaging parameters. ; (b) After adjusting the imaging illumination and / or exposure conditions, re-acquire the noise sample set under the new imaging conditions and recalculate. ; (c) Adjust the high quantile level while keeping the noise sample set unchanged. To redetermine ; Until satisfied Under the constraints, the calibration was completed.

6. The high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 1, characterized in that, The process involves extracting the main weld point cloud and the base material point clouds on both sides of the weld from the 3D point cloud based on the global weld mask; fitting a zero-height reference plane based on the base material point clouds on both sides; and quantitatively analyzing the overall macroscopic geometric indices of the weld. According to the calibrated mapping relationship The pixels covered by the global mask of the weld are mapped to 3D space, and the corresponding part of the point cloud in the 3D point cloud is extracted to generate the main point cloud of the weld. ; The pixels in the background area, excluding the global mask of the weld, are mapped to 3D space. Partial point clouds corresponding to these spatial locations are extracted from the 3D point cloud to generate the original point clouds of the base material on both sides of the weld. ; Calculate the outer contour of the global mask for the weld and decompose it into the left contour. and right outline Define a neighborhood width that simultaneously avoids the heat-affected zone and ensures stable fitting. ,Will Shift to the left 1 pixel, generate the left parent material mask ;Will Translate to the right 1 pixel, generate the right parent material mask ; The RANSAC algorithm is used to fit a reference plane on the base material point cloud on both sides of the weld: use from Extracting the original point cloud of the left parent material ,right The RANSAC plane fitting algorithm is applied to find the fitting plane that receives the most inlier support. Outliers are then removed to obtain the left reference plane. Similarly, the original point cloud of the parent material on the right side is obtained. and the reference plane on the right ; Based on weld seam main point cloud Left reference plane Right reference plane The overall macroscopic geometric parameters of the weld are quantified; these parameters include: misalignment, excess height, and indentation. The specific quantification process is as follows: Calculate the left reference plane and the right reference plane The average vertical height difference between them is used as the misalignment amount. ; for One of the weld points ,calculate to and arithmetic mean of signed vertical distances ,Will As Zero-height reference distance; traversal For each weld point, count all zero-height reference distances. The largest positive value among them is taken as the residual height. The smallest negative value is taken as the indentation amount. A zero-height reference plane is defined based on all the zero-height reference distances collected. ; Encapsulated , as a macro-geometric indicator for subsequent rulings.

7. The high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 1, characterized in that, The process involves extracting the local 3D point cloud of each suspected defect region from the suspected defect list, extracting key cross-sections from the local 3D point cloud using a maximum feature extraction strategy, and quantifying a set of microscopic geometric feature descriptors for the suspected defect regions based on the key cross-sections and the zero-height reference plane. Specifically, this process is as follows: Iterate through each suspected defect region in the suspected defect list, for the first... A suspected defect area According to the mapping relationship This process maps pixel regions on a 2D color image to 3D space, and extracts pixels from the 3D point cloud based on the extent of the mapped region in the 3D space. Corresponding local 3D point cloud ; For each extracted local 3D point cloud First, the maximum feature strategy is used to extract the key cross-sectional contour lines. Then, the key cross-sectional contour line Preprocessing is performed using a Savitzky-Golay filter to filter out impurities while preserving the contour shape. High-frequency noise in the medium; after filtering The zero-height reference plane Compare and calculate Compared to 3D geometric feature descriptor, ; That is, the first Microscopic geometric feature descriptor for a suspected defect region; in, Outline Compared to Maximum depth; Outline Compared to Maximum height; Outline The curvature; Outline exist The cross-sectional area below; Outline The shape factor, a dimensionless parameter used to distinguish between sharp V-shaped grooves and gentle U-shaped grooves, is calculated... get; Encapsulate the microscopic analysis results for each Calculate 3D geometric feature descriptors And associate it with its corresponding 2D reconnaissance category; all The 3D geometric feature descriptors are combined and encapsulated into a structured list of micro-geometric feature descriptors. .

8. A high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 7, characterized in that, The macroscopic geometric indices based on the overall weld and the microscopic geometric feature descriptors of suspected defect areas are used to first make macroscopic decisions on weld defects, and then perform microscopic geometric deambiguation decisions on the 2D reconnaissance results according to the AWS D17.1 Class A standard. The final output weld defect decision list is as follows: Define an acceptance criteria library, clarify the quantitative rules for weld defect fusion adjudication based on AWS D17.1 Class A standard, and formulate weld inspection acceptance criteria and threshold definition documents, including: Defects are categorized into zero-tolerance and quantitative categories. Zero-tolerance categories include cracks, weld beads, weld oxidation, and burn-through. Quantitative categories include undercut, porosity, and spatter. Set acceptable macro-geometry quantization thresholds for the quantization category, including: the maximum allowable depth of undercut. Maximum permissible pore diameter Maximum permissible height of splash and geometric tolerances used to determine flatness. ; Define a shape factor threshold to distinguish between sharp V-shaped grooves and gentle U-shaped grooves. ; After defining the acceptance criteria library, the integration and adjudication process is carried out. The first step is to perform macroscopic defect adjudication, based on the overall macroscopic geometric indicators of the weld. The macroscopic geometric indicators are compared with the set macroscopic geometric quantification thresholds for the weld. If any macroscopic geometric indicator exceeds the set threshold, the entire weld is directly judged as unqualified, and no further microscopic defect judgment process is required. If all macroscopic indicators meet the requirements, the microscopic defect judgment process is executed, and the macroscopic judgment result is output. ; Perform serial adjudication of micro-defects, using a list of sub-descriptions based on micro-geometric features. For input, iterate through each suspected defect area in the list. The corresponding 3D geometric feature descriptor ; Again Primary key, 2D reconnaissance category To aid in the adjudication process, the specific defect category to which each suspected defect area belongs is determined, and a preliminary microscopic adjudication result is output. ; Summary of macro-level rulings and preliminary micro-level rulings Based on the defect categories determined by the preliminary micro-level adjudication results, all defects adjudicated as zero-tolerance categories are directly marked as rejected. For all defects adjudicated as quantified categories, the geometric features in the comparison are compared with the macro-geometric quantification threshold of the weld. Quantified category defects that exceed the threshold are marked as rejected, and quantified category defects that do not exceed the threshold are marked as accepted. All defect entries that are judged as either rejected or accepted are combined to generate a complete and unique weld defect decision list. The defect entry is a structured data unit that includes defect spatial location information, defect category, and final decision mark (accept / reject).

9. A high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 8, characterized in that, The above again Primary key, 2D reconnaissance category To aid in the adjudication process, the specific defect category to which each suspected defect area belongs is determined, and a preliminary microscopic adjudication result is output. Specifically: First, based on geometric tolerances... and , The size of the defects is used to classify them into three main categories: depressions, bulges, and flat surfaces. when At that time, it was judged to be a dent-type defect; when At that time, it was judged to be a protrusion-type defect; when and At that time, it was determined to be a flat defect; Within each major category, the specific defect type is further determined based on the 2D reconnaissance category. If the specific defect type obtained conflicts with the classification result based on the major category, the major category classification result shall prevail for deambiguation.

10. A high-precision detection method for rocket wall panel weld defects based on vision guidance and geometric deambiguation according to claim 1, characterized in that, The structured digital quality archive report includes the following information: Metadata, including the timestamp of the current inspection and the unique identifier of the inspected workpiece; Macroeconomic analysis results, including macroeconomic geometric indicators and macroeconomic adjudication results; Weld Defect Judgment List ; Defect snapshots in the original data; the defect snapshot acquisition process is as follows: traversal For all defect entries deemed rejected, a data cropping operation is performed on each suspected defect region marked as rejected, from the original 2D color image. The local 2D pixel region corresponding to the suspected defect area is cropped out from the original 3D point cloud to generate a 2D snapshot. In the process, a subset of the local 3D point cloud corresponding to the suspected defect area is extracted to obtain a 3D snapshot; the defect snapshot consists of a 2D snapshot and a 3D snapshot, serving as visual evidence of the suspected defect area.

Images