An outer surface residual coating area AI automatic detection method

Abstract

The embodiment discloses an AI automatic detection method for residual coating area on the outer surface. Firstly, a two-dimensional image, a depth map and three-dimensional point cloud data of the outer surface of a workpiece to be detected are collected, and a mapping relationship between two-dimensional pixel coordinates and three-dimensional point cloud coordinates is established. Then, an AI semantic segmentation model is used to automatically detect the residual coating area in the two-dimensional image, so as to obtain a two-dimensional residual coating probability graph and a pixel-level residual coating area mask. Next, the residual coating boundary is sub-pixel positioned by using the gray gradient information in the two-dimensional image, so as to generate a two-dimensional sub-pixel residual coating boundary point set, a continuous sub-pixel-level residual coating boundary and a sub-pixel-level residual coating area mask. Finally, the sub-pixel-level residual coating area mask is mapped to the three-dimensional point cloud space, the residual coating label and the residual coating confidence are given to the three-dimensional point cloud points, and a detection judgment result is output.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection and 3D point cloud fusion inspection technology, and in particular to an AI automatic detection method for residual coating areas on outer surfaces. Background Technology

[0002] Existing methods for detecting residual coatings on external surfaces mainly include manual visual inspection, 2D image detection based on color thresholds, 2D image detection based on semantic segmentation models, and geometric detection methods based on 3D point clouds. Manual visual inspection relies on operator experience, resulting in low efficiency and significant subjective influence on results. 2D image detection methods based on color thresholds can identify some areas with significant color differences, but they are poorly adaptable to changes in lighting, surface reflection, color gradations, and complex textures. 2D image detection methods based on semantic segmentation models can automatically identify residual coating areas, but their output is typically a pixel-level mask, with pixel quantization errors at the boundaries. Geometric detection methods based on 3D point clouds can obtain the 3D morphology of the workpiece surface, but they are difficult to accurately identify residual coating areas that only exhibit differences in color, texture, or grayscale without significant changes in geometric height.

[0003] Existing technologies for detecting residual coatings on the outer surface of complex curved workpieces mainly suffer from the following problems: 1. When relying solely on two-dimensional image detection, the detection results remain within the two-dimensional image coordinate system, making it difficult to directly obtain the true spatial position of the residual coating area on the three-dimensional workpiece surface; 2. The pixel-level mask output by ordinary semantic segmentation models has edge jaggedness and pixel quantization errors, affecting the accuracy of residual coating boundary positioning; 3. When relying solely on three-dimensional point cloud geometric information, it is difficult to identify residual coating areas with only color, texture, or grayscale differences and small geometric shape changes; 4. Existing detection methods typically only output two-dimensional detection boxes or two-dimensional masks, lacking three-dimensional detection output that simultaneously includes three-dimensional spatial position, three-dimensional boundary, region area estimate, region confidence, and detection judgment result. Summary of the Invention

[0004] This invention discloses an AI-based automatic detection method for residual coating areas on outer surfaces to overcome the aforementioned technical problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: An AI-based automatic detection method for residual coating areas on an outer surface includes the following steps: S1: Acquire the original two-dimensional image, original depth map, and original three-dimensional point cloud data of the workpiece under the same field of view, and preprocess the original two-dimensional image and original three-dimensional point cloud data to obtain the mapping relationship from two-dimensional pixel coordinates to three-dimensional point cloud coordinates; and obtain the effective set of two-dimensional pixel coordinates based on the original depth map. S2: Based on the preprocessed two-dimensional image, an AI semantic segmentation model with an encoder-decoder structure is used to obtain a two-dimensional residual coating probability map in order to obtain a pixel-level residual coating region mask. S3: Based on the pixel-level residual coating region mask and the preprocessed 2D image, obtain the sub-pixel-level boundary point set, then obtain the 2D sub-pixel residual coating boundary point set, and finally obtain the sub-pixel-level residual coating region mask. S4: Based on the two-dimensional sub-pixel residual coating boundary point set and the mapping relationship from two-dimensional pixel coordinates to three-dimensional point cloud coordinates, obtain the three-dimensional spatial point coordinates corresponding to the two-dimensional sub-pixel residual coating boundary points, so as to obtain the three-dimensional sub-pixel residual coating boundary point set. S5: Based on the sub-pixel level residual coating area mask, obtain the confidence level of the 3D point cloud residual coating to obtain the 3D point cloud residual coating label; S6: Based on the confidence of the residual coating in the 3D point cloud, obtain 3D points with attributes, and obtain a set of candidate point clouds for the 3D residual coating based on the residual coating label in the 3D point cloud; S7: Based on the set of candidate point clouds of 3D residual coating, obtain multiple connected regions of 3D residual coating, and then obtain the average confidence of residual coating and the area estimate of 3D residual coating region based on the set of effective 2D pixel coordinates. S8: Based on the estimated area of ​​the three-dimensional residual coating region and the average confidence level of the residual coating, obtain the effective set of three-dimensional residual coating detection regions to obtain the residual coating detection judgment result, and then obtain the final detection result to complete the AI ​​automatic detection of the residual coating region on the outer surface.

[0006] Furthermore, S3 includes: S31: Obtain the set of pixel-level boundary points of the pixel-level residual coating region based on the pixel-level residual coating region mask; S32: Based on the preprocessed two-dimensional image, obtain the corresponding grayscale image to obtain the local gradient magnitude in the grayscale image; S33: Obtain the sub-pixel level boundary point set based on the local gradient magnitude and pixel-level boundary point set in the grayscale image; S34: Perform local smoothing on the sub-pixel level boundary point set to obtain the smoothed sub-pixel level boundary point coordinates, so as to obtain the two-dimensional sub-pixel residual coating boundary point set. S35: Obtain the continuous sub-pixel level residual coating boundary based on the two-dimensional sub-pixel residual coating boundary point set; S36: Obtain the sub-pixel level residual coating area mask based on the continuous sub-pixel level residual coating boundary.

[0007] Furthermore, the formula used to obtain the local gradient magnitude in a grayscale image is as follows:

[0008] In the formula: grayscale image in pixel coordinates Gradient magnitude at; This is a grayscale image obtained by converting a two-dimensional image. This represents the rate of grayscale change of a grayscale image along the horizontal pixel direction. This represents the rate of grayscale change of a grayscale image along the vertical pixel direction. All are pixel coordinates in a two-dimensional image;

[0009] In the formula: This is a grayscale conversion operator; This is the preprocessed two-dimensional image.

[0010] Furthermore, the formula used to obtain sub-pixel boundary points is as follows:

[0011] In the formula: For the first Sub-pixel level boundary point coordinates; A set of pixel-level boundary points The first in Coordinates of the boundary points; For the first pixel-level boundary points A local computational window centered on the target; To calculate the pixel coordinate vector within the window; To prevent the gradient summation denominator from being zero, a smoothing coefficient is used.

[0012] Furthermore, the formula used to obtain the sub-pixel level residual coating area mask is as follows:

[0013] In the formula: Masking for subpixel-level residual coating areas; For pixels Located within the boundary of continuous subpixel level residual coating Internal area; This represents the total area of ​​a single pixel.

[0014] Furthermore, the formula used to obtain the confidence level of the residual coating in the 3D point cloud is as follows:

[0015] In the formula: For the first The confidence level that a 3D point cloud point belongs to the residual coating area; Masking subpixel residual coating areas in coordinates The value at; For the first The two-dimensional pixel coordinates corresponding to each three-dimensional point cloud point; The formula used to obtain the residual coating label of the 3D point cloud is as follows:

[0016] In the formula: For the first Residual coating label of a 3D point cloud point; The confidence threshold for residual coating in 3D point clouds; where, To indicate the first The three-dimensional point cloud points belong to the area of ​​residual coating; To indicate the first The three-dimensional point cloud points do not belong to the residual coating area.

[0017] Furthermore, S6 includes: S61: Obtain a 3D point with attributes, as shown below:

[0018] In the formula: For the first A three-dimensional point with attributes; For the first Spatial coordinates of one valid three-dimensional point; For the first The confidence level that a 3D point cloud point belongs to the residual coating area; For the first The two-dimensional pixel coordinates corresponding to each three-dimensional point cloud point; S62: Obtain the three-dimensional residual coating point cloud set, represented as follows:

[0019] In the formula: A collection of point clouds representing the residual coating in three dimensions; To preserve the 2D pixel index in the preprocessed 3D point cloud data; For the first Residual coating label of a 3D point cloud point; S63: Based on the neighborhood consistency screening formula for the three-dimensional residual coating point cloud, obtain the set of candidate point clouds for the three-dimensional residual coating after neighborhood consistency screening. The neighborhood consistency screening formula for the three-dimensional residual coating point cloud is expressed as follows:

[0020] In the formula: The set of candidate point clouds for 3D residual coatings after neighborhood consistency screening; The first point cloud in the three-dimensional residual coating set A three-dimensional point with attributes; For the first The spatial coordinates of a valid 3D point are also 3D points with attributes. Three-dimensional coordinates in; The neighborhood search radius; The threshold for the number of residual coating points within the neighborhood; All are indices of three-dimensional points; for and The Euclidean distance between them.

[0021] Furthermore, the formula used to obtain the average residual coating confidence level of the three-dimensional residual coating connected regions is as follows:

[0022] In the formula: For the first Average confidence level of residual coating in connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; For the first The number of attributed 3D points in a connected region of a 3D residual coating; For the first A three-dimensional point with attributes; For the first The confidence level that a 3D point cloud point belongs to the residual coating area.

[0023] Furthermore, the formula used to obtain the estimated area of ​​the three-dimensional residual coating region is as follows:

[0024] In the formula: For the first Area estimates of connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; For the first A three-dimensional point with attributes; For the first Local surface area weights corresponding to three-dimensional points with attributes; in,

[0025] In the formula: pixel coordinates The corresponding three-dimensional point coordinates; For the first The two-dimensional pixel coordinates corresponding to each 3D point cloud point are also 3D points with attributes. The corresponding two-dimensional pixel coordinates; The coordinates of the three-dimensional point corresponding to the horizontally effective neighboring pixel coordinates; The coordinates of the three-dimensional point corresponding to the effective neighboring pixel coordinates in the vertical direction; For vector cross product; The vector magnitude; The effective neighbor pixel coordinates are obtained by searching along the horizontal pixel direction; These are the coordinates of the effective neighboring pixels obtained by searching along the vertical pixel direction;

[0026]

[0027] In the formula: For Starting from the horizontal pixel direction, search for distance. A function of the nearest valid pixel coordinates; For Starting from the vertical pixel direction, search for distance. A function of the nearest valid pixel coordinates; It is a valid set of two-dimensional pixel coordinates.

[0028] Furthermore, S8 includes: The formula used to obtain the effective set of three-dimensional residual coating detection areas is as follows:

[0029] In the formula: This is a collection of effective three-dimensional residual coating detection areas; For the first Average confidence level of residual coating in connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; It is a set of connected regions of the three-dimensional residual coating; For the first Area estimates of connected regions of three-dimensional residual coating; The minimum effective detection area threshold; The minimum effective average confidence threshold; Based on the set of effective three-dimensional residual coating detection areas, the residual coating detection judgment result is obtained as follows:

[0030] In the formula: The result of residual coating detection; The number of effective three-dimensional residual coating detection areas; among which, To indicate that an area with valid residual coating has been detected; This indicates areas where no effective residual coating was detected; Based on the residual coating detection results, the final detection results are obtained as follows:

[0031] In the formula: This is the final set of test results; pixel coordinates The probability that the area belongs to the residual coating area; A mask for pixel-level residual coating areas; Masking for subpixel-level residual coating areas; This is the set of boundary points of the three-dimensional sub-pixel residual coating.

[0032] Beneficial Effects: The present invention provides an AI-based automatic detection method for residual coating areas on the outer surface of a workpiece. First, it acquires two-dimensional images, depth maps, and three-dimensional point cloud data of the outer surface of the workpiece to be inspected, and establishes a mapping relationship between two-dimensional pixel coordinates and three-dimensional point cloud coordinates. Then, it uses an AI semantic segmentation model to automatically detect residual coating areas in the two-dimensional image, obtaining a two-dimensional residual coating probability map and a pixel-level residual coating area mask. Next, it uses grayscale gradient information in the two-dimensional image to perform sub-pixel localization of the residual coating boundary, generating a set of two-dimensional sub-pixel residual coating boundary points, continuous sub-pixel-level residual coating boundaries, and a sub-pixel-level residual coating area mask. Finally, it maps the sub-pixel-level residual coating area mask to the three-dimensional point cloud space, assigns residual coating labels and residual coating confidence scores to the three-dimensional point cloud points, and outputs the three-dimensional sub-pixel residual coating boundary, the three-dimensional residual coating detection area, the area estimate, the area confidence score, and the detection judgment result.

[0033] This invention achieves automatic identification of residual coating areas through AI semantic segmentation, improves the accuracy of 2D boundary positioning through sub-pixel boundary extraction, and converts 2D color, texture, or grayscale detection results into 3D spatial detection results through 2D-3D mapping, thereby realizing automated, refined, and quantifiable detection of residual coating areas on external surfaces. It is applicable to detection scenarios involving aircraft skin, complex curved surfaces, spray-repaired surfaces, and other external surfaces with residual areas exhibiting color, texture, or grayscale differences. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the AI ​​automatic detection method for residual coating areas on the outer surface according to the present invention; Figure 2 This is a schematic diagram of the AI ​​automatic detection method for residual coating areas on the outer surface in an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] This embodiment describes an AI-based automatic detection method for residual coating areas on external surfaces, including the following steps: Figure 1 and Figure 2 As shown: S1: Acquire the original two-dimensional image, original depth map, and original three-dimensional point cloud data of the workpiece under the same field of view, and preprocess the original two-dimensional image and original three-dimensional point cloud data to obtain the mapping relationship from two-dimensional pixel coordinates to three-dimensional point cloud coordinates; and obtain the effective set of two-dimensional pixel coordinates based on the original depth map.

[0038] Specifically, S1 includes: S11: An industrial-grade 3D camera is used to scan the outer surface of the workpiece to be inspected, acquiring the original two-dimensional image, the original depth map, and the original three-dimensional point cloud data under the same field of view. The original two-dimensional image is one of a color image, a texture image, or a grayscale intensity image.

[0039] S12: Perform image denoising, brightness equalization, and color correction on the original two-dimensional image to reduce the impact of ambient light changes, camera noise, and local reflections on the identification of residual coatings, and obtain a preprocessed two-dimensional image.

[0040] The preprocessed two-dimensional image is represented as follows:

[0041] In the formula: Raw two-dimensional images captured by an industrial-grade 3D camera; This is the preprocessed two-dimensional image; Image preprocessing operators include image denoising, brightness equalization, and color correction.

[0042] S13: Perform outlier removal, invalid point filtering, and coordinate unification on the original 3D point cloud data to obtain geometrically preprocessed 3D point cloud data, which is used for subsequent screening of valid 3D points and establishing a correlation with 2D pixel coordinates.

[0043] The formula used in the 3D point cloud preprocessing is as follows:

[0044] In the formula: Raw 3D point cloud data acquired by an industrial-grade 3D camera; This refers to 3D point cloud data after geometric preprocessing. The point cloud preprocessing operators include outlier removal, invalid point filtering, and coordinate unification.

[0045] S14: Based on the original depth map, camera intrinsic matrix, camera extrinsic parameters, and detection coordinate system, establish the mapping relationship between two-dimensional pixel coordinates and three-dimensional spatial point coordinates to convert the two-dimensional pixel position into three-dimensional spatial point coordinates.

[0046] Specifically, the mapping relationship between two-dimensional pixel coordinates and three-dimensional spatial points is represented as follows:

[0047] In the formula: pixel coordinates The corresponding three-dimensional spatial coordinates; It is a mapping function from two-dimensional pixel coordinates to three-dimensional point coordinates; All are pixel coordinates in a two-dimensional image; pixel coordinates The corresponding effective depth value; This is the camera intrinsic parameter matrix; This is the rotation matrix between the camera coordinate system and the detection coordinate system; This is the translation vector between the camera coordinate system and the detection coordinate system.

[0048] S15: Based on the effective depth values ​​and the geometrically preprocessed 3D point cloud data, filter the set of effective 2D pixel coordinates.

[0049] Specifically, in this embodiment, the effective depth value corresponding to a two-dimensional pixel is represented as follows:

[0050] In the formula: pixel coordinates The corresponding effective depth value; Raw depth map output by an industrial-grade 3D camera in pixel coordinates The depth value at that location; All are pixel coordinates in a two-dimensional image.

[0051] Specifically, if the depth value corresponding to a pixel in the original depth map is null, invalid, or outside the detection range, then that pixel will not be included in the subsequent set of valid two-dimensional pixel coordinates.

[0052] S16: Based on the set of valid 2D pixel coordinates, construct preprocessed 3D point cloud data that retains the 2D pixel indices, ensuring that each valid 3D point retains its corresponding 2D pixel coordinates. This guarantees that each 3D point can be processed... Query the detection results of two-dimensional residual coatings to avoid losing pixel correspondences after point cloud filtering.

[0053] Specifically, the set of valid two-dimensional pixel coordinates is represented as follows:

[0054] In the formula: A set of valid two-dimensional pixel coordinates; For three-dimensional space points To point cloud collection Distance to the nearest point in the middle; The effective matching distance threshold for point clouds.

[0055] Specifically, by using valid 2D pixel coordinates, the data sources between 2D pixels, depth maps, and 3D point clouds are unified, ensuring that subsequent 3D point cloud labeling is performed only on 2D pixels with valid 3D correspondences. A distance threshold is used. Effective matching can avoid ambiguity in the precise set attribution caused by point cloud filtering, coordinate transformation, or floating-point errors.

[0056] Specifically, the 3D point cloud data representation that retains the 2D pixel index is as follows:

[0057] In the formula: To preserve the 2D pixel index in the preprocessed 3D point cloud data; For the first Spatial coordinates of one valid three-dimensional point; All are the first The two-dimensional pixel coordinates corresponding to each valid three-dimensional point; for The corresponding three-dimensional spatial coordinates; For indexes of three-dimensional points; This represents the number of valid 3D points retained after preprocessing.

[0058] Specifically, after the above preprocessing, the following are obtained: a preprocessed 2D image, a set of valid 2D pixel coordinates, preprocessed 3D point cloud data retaining the 2D pixel indices, and a mapping relationship from 2D pixel coordinates to 3D point cloud coordinates. The preprocessed 2D image is used for AI semantic segmentation in step S2, and the preprocessed 3D point cloud data and mapping relationship are used for 3D point cloud labeling in step S4.

[0059] S2: Based on the preprocessed two-dimensional image, an AI semantic segmentation model with an encoder-decoder structure is used to obtain a two-dimensional residual coating probability map in order to obtain a pixel-level residual coating region mask. Specifically, S2 includes: S21: Based on preprocessed historical 2D image samples, establish a training dataset for residual coating regions. Residual coating regions in the 2D images are labeled as foreground, and non-residual coating regions are labeled as background. The residual coating region is defined as a region in the 2D image that differs from the substrate surface in color, texture, or grayscale features.

[0060] S22: Perform random rotation, brightness adjustment, color perturbation, noise addition, and scale transformation on the labeled sample images to obtain the enhanced training and validation datasets.

[0061] S23: An AI semantic segmentation model with an encoder-decoder structure is used for training. The encoder is used to extract color, texture, grayscale, and semantic features, while the decoder is used to restore the spatial resolution of the image and output the probability that each pixel belongs to the residual coating region.

[0062] S24: Input the preprocessed 2D image into the trained AI semantic segmentation model to obtain a 2D residual coating probability map.

[0063] S25: Threshold segmentation is performed on the two-dimensional residual coating probability map to obtain the initial binary residual coating mask.

[0064] S26: Perform morphological opening and closing operations on the initial binary residual coating mask to remove isolated false detection areas and fill in the internal holes of the areas, thereby obtaining a pixel-level residual coating area mask.

[0065] Specifically, the training dataset in this embodiment is defined by the following formula table:

[0066] In the formula: A training dataset for residual coating regions is used to train an AI semantic segmentation model; For the first Zhang training sample images; For the first The manually labeled mask corresponding to the training sample image; This represents the number of training samples; This is the index for the training samples.

[0067] The total loss function for training the AI ​​semantic segmentation model in this embodiment is expressed as follows:

[0068] In the formula: This represents the total loss value of the AI ​​semantic segmentation model. This represents the binary cross-entropy loss value. This represents the Dice loss value. These are the weighting coefficients for the binary cross-entropy loss. These are the Dice loss weighting coefficients.

[0069] in,

[0070] In the formula: This represents the binary cross-entropy loss value. The total number of pixels in a training image; To train the pixel index in the image; For the first The actual label of each pixel, with a value of 0 or 1; Predicting the first for AI semantic segmentation model The probability that a pixel belongs to a residual coating area.

[0071]

[0072] In the formula: This represents the Dice loss value. The set of pixels predicted by the model as the area of ​​residual coating; This is the set of pixels in the actual annotation that belong to the residual coating area; This is the smoothing coefficient in the Dice loss, used to prevent the denominator from being zero; The number of overlapping pixels between the predicted region and the real region; To predict the number of pixels in the area with residual coating; This represents the number of pixels in the actual residual coating area.

[0073] Specifically, the total loss function is used to constrain the overall overlap between the model output region and the actual residual coating region.

[0074] In this embodiment, the formula used for training the model parameters during the training process of the AI ​​semantic segmentation model is as follows:

[0075] In the formula: These are the parameters of the AI ​​semantic segmentation model after training. AI semantic segmentation model parameters to be optimized; In the training dataset The total loss value calculated above; The training dataset is for the residual coating area.

[0076] Specifically, the two-dimensional residual coating probability map is represented as follows:

[0077] In the formula: pixel coordinates The probability that the area belongs to the residual coating area; This is the AI ​​semantic segmentation model after training. These are the parameters of the trained AI semantic segmentation model; This is the preprocessed two-dimensional image.

[0078] Specifically, based on the two-dimensional residual coating probability map, the initial binary residual coating mask is represented as follows:

[0079] In the formula: This is an initial binary residual coating mask; pixel coordinates The probability that the area belongs to the residual coating area; The threshold for determining the probability of two-dimensional residual coating; When, it represents a pixel. Preliminary assessment indicates this is an area with residual coating. When, it represents a pixel. The area was initially determined to be a non-residual coating area.

[0080] Specifically, the formula for generating a pixel-level residual coating area mask is as follows:

[0081] In the formula: A mask for pixel-level residual coating areas; This is a morphological opening operation used to remove isolated small regions; This is a morphological closing operation used to fill holes inside a region.

[0082] S3: Based on the pixel-level residual coating region mask and the preprocessed 2D image, obtain the sub-pixel-level boundary point set, then obtain the 2D sub-pixel residual coating boundary point set, and finally obtain the sub-pixel-level residual coating region mask. Preferably, S3 includes: S31: Based on the pixel-level residual coating area mask, extract the set of pixel-level boundary points of the pixel-level residual coating area to provide initial boundary point positions for sub-pixel boundary localization.

[0083] The formula used to extract the set of pixel-level boundary points for the pixel-level residual coating area is as follows:

[0084] In the formula: This is a set of pixel-level residual coating boundary points; A mask for pixel-level residual coating areas; Extract operators for boundaries.

[0085] S32: Based on the preprocessed two-dimensional image, convert the preprocessed two-dimensional image into a grayscale image, obtain the corresponding grayscale image, and obtain the local gradient magnitude in the grayscale image to describe the intensity of grayscale change near the boundary of the residual coating.

[0086] Preferably, the formula used to convert the preprocessed 2D image into a grayscale image is as follows:

[0087] In the formula: This is a grayscale image obtained by converting a two-dimensional image. This is a grayscale conversion operator; This is the preprocessed two-dimensional image; The formula used to obtain the local gradient magnitude in a grayscale image is as follows:

[0088] In the formula: grayscale image in pixel coordinates Gradient magnitude at; This is a grayscale image obtained by converting a two-dimensional image. This represents the rate of grayscale change of a grayscale image along the horizontal pixel direction. This represents the rate of grayscale change of a grayscale image along the vertical pixel direction. All are pixel coordinates in a two-dimensional image; S33: Obtain the sub-pixel level boundary point set based on the local gradient magnitude and pixel-level boundary point set in the grayscale image; Specifically, for each pixel-level boundary point, the sub-pixel offset relative to the pixel-level boundary point is calculated within its neighborhood window using the gray-level gradient distribution, resulting in a set of sub-pixel-level boundary points. This allows the calculation of the sub-pixel offset relative to the pixel-level boundary point using the gray-level variation information of the boundary neighborhood. This formula maintains the result within a range where local gradients are weak. Nearby, to avoid abnormal jumps in subpixel coordinates.

[0089] The formula for calculating sub-pixel boundary points is as follows:

[0090] In the formula: For the first Sub-pixel level boundary point coordinates; A set of pixel-level boundary points The first in Coordinates of the boundary points; For the first pixel-level boundary points A local computational window centered on the target; To calculate the pixel coordinate vector within the window; To prevent the gradient summation denominator from being zero, a smoothing coefficient is used; S34: Perform local smoothing on the sub-pixel level boundary point set to obtain the smoothed sub-pixel level boundary point coordinates, thereby obtaining a two-dimensional sub-pixel residual coating boundary point set, reducing local jitter of sub-pixel boundary points, and obtaining a more continuous boundary point sequence. The formula used for local smoothing of the sub-pixel level boundary point set is as follows:

[0091] In the formula: For the smoothed first Sub-pixel level boundary point coordinates; For the first Sub-pixel level boundary point coordinates; The offset index within the smoothing window; To smooth out half the window width; These are the local smoothing weighting coefficients.

[0092] The smoothing weights satisfy the following:

[0093] Specifically, the sum of the smoothing weights is set to 1 to ensure that the smoothed boundary points remain within the neighborhood of the original local boundary points, thus avoiding overall scale shift.

[0094] Based on the sub-pixel level boundary point coordinates, obtain the set of two-dimensional sub-pixel residual coating boundary points, defined by the following formula:

[0095] In the formula: This is the set of boundary points of the two-dimensional sub-pixel residual coating. For the smoothed first Sub-pixel level boundary point coordinates; This represents the number of two-dimensional sub-pixel boundary points. This is the index for the two-dimensional sub-pixel boundary points.

[0096] Among them, the set of two-dimensional subpixel residual coating boundary points serves as the basis for generating discrete boundary points for continuous subpixel level residual coating boundaries and three-dimensional subpixel residual coating boundaries.

[0097] S35: Based on the set of boundary points of the two-dimensional subpixel residual coating, the set of boundary points of the two-dimensional subpixel residual coating is continuously connected; for local boundary segments with discontinuities, cubic spline interpolation is used to complete them; for boundary segments with local jitter, local polynomial smoothing is used to smooth them, so as to obtain continuous subpixel level residual coating boundaries, thereby converting discrete two-dimensional subpixel boundary points into continuous closed boundaries.

[0098] The formula used to obtain the boundary of continuous sub-pixel level residual coating is as follows:

[0099] In the interval Inside:

[0100]

[0101] In the formula: The boundary of the continuous subpixel-level residual coating; The coordinates of the boundary curve of the continuous sub-pixel level residual coating in the lateral pixel direction; The coordinates of the boundary curve in the vertical pixel direction; These are the boundary curve parameters; For the first The initial parameters of the boundary curve of the continuous subpixel level residual coating; For the first The end parameters of the boundary of the continuous subpixel level residual coating; For the segmented index of continuous subpixel boundary curves; , , , All are the first Interpolation coefficients for the horizontal coordinates of the segment; , , , All are the first Interpolation coefficients for segment longitudinal coordinates; S36: Obtain a mask for the sub-pixel residual coating region based on the continuous sub-pixel residual coating boundary. Specifically, for pixels completely inside the continuous sub-pixel residual coating boundary, the mask value is 1; for pixels completely outside the continuous sub-pixel residual coating boundary, the mask value is 0; for pixels whose continuous sub-pixel residual coating boundary is cut by the sub-pixel boundary, the mask value is the proportion of the area within that pixel that belongs to the residual coating region.

[0102] The formula used to obtain the mask for the sub-pixel level residual coating area is as follows:

[0103] In the formula: Masking for subpixel-level residual coating areas; For pixels Located within the boundary of continuous subpixel level residual coating Internal area; The total area of ​​a single pixel; in, This indicates that the pixel is completely located inside the residual coating area; This indicates that the pixel is completely outside the area of ​​residual coating; This indicates that the pixel is cut by a subpixel boundary and belongs to the boundary transition region; Specifically, the pixel-level residual coating region mask obtained by AI semantic segmentation is further converted into a sub-pixel-level residual coating region mask with continuous boundary transition values, so that subsequent 3D point cloud mapping no longer depends on integer pixel hard boundaries, but can retain the continuous membership information of the boundary region.

[0104] S4: Based on the set of boundary points of the two-dimensional sub-pixel residual coating and the mapping relationship from two-dimensional pixel coordinates to three-dimensional point cloud coordinates, obtain the three-dimensional spatial point coordinates corresponding to the boundary points of the two-dimensional sub-pixel residual coating, so as to obtain the set of boundary points of the three-dimensional sub-pixel residual coating, so as to transfer the two-dimensional sub-pixel boundary closure to the three-dimensional space, as the three-dimensional boundary output in the final detection result.

[0105] This embodiment converts the two-dimensional sub-pixel boundary position into a three-dimensional spatial boundary position, improving the accuracy of three-dimensional detection area boundary positioning; when the four adjacent integer pixels of the sub-pixel position all belong to the valid two-dimensional pixel coordinate set, that is:

[0106] in, It is a valid set of two-dimensional pixel coordinates.

[0107] When the above conditions are met:

[0108] In the formula: Subpixel coordinates The corresponding three-dimensional spatial coordinates; The fractional offset of the subpixel coordinates in the horizontal pixel direction satisfies ; The fractional offset of the sub-pixel coordinates in the vertical pixel direction satisfies ; pixel coordinates The corresponding three-dimensional spatial coordinates; Integer pixel coordinates The corresponding three-dimensional spatial coordinates; Integer pixel coordinates The corresponding three-dimensional spatial coordinates; Integer pixel coordinates The corresponding three-dimensional spatial coordinates.

[0109] Specifically, if among four adjacent integer pixels there exists one that does not belong to... For invalid pixels, instead of directly using the bilinear interpolation formula described above, effective 3D points are selected within the sub-pixel neighborhood for local planar interpolation or triangular interpolation to obtain... .

[0110] In this embodiment, the formula used to obtain the set of boundary points of the three-dimensional sub-pixel residual coating is as follows:

[0111] In the formula: This is the set of boundary points of the three-dimensional sub-pixel residual coating. For the first Subpixel-level boundary point coordinates The corresponding three-dimensional spatial coordinates; For the first The fractional offset of sub-pixel level boundary point coordinates in the horizontal pixel direction satisfies ; For the first The fractional offset of sub-pixel level boundary point coordinates in the vertical pixel direction satisfies ; For the smoothed first Sub-pixel level boundary point coordinates; For the first The integer part of the horizontal coordinates of sub-pixel level boundary points; For the first The integer part of the vertical coordinates of sub-pixel level boundary points; This is the set of boundary points of the two-dimensional sub-pixel residual coating.

[0112] In the formula: It is a mapping function from two-dimensional pixel coordinates to three-dimensional point coordinates; For the first Subpixel-level boundary point coordinates The corresponding effective depth value; This is the camera intrinsic parameter matrix; This is the rotation matrix between the camera coordinate system and the detection coordinate system; This is the translation vector between the camera coordinate system and the detection coordinate system.

[0113] Specifically, for integer pixel coordinates, the corresponding 3D point cloud coordinates are obtained directly based on the mapping relationship between 2D pixel coordinates and 3D spatial point coordinates established in step S1. For 2D subpixel boundary point coordinates, the 3D spatial coordinates corresponding to the subpixel boundary position are calculated using bilinear interpolation based on the 3D point cloud coordinates of its four adjacent integer pixels, thus obtaining the set of 3D subpixel residual coating boundary points.

[0114] S5: Based on the subpixel-level residual coating region mask, obtain the residual coating confidence of the 3D point cloud, so that each 3D point cloud point has a residual coating confidence attribute; to obtain the 3D point cloud residual coating label; In this embodiment, the mask value of the sub-pixel level residual coating area mask is assigned to the corresponding 3D point cloud point, serving as the residual coating confidence level for that 3D point cloud point. The formula used is as follows:

[0115] In the formula: For the first The confidence level that a 3D point cloud point belongs to the residual coating area; Masking subpixel residual coating areas in coordinates The value at; For the first The two-dimensional pixel coordinates corresponding to each three-dimensional point cloud point; The formula used to obtain the residual coating label of the 3D point cloud is as follows:

[0116] In the formula: For the first Residual coating label of a 3D point cloud point; The confidence threshold for residual coating in 3D point clouds; where, To indicate the first The three-dimensional point cloud points belong to the area of ​​residual coating; To indicate the first The three-dimensional point cloud points do not belong to the residual coating area.

[0117] Specifically, based on the confidence level of the residual coating in the 3D point cloud, and using a residual coating confidence threshold, the 3D point cloud points are labeled, and those meeting the threshold conditions are marked as residual coating points. The 3D point coordinates, residual coating confidence level, and 2D pixel index are then combined to form attributed 3D points.

[0118] S6: Based on the confidence level of the residual coating in the 3D point cloud, obtain attributed 3D points. Perform neighborhood consistency screening on the set of attributed 3D points to eliminate isolated false positives, obtaining a candidate point cloud set for the 3D residual coating. This unifies the point data representation in subsequent 3D point cloud screening, connectivity analysis, and region statistics, avoiding the mixing of 3D coordinate points and attributed points. Simultaneously, spatially isolated false positives are eliminated, improving the spatial continuity of the 3D residual coating detection area.

[0119] Preferably, S6 includes: S61: Obtain a 3D point with attributes, as shown below:

[0120] In the formula: For the first A three-dimensional point with attributes; For the first Spatial coordinates of one valid three-dimensional point; For the first The confidence level that a 3D point cloud point belongs to the residual coating area; For the first The two-dimensional pixel coordinates corresponding to each three-dimensional point cloud point; S62: Obtain a set of three-dimensional residual coating point clouds to obtain candidate point clouds of residual coating in three-dimensional space, as shown below:

[0121] In the formula: A collection of point clouds representing the residual coating in three dimensions; To preserve the 2D pixel index in the preprocessed 3D point cloud data; For the first Residual coating label of a 3D point cloud point; S63: Based on the neighborhood consistency screening formula for the three-dimensional residual coating point cloud, obtain the set of candidate point clouds for the three-dimensional residual coating after neighborhood consistency screening. The neighborhood consistency screening formula for the three-dimensional residual coating point cloud is expressed as follows:

[0122] In the formula: The set of candidate point clouds for 3D residual coatings after neighborhood consistency screening; The first point cloud in the three-dimensional residual coating set A three-dimensional point with attributes; For the first The spatial coordinates of a valid 3D point are also 3D points with attributes. Three-dimensional coordinates in; The neighborhood search radius; The threshold for the number of residual coating points within the neighborhood; All are indices of three-dimensional points; for and The Euclidean distance between them.

[0123] Among them, there is a one-to-one correspondence between valid 3D points and 3D points with attributes.

[0124] In this embodiment, the obtained sub-pixel-level residual coating region mask value is directly used as the residual coating confidence score of the 3D point cloud points, and then used as the attributed 3D points. A unified expression of three-dimensional spatial coordinates, two-dimensional pixel indexes, and residual coating confidence levels enables a continuous transmission relationship between two-dimensional AI detection results, sub-pixel boundary information, and three-dimensional spatial point cloud labels.

[0125] S7: Based on the set of candidate point clouds for 3D residual coating, divide the candidate point cloud for 3D residual coating into multiple connected regions to obtain multiple connected regions of 3D residual coating. This provides region objects for subsequent region area estimation, region confidence, and effective region screening. In turn, obtain the number of points in the connected regions of 3D residual coating, the estimated area of ​​the 3D residual coating region, and the average residual coating confidence to evaluate the detection reliability of each connected region of 3D residual coating.

[0126] Specifically, connectivity analysis is performed on the candidate point cloud set of 3D residual coatings to divide spatially continuous 3D points with attributes into different 3D residual coating connected regions. The formula for defining the connectivity relationship of 3D points with attributes is as follows:

[0127] If and only if there exists a sequence of three-dimensional points with attributes:

[0128] satisfy:

[0129] In the formula: , All are attributed 3D points from the candidate point cloud set of 3D residual coatings; This refers to the connectivity between 3D points with attributes. The set of candidate point clouds for 3D residual coatings after neighborhood consistency screening; For connection and The A three-dimensional point with intermediate attributes; For the connected path, the first An index of a 3D point with attributes; This is the index of the first sequence in the connected path; This refers to the index of the terminal sequence in the connected path; This is the connectivity distance threshold; 3D points with attributes 3D coordinates; The Euclidean distance between the corresponding 3D coordinates of adjacent 3D points with attributes; The set of connected regions of the three-dimensional residual coating is represented as follows:

[0130] In the formula: It is a set of connected regions of the three-dimensional residual coating; For the first A three-dimensional residual coating connected region; This represents the number of connected regions in the three-dimensional residual coating. The total number of connected regions of the three-dimensional residual coating; each All are composed of three-dimensional points with attributes that satisfy connectivity. composition.

[0131] Preferably, the formula used to obtain the average residual coating confidence level of the three-dimensional residual coating connected regions is as follows:

[0132] In the formula: For the first Average confidence level of residual coating in connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; For the first The number of attributed 3D points in a connected region of a 3D residual coating; For the first A three-dimensional point with attributes; For the first The confidence level that a 3D point cloud point belongs to the residual coating area.

[0133] Preferably, the method for obtaining the estimated area of ​​the three-dimensional residual coating region is as follows:

[0134] In the formula: For the first Area estimates of connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; For the first A three-dimensional point with attributes; For the first Local surface area weights corresponding to three-dimensional points with attributes; in,

[0135] In the formula: pixel coordinates The corresponding three-dimensional point coordinates; For the first The two-dimensional pixel coordinates corresponding to each 3D point cloud point are also 3D points with attributes. The corresponding two-dimensional pixel coordinates; The coordinates of the three-dimensional point corresponding to the horizontally effective neighboring pixel coordinates; The coordinates of the three-dimensional point corresponding to the effective neighboring pixel coordinates in the vertical direction; For vector cross product; The vector magnitude; The effective neighbor pixel coordinates are obtained by searching along the horizontal pixel direction; These are the coordinates of the effective neighboring pixels obtained by searching along the vertical pixel direction;

[0136]

[0137] In the formula: For Starting from the horizontal pixel direction, search for distance. A function of the nearest valid pixel coordinates; For Starting from the vertical pixel direction, search for distance. A function of the nearest valid pixel coordinates; It is a valid set of two-dimensional pixel coordinates.

[0138] Specifically, this embodiment obtains the coordinates of effective neighboring pixels to ensure that all neighboring pixels used in local area calculation have effective depth values ​​and effective 3D spatial points. By calculating the area weight of local 3D patches, local 3D patches are constructed using effective neighboring 3D points, providing point-level area weights for the area estimation of the 3D residual coating region. Finally, by accumulating the local surface area weights corresponding to each attributed 3D point within the connected region of the 3D residual coating, the area estimate of the connected region is obtained, which is used to determine whether the residual coating region has reached the effective detection area.

[0139] S8: Based on the estimated area of ​​the three-dimensional residual coating region and the average confidence level of the residual coating, the effective three-dimensional residual coating detection region screening formula is used to obtain the effective three-dimensional residual coating detection region set, so as to obtain the final detection result.

[0140] Preferably, S8 includes: The formula used to obtain the effective set of three-dimensional residual coating detection areas is as follows:

[0141] In the formula: This is a collection of effective three-dimensional residual coating detection areas; For the first Average confidence level of residual coating in connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; It is a set of connected regions of the three-dimensional residual coating; For the first Area estimates of connected regions of three-dimensional residual coating; The minimum effective detection area threshold; This is the minimum effective average confidence threshold.

[0142] Based on the set of effective three-dimensional residual coating detection areas, the residual coating detection judgment result is obtained as follows:

[0143] In the formula: The result of residual coating detection; The number of effective three-dimensional residual coating detection areas; among which, To indicate that an area with valid residual coating has been detected; This indicates areas where no effective residual coating was detected; Based on the residual coating detection results, the final detection results are obtained as follows:

[0144] In the formula: This is the final set of test results; pixel coordinates The probability that the area belongs to the residual coating area; A mask for pixel-level residual coating areas; Masking for subpixel-level residual coating areas; This is the set of boundary points of the three-dimensional sub-pixel residual coating. For the first Area estimates of connected regions of three-dimensional residual coating; For the first Average residual coating confidence level of each three-dimensional residual coating connected region.

[0145] In this embodiment, effective three-dimensional residual coating detection areas are selected based on the minimum effective area threshold and the minimum average confidence threshold. The number of effective three-dimensional residual coating detection areas is determined to be greater than zero, and the detection result indicating whether a residual coating area exists on the workpiece's outer surface is output. The final output detection result includes a two-dimensional residual coating probability map, a pixel-level residual coating area mask, a sub-pixel-level residual coating area mask, a set of three-dimensional sub-pixel residual coating boundary points, effective three-dimensional residual coating detection areas, an estimated effective detection area area, an average confidence score for the effective detection area, and a determination result indicating whether a residual coating area exists.

[0146] Specifically, this embodiment uses an effective three-dimensional residual coating detection area screening formula to eliminate false detection areas with too small an area estimate or too low confidence.

[0147] This embodiment takes the detection of residual coating after the repair of the outer surface of the aircraft skin as an example. First, an industrial-grade 3D camera is used to scan the outer surface of the aircraft skin to obtain the original two-dimensional image, the original depth map and the original three-dimensional point cloud data under the same field of view.

[0148] Then, the original 2D image is subjected to image denoising, brightness equalization, and color correction to obtain a preprocessed 2D image. Simultaneously, the original 3D point cloud data undergoes outlier removal, invalid point filtering, and coordinate unification to obtain geometrically preprocessed 3D point cloud data. Based on the depth map and calibration parameters output by the industrial-grade 3D camera, a mapping relationship between 2D pixel coordinates and 3D spatial coordinates is established, and a set of valid 2D pixel coordinates is selected. Based on this set of valid 2D pixel coordinates, preprocessed 3D point cloud data retaining the 2D pixel index is constructed.

[0149] Next, the preprocessed 2D image is input into the trained AI semantic segmentation model to obtain a 2D residual coating probability map. Based on the residual coating probability judgment threshold, the 2D residual coating probability map is converted into an initial binary residual coating mask. Then, morphological opening and closing operations are used to remove isolated regions and fill internal holes to obtain a pixel-level residual coating region mask.

[0150] Subsequently, a set of pixel-level boundary points is extracted from the pixel-level residual coating region mask, and the gray-level gradient information corresponding to the preprocessed 2D image is used to perform sub-pixel-level localization of the boundary points, resulting in a sub-pixel-level boundary point set. The sub-pixel-level boundary point set is then smoothed to obtain a 2D sub-pixel residual coating boundary point set, which is further used to generate continuous sub-pixel-level residual coating boundaries and sub-pixel-level residual coating region masks.

[0151] Then, the set of 2D subpixel residual coating boundary points is mapped to the 3D point cloud space using bilinear interpolation or local effective neighborhood interpolation to obtain the 3D subpixel residual coating boundary point set. Simultaneously, the subpixel-level residual coating region mask values ​​are mapped to the residual coating confidence scores of the 3D point cloud points, and 3D residual coating labels are generated based on the confidence score threshold. Subsequently, the 3D point coordinates, confidence scores, and 2D pixel indices are combined into attributed 3D points, and the resulting 3D residual coating point cloud set is obtained through filtering.

[0152] Finally, the neighborhood consistency screening, connected region division, region area estimation and average confidence calculation are performed on the three-dimensional residual coating point cloud set to screen out the effective three-dimensional residual coating detection areas, and output the effective detection areas, region area estimates, average confidence, three-dimensional sub-pixel boundaries and the determination results of whether there are residual coating areas.

[0153] This embodiment uses an AI semantic segmentation model to perform pixel-level recognition of residual coating areas in two-dimensional images. It can automatically identify residual coating areas on the outer surface of workpieces, reducing the reliance on operational experience for manual visual inspection. Compared with manual visual inspection and fixed color threshold detection, it can better adapt to different lighting conditions, different surface colors, different texture states, and different residual coating morphologies, thereby improving detection efficiency and consistency.

[0154] This embodiment, based on the pixel-level residual coating region mask output by AI semantic segmentation, calculates sub-pixel-level boundary points using grayscale gradient information near the residual coating boundary. It then generates a sub-pixel-level residual coating region mask through boundary smoothing and continuous curve fitting, thereby reducing pixel-level mask boundary jaggedness and quantization errors. This improves the residual coating region boundary positioning accuracy from pixel-level to sub-pixel level, enhancing the residual coating boundary detection accuracy.

[0155] This embodiment uses the calibration mapping relationship between two-dimensional pixel coordinates, depth map and three-dimensional point cloud coordinates to map the sub-pixel level residual coating area mask to the three-dimensional point cloud space, so as to map the two-dimensional residual coating detection result to the three-dimensional point cloud space. The spatial position of the residual coating area on the actual outer surface of the workpiece is obtained by sub-pixel position three-dimensional coordinate interpolation or local effective neighborhood interpolation, thereby improving the spatial positioning accuracy of the three-dimensional detection boundary.

[0156] In this embodiment, the subpixel-level residual coating area mask value is used as the residual coating confidence of the 3D point cloud points, and the 3D point coordinates, confidence, and 2D pixel index are combined into attributed 3D points. This makes the 3D detection results not only include residual coating labels, but also include confidence information that each 3D point belongs to the residual coating area, thereby improving the quantifiability and reliability of the detection results.

[0157] This invention performs neighborhood consistency screening and connectivity analysis on the marked 3D residual coating point cloud, eliminating isolated false detection points and false detection areas with too small an area estimate or too low confidence, and retaining the effective 3D residual coating detection areas that meet the spatial continuity, area conditions and confidence conditions. This can reduce the boundary offset and isolated false detection problems that occur when 2D detection results are directly mapped to 3D point clouds.

[0158] This embodiment maps a set of 2D subpixel residual coating boundary points to a 3D point cloud space using bilinear interpolation or local effective neighborhood interpolation, resulting in a 3D subpixel residual coating boundary point set. This allows the output of the 3D subpixel residual coating boundary, ensuring the detection result includes not only the region point cloud but also the fine boundary of the residual coating region in 3D space. Compared to simply outputting a 2D mask or ordinary 3D point cloud labels, this method more accurately describes the true boundary position of the residual coating region on complex curved surfaces. It is an automatic detection method for residual coating regions on external surfaces that combines 2D AI detection, subpixel boundary refinement, and 3D point cloud confidence labeling.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI-based automatic detection method for residual coating areas on an outer surface, characterized in that, The steps include the following: S1: Acquire the original two-dimensional image, original depth map, and original three-dimensional point cloud data of the workpiece under the same field of view, and preprocess the original two-dimensional image and original three-dimensional point cloud data to obtain the mapping relationship from two-dimensional pixel coordinates to three-dimensional point cloud coordinates; and obtain the effective set of two-dimensional pixel coordinates based on the original depth map. S2: Based on the preprocessed two-dimensional image, an AI semantic segmentation model with an encoder-decoder structure is used to obtain a two-dimensional residual coating probability map in order to obtain a pixel-level residual coating region mask. S3: Based on the pixel-level residual coating region mask and the preprocessed 2D image, obtain the sub-pixel-level boundary point set, then obtain the 2D sub-pixel residual coating boundary point set, and finally obtain the sub-pixel-level residual coating region mask. S4: Based on the two-dimensional sub-pixel residual coating boundary point set and the mapping relationship from two-dimensional pixel coordinates to three-dimensional point cloud coordinates, obtain the three-dimensional spatial point coordinates corresponding to the two-dimensional sub-pixel residual coating boundary points, so as to obtain the three-dimensional sub-pixel residual coating boundary point set. S5: Based on the sub-pixel level residual coating area mask, obtain the confidence level of the 3D point cloud residual coating to obtain the 3D point cloud residual coating label; S6: Based on the confidence of the residual coating in the 3D point cloud, obtain 3D points with attributes, and obtain a set of candidate point clouds for the 3D residual coating based on the residual coating label in the 3D point cloud; S7: Based on the set of candidate point clouds of 3D residual coating, obtain multiple connected regions of 3D residual coating, and then obtain the average confidence of residual coating and the area estimate of 3D residual coating region based on the set of effective 2D pixel coordinates. S8: Based on the estimated area of ​​the three-dimensional residual coating region and the average confidence level of the residual coating, obtain the effective set of three-dimensional residual coating detection regions to obtain the residual coating detection judgment result, and then obtain the final detection result to complete the AI ​​automatic detection of the residual coating region on the outer surface.

2. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 1, characterized in that, S3 includes: S31: Obtain the set of pixel-level boundary points of the pixel-level residual coating region based on the pixel-level residual coating region mask; S32: Based on the preprocessed two-dimensional image, obtain the corresponding grayscale image to obtain the local gradient magnitude in the grayscale image; S33: Obtain the sub-pixel level boundary point set based on the local gradient magnitude and pixel-level boundary point set in the grayscale image; S34: Perform local smoothing on the sub-pixel level boundary point set to obtain the smoothed sub-pixel level boundary point coordinates, so as to obtain the two-dimensional sub-pixel residual coating boundary point set. S35: Obtain the continuous sub-pixel level residual coating boundary based on the two-dimensional sub-pixel residual coating boundary point set; S36: Obtain the sub-pixel level residual coating area mask based on the continuous sub-pixel level residual coating boundary.

3. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 2, characterized in that, The formula used to obtain the local gradient magnitude in a grayscale image is as follows: In the formula: grayscale image in pixel coordinates Gradient magnitude at; This is a grayscale image obtained by converting a two-dimensional image. This represents the rate of grayscale change of a grayscale image along the horizontal pixel direction. This represents the rate of grayscale change of a grayscale image along the vertical pixel direction. All are pixel coordinates in a two-dimensional image; In the formula: This is a grayscale conversion operator; This is the preprocessed two-dimensional image.

4. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 3, characterized in that, The formula used to obtain sub-pixel boundary points is as follows: In the formula: For the first Sub-pixel level boundary point coordinates; A set of pixel-level boundary points The first in Coordinates of the boundary points; For the first pixel-level boundary points A local computational window centered on the target; To calculate the pixel coordinate vector within the window; To prevent the gradient summation denominator from being zero, a smoothing coefficient is used.

5. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 4, characterized in that, The formula used to obtain the mask for the sub-pixel level residual coating area is as follows: In the formula: Masking for subpixel-level residual coating areas; For pixels Located within the boundary of continuous subpixel level residual coating Internal area; This represents the total area of ​​a single pixel.

6. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 5, characterized in that, The formula used to obtain the confidence level of the residual coating in the 3D point cloud is as follows: In the formula: For the first The confidence level that a 3D point cloud point belongs to the residual coating area; Masking subpixel residual coating areas in coordinates The value at; For the first The two-dimensional pixel coordinates corresponding to each three-dimensional point cloud point; The formula used to obtain the residual coating label of the 3D point cloud is as follows: In the formula: For the first Residual coating label of a 3D point cloud point; The confidence threshold for residual coating in 3D point clouds; where, To indicate the first The three-dimensional point cloud points belong to the area of ​​residual coating; To indicate the first The three-dimensional point cloud points do not belong to the residual coating area.

7. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 6, characterized in that, S6 includes: S61: Obtain a 3D point with attributes, as shown below: In the formula: For the first A three-dimensional point with attributes; For the first Spatial coordinates of one valid three-dimensional point; For the first The confidence level that a 3D point cloud point belongs to the residual coating area; For the first The two-dimensional pixel coordinates corresponding to each three-dimensional point cloud point; S62: Obtain the three-dimensional residual coating point cloud set, represented as follows: In the formula: A collection of point clouds representing the residual coating in three dimensions; To preserve the 2D pixel index in the preprocessed 3D point cloud data; For the first Residual coating label of a 3D point cloud point; S63: Based on the neighborhood consistency screening formula for the three-dimensional residual coating point cloud, obtain the set of candidate point clouds for the three-dimensional residual coating after neighborhood consistency screening. The neighborhood consistency screening formula for the three-dimensional residual coating point cloud is expressed as follows: In the formula: The set of candidate point clouds for 3D residual coatings after neighborhood consistency screening; The first point cloud in the three-dimensional residual coating set A three-dimensional point with attributes; For the first The spatial coordinates of a valid 3D point are also 3D points with attributes. Three-dimensional coordinates in; The neighborhood search radius; The threshold for the number of residual coating points within the neighborhood; All are indices of three-dimensional points; for and The Euclidean distance between them.

8. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 7, characterized in that, The formula used to obtain the average residual coating confidence level of the three-dimensional residual coating connected regions is as follows: In the formula: For the first Average confidence level of residual coating in connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; For the first The number of attributed 3D points in a connected region of a 3D residual coating; For the first A three-dimensional point with attributes; For the first The confidence level that a 3D point cloud point belongs to the residual coating area.

9. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 8, characterized in that, The formula used to obtain the estimated area of ​​the three-dimensional residual coating region is as follows: In the formula: For the first Area estimates of connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; For the first A three-dimensional point with attributes; For the first Local surface area weights corresponding to three-dimensional points with attributes; in, In the formula: pixel coordinates The corresponding three-dimensional point coordinates; For the first The two-dimensional pixel coordinates corresponding to each 3D point cloud point are also 3D points with attributes. The corresponding two-dimensional pixel coordinates; The coordinates of the three-dimensional point corresponding to the horizontally effective neighboring pixel coordinates; The coordinates of the three-dimensional point corresponding to the effective neighboring pixel coordinates in the vertical direction; For vector cross product; The vector magnitude; The effective neighbor pixel coordinates are obtained by searching along the horizontal pixel direction; These are the coordinates of the effective neighboring pixels obtained by searching along the vertical pixel direction; In the formula: For Starting from the horizontal pixel direction, search for distance. A function of the nearest valid pixel coordinates; For Starting from the vertical pixel direction, search for distance. A function of the nearest valid pixel coordinates; It is a valid set of two-dimensional pixel coordinates.

10. The AI ​​automatic detection method for residual coating areas on an outer surface according to claim 9, characterized in that, S8 includes: The formula used to obtain the effective set of three-dimensional residual coating detection areas is as follows: In the formula: This is a collection of effective three-dimensional residual coating detection areas; For the first Average confidence level of residual coating in connected regions of three-dimensional residual coating; For the first A three-dimensional residual coating connected region; It is a set of connected regions of the three-dimensional residual coating; For the first Area estimates of connected regions of three-dimensional residual coating; The minimum effective detection area threshold; The minimum effective average confidence threshold; Based on the set of effective three-dimensional residual coating detection areas, the residual coating detection judgment result is obtained as follows: In the formula: The result of residual coating detection; The number of effective three-dimensional residual coating detection areas; among which, To indicate that an area with valid residual coating has been detected; This indicates areas where no effective residual coating was detected; Based on the residual coating detection results, the final detection results are obtained as follows: In the formula: This is the final set of test results; pixel coordinates The probability that the area belongs to the residual coating area; A mask for pixel-level residual coating areas; Masking for subpixel-level residual coating areas; This is the set of boundary points of the three-dimensional sub-pixel residual coating.

Images