Vehicle body appearance spraying defect detection method based on deep learning

Abstract

The invention discloses a vehicle body appearance spraying defect detection method based on deep learning, and relates to the technical field of computer vision detection, and the method comprises the steps: carrying out the coordinate mapping of a detection image according to a calibration parameter set, generating a standardized image, carrying out the illumination color unified processing through combining a vehicle body coordinate mask, and generating a spraying unified standard image, executing data enhancement processing to generate an enhanced training sample set; training a spraying defect segmentation model based on the enhanced training sample set and the defect mark data set, and generating a defect region probability distribution diagram and a defect candidate region set by taking a spraying unified standard image and a vehicle body coordinate mask as input; and geometric features and optical features of the defect candidate region set are calculated, spraying defect types are judged, and a spraying defect feature table is generated. According to the method, the response precision of the spraying defect segmentation model in the aspects of brightness balance and edge identification is optimized through a loss function of reflection comparison and boundary consistency constraint.

Description

Technical Field

[0001] This invention relates to the field of computer vision inspection technology, and in particular to a method for detecting defects in vehicle exterior coating based on deep learning. Background Technology

[0002] In the automotive manufacturing industry, the quality of the vehicle's exterior paint coating is a crucial indicator of overall appearance consistency and manufacturing precision. With the development of industrial automation and intelligent manufacturing, paint coating inspection is gradually shifting from manual visual inspection to machine vision inspection. The application of deep learning technology in industrial vision inspection is continuously expanding. Through convolutional neural networks, multi-layer extraction of paint surface features and defect identification are achieved, improving the intelligence level and recognition efficiency of the inspection process. Existing methods typically combine camera calibration, image enhancement, and defect annotation data to achieve automatic extraction and classification of paint surface features, and are widely used in modern automotive paint coating inspection.

[0003] However, existing methods for detecting coating defects still have two limitations under complex working conditions. On the one hand, during the vehicle body coating inspection process, changes in lighting environment and differences in camera angle can lead to uneven distribution of image brightness and color, affecting the ability of deep learning models to recognize coating reflection features. On the other hand, existing detection results are mostly focused on the extraction of defect areas, lacking comprehensive analysis of the geometric and optical features of defects, making it difficult to accurately determine the type of coating defect and describe its multi-dimensional features. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a deep learning-based method for detecting defects in vehicle exterior coating, which solves the problems of uneven lighting affecting recognition accuracy and inaccurate defect type judgment in coating inspection.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a deep learning-based method for detecting defects in vehicle exterior painting, comprising: performing coordinate mapping on the detected image according to a calibration parameter set to generate a standardized image; performing unified lighting and color processing in conjunction with a vehicle coordinate mask to generate a unified standard painting image; and performing data augmentation processing to generate an enhanced training sample set.

[0008] A spraying defect segmentation model is trained based on an enhanced training sample set and a defect labeling dataset. The model takes a standardized spraying image and a vehicle body coordinate mask as input to generate a probability distribution map of defect regions and a set of defect candidate regions.

[0009] Calculate the geometric and optical features of the defect candidate region set, determine the type of spraying defect, and generate a spraying defect feature table;

[0010] Based on the calibration parameter set, the geometric center coordinates of the defects in the paint defect feature table are converted into spatial coordinates of the defects in the vehicle body three-dimensional coordinate system, generating spatial description data of paint defects.

[0011] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, the generation of standardized images includes acquiring detection images of the vehicle body after painting is completed by using an industrial camera installed on the top of the painting inspection station.

[0012] The calibration procedure is used to obtain the intrinsic and extrinsic parameters of the industrial camera, as well as the pose parameters from the spraying inspection station to the car body, and these parameters are combined to form a calibration parameter set.

[0013] Based on the calibration parameter set, coordinate mapping is performed on the detection images acquired by the industrial camera. The position of each pixel in the detection images acquired by the industrial camera in the industrial camera coordinate system is converted into the spatial position in the vehicle body three-dimensional coordinate system, generating a standardized image in the vehicle body three-dimensional coordinate system.

[0014] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, the step of generating a unified standard painting image by performing unified lighting and color processing in conjunction with the vehicle coordinate mask includes determining the spatial distribution range of the vehicle exterior surface based on the standard surface coordinate set of the vehicle body, and generating a vehicle coordinate mask within the spatial distribution range.

[0015] Within the vehicle body surface area defined by the vehicle body coordinate mask, brightness alignment and tone correction are performed on the standardized image to generate a uniform standard image for spraying.

[0016] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, the generation of the enhanced training sample set includes: scaling and translating the uniform standard image of the painting in the three-dimensional coordinate system of the vehicle body to maintain the same spatial pose as the uniform standard image of the painting, generating the painting enhanced image, and forming the enhanced training sample set with all the painting enhanced images.

[0017] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, the step of training the painting defect segmentation model based on the enhanced training sample set and the defect label dataset includes supervising the training of a preset deep learning model in the three-dimensional coordinate system of the vehicle body based on the enhanced training sample set and the defect label dataset.

[0018] The weight and bias parameters of the convolutional neural network segmentation model are optimized by backpropagation and gradient descent algorithms, and the training process is constrained by reflection contrast and boundary consistency loss functions to obtain the spraying defect segmentation model.

[0019] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, wherein: the generation of the defect region probability distribution map and the defect candidate region set includes inputting the unified standard image of painting and the vehicle body coordinate mask into the painting defect segmentation model to generate a defect region probability distribution map with the same size as the unified standard image of painting;

[0020] Within the vehicle surface area defined by the vehicle coordinate mask, the probability levels are traversed and the difference modulus between the predicted defect area and the brightness contrast deviation area is calculated. The probability level corresponding to the smallest difference modulus is taken as the probability threshold.

[0021] Binarization is performed on the probability distribution map of the defect area based on the probability threshold to generate an initial binary image;

[0022] Within the vehicle body surface area defined by the vehicle body coordinate mask, connectivity analysis is performed on the initial binary image to identify connected regions, calculate the area of ​​connected regions and the geometric center coordinates of defects, delete small noise regions, obtain a set of defect candidate regions, and record the geometric center coordinates of each defect candidate region.

[0023] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, the calculation of geometric and optical features of the defect candidate region set includes: calculating geometric features for each defect candidate region in the defect candidate region set based on the defect candidate region set to obtain the defect area, defect boundary length and shape ratio.

[0024] Based on the unified standard image of the spray coating and the reference surface image, the optical characteristics of each defect candidate region in the defect candidate region set are calculated to obtain the brightness deviation and brightness fluctuation.

[0025] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, the painting defect type determination includes determining the painting defect type of each candidate defect region based on a multi-dimensional feature combination of defect area, defect boundary length, shape ratio, brightness deviation and brightness fluctuation.

[0026] When the defect area is the minimum value in the defect candidate region set and the defect boundary length is the minimum value in the defect candidate region set, it is identified as a pinhole type spraying defect.

[0027] When the shape ratio reaches its maximum value in the defect candidate region set, and the defect boundary length reaches its maximum value in the defect candidate region set, it is identified as a sagging spray defect.

[0028] When the brightness fluctuation reaches its maximum value in the defect candidate region set, and the brightness deviation reaches its maximum value in the defect candidate region set, it is identified as a bubble-type spraying defect.

[0029] When the same defect candidate area meets the judgment conditions of multiple spraying defect types at the same time, it shall be judged in the order of pinhole spraying defects, sagging spraying defects, and bubble spraying defects.

[0030] Associate the spraying defect type with the corresponding defect candidate area and record it.

[0031] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, wherein: the generation of the painting defect feature table includes recording the geometric features, optical features, geometric center coordinates of the defect and the painting defect type of each defect candidate region as a set of feature data;

[0032] All feature data are summarized to generate a spraying defect feature table.

[0033] As a preferred embodiment of the deep learning-based vehicle exterior painting defect detection method of the present invention, wherein: the generation of painting defect spatial description data includes, according to the industrial camera internal parameters, industrial camera external parameters and pose parameters from the painting detection station to the vehicle body in the calibration parameter set, the defect geometric center coordinates of each defect candidate region in the painting defect feature table are converted from the pixel coordinate system of the unified standard image of the painting to the defect spatial coordinates in the three-dimensional coordinate system of the vehicle body through spatial back projection calculation;

[0034] By mapping the spatial coordinates of the defects to the geometric features, optical features, and types of spraying defects recorded in the spraying defect feature table, spatial description data of the spraying defects is generated.

[0035] The beneficial effects of this invention are as follows: by using a loss function constrained by reflection contrast and boundary consistency, the response accuracy of the spraying defect segmentation model in terms of brightness balance and edge recognition is optimized; by combining the geometric and optical features of the defect area for multi-dimensional analysis, the accurate judgment of the spraying defect type and the quantitative expression of the spraying defect features are achieved. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of a deep learning-based method for detecting defects in vehicle exterior paint coating.

[0038] Figure 2 A flowchart for generating an enhanced training sample set.

[0039] Figure 3 The flowchart describes the process of training a spray coating defect segmentation model and generating a set of defect candidate regions.

[0040] Figure 4 This is a flowchart for determining the type of coating defect. Detailed Implementation

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0043] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0044] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for detecting defects in vehicle exterior painting based on deep learning, including the following steps:

[0045] S1. Based on the calibration parameter set, perform coordinate mapping on the detected image to generate a standardized image. Combine the vehicle body coordinate mask to perform unified lighting and color processing to generate a unified standard image for spraying. Then, perform data augmentation processing to generate an enhanced training sample set.

[0046] Furthermore, an industrial camera installed at the top of the paint spraying inspection station captures inspection images of the car body after the paint spraying is completed.

[0047] The calibration program obtains the intrinsic and extrinsic parameters of the industrial camera, as well as the pose parameters from the spraying inspection station to the vehicle body.

[0048] The intrinsic parameters of the industrial camera, the extrinsic parameters of the industrial camera, and the pose parameters from the spraying inspection station to the vehicle body are combined to form a calibration parameter set. The calibration parameter set is used to describe the spatial projection mapping relationship from the industrial camera imaging coordinate system to the vehicle body three-dimensional coordinate system.

[0049] It should be noted that the internal and external parameters of an industrial camera include lens focal length, principal point coordinates, and lens distortion coefficient. These are optical imaging characteristic parameters inherent to the industrial camera and are measured through calibration procedures during the factory or equipment installation phase.

[0050] The external parameters of the industrial camera are its three-dimensional position and orientation angle in the coordinate system of the spraying inspection station, representing the spatial positional relationship between the industrial camera coordinate system and the coordinate system of the spraying inspection station.

[0051] The pose parameters from the painting inspection station to the vehicle body refer to the rotation matrix and translation vector between the coordinate system of the painting inspection station and the three-dimensional coordinate system of the vehicle body, representing the actual spatial posture of the vehicle body in the painting inspection station.

[0052] The calibration procedure is a one-time spatial mapping relationship establishment process. When executing the calibration procedure, a calibration target with regular geometric features (such as a checkerboard or dot array) is placed in the spraying inspection station. Images of the calibration target at different positions and angles are captured by an industrial camera. The correspondence between the imaging coordinates and actual spatial coordinates of the feature points of the calibration target is extracted using computer vision algorithms, and the intrinsic and extrinsic parameters of the industrial camera are calculated. Subsequently, the vehicle body is placed in the calibration reference position, and the rotation and translation parameters from the coordinate system of the spraying inspection station to the three-dimensional coordinate system of the vehicle body are obtained through a laser positioning device or a three-dimensional measurement device, thus obtaining the pose parameters from the spraying inspection station to the vehicle body.

[0053] Furthermore, coordinate mapping is performed on the detection images acquired by the industrial camera based on the calibration parameter set. Specifically, by using the industrial camera intrinsic parameters, industrial camera extrinsic parameters, and pose parameters from the spraying inspection station to the vehicle body in the calibration parameter set, the position of each pixel in the detection image acquired by the industrial camera in the industrial camera coordinate system is converted into the spatial position in the vehicle body three-dimensional coordinate system, generating a standardized image in the vehicle body three-dimensional coordinate system, denoted as the standardized image.

[0054] Read the set of standard surface coordinates of the vehicle body, and determine the spatial distribution range of the vehicle body's outer surface in the three-dimensional coordinate system based on the spatial coordinate points of the vehicle body's outer surface in the set of standard surface coordinates.

[0055] Within the spatial distribution range, the spatial coordinate points of the vehicle's outer surface are projected onto a standardized image in the vehicle's three-dimensional coordinate system. The value is set to one at the projection position corresponding to the spatial coordinate point of the vehicle's outer surface, and to zero at the non-projection position, thus generating a vehicle coordinate mask.

[0056] The vehicle coordinate mask represents the spatial distinction between the outer surface area of ​​the vehicle body and the background area. It takes a value of one in the vehicle surface area and a value of zero in the background area.

[0057] After completing coordinate mapping and generating the vehicle body coordinate mask, the acquisition time of the detection image is recorded, and the spatial pose of the vehicle body at the painting inspection station is saved to obtain temporal pose information.

[0058] It should be noted that the standard surface coordinate set of the vehicle body is a set of standard coordinate data established based on the three-dimensional design model of the vehicle body during the production preparation stage, which records the reference coordinates of each spatial position on the outer surface of the vehicle body.

[0059] The set of standard surface coordinates for the vehicle body is generated before deployment on the production line and stored in the data server at the painting inspection station.

[0060] Furthermore, within the vehicle body surface area defined by the vehicle body coordinate mask, the standardized image undergoes uniform lighting and color processing to generate a uniform standard image for spraying.

[0061] Among them, the vehicle body surface area defined by the vehicle body coordinate mask refers to the continuous area of ​​the vehicle body surface area with a value of one, which corresponds to the projection range of the standard surface coordinate set of the vehicle body on the standardized image.

[0062] Lighting color unification processing includes brightness alignment and hue correction.

[0063] Brightness alignment adjusts the overall brightness of the normalized image to match that of the reference surface image by comparing the differences in brightness distribution between the normalized image and the reference surface image.

[0064] Color correction adjusts the color distribution of the normalized image to match that of the reference surface image by comparing the color differences in each color channel between the normalized image and the reference surface image.

[0065] The reference surface image is a standard vehicle body exterior surface image collected at the spraying inspection station during the production line calibration stage, used to characterize the brightness and color features of the vehicle body surface under normal spraying conditions.

[0066] The lighting color uniformity process performs brightness alignment and hue correction sequentially, with the result of brightness alignment serving as the input for hue correction.

[0067] The uniform processing of lighting color is only performed within the vehicle surface area defined by the vehicle coordinate mask; non-vehicle areas are not processed.

[0068] The standardized image after unifying the lighting color is recorded as the spraying unified standard image.

[0069] Furthermore, the system reads the standardized spraying image, the vehicle body coordinate mask, the reference surface image, and the temporal attitude information, performs data augmentation operations in the vehicle body's three-dimensional coordinate system, and generates an augmentation training sample set.

[0070] Data augmentation operations include scaling and translation.

[0071] Scaling allows for slight magnification or reduction (e.g., within 5%) of the uniform standard image being painted in the vehicle's three-dimensional coordinate system; translation allows for slight coordinate movement (e.g., within three pixels) along the horizontal or vertical axis in the vehicle's three-dimensional coordinate system.

[0072] Scaling and translation are performed only within the vehicle surface area defined by the vehicle coordinate mask. When scaling or translation causes blank areas to appear at the edges of the uniform standard image, they are filled from the corresponding positions in the reference surface image to maintain boundary continuity and vehicle surface integrity.

[0073] The standardized image of the spray coating after data augmentation is denoted as the spray coating augmentation image.

[0074] Temporal attitude information is used to determine the spatial reference position of the paint enhancement image in the vehicle body's three-dimensional coordinate system, so that the paint enhancement image and the unified standard paint image are consistent in spatial attitude.

[0075] By repeatedly scaling and translating the standardized spraying image, multiple spraying enhancement images are generated, and all the spraying enhancement images constitute an enhancement training sample set.

[0076] S2. Train the painting defect segmentation model based on the enhanced training sample set and the defect label dataset. Take the unified standard image of painting and the vehicle body coordinate mask as input to generate a probability distribution map of defect areas and a set of defect candidate areas.

[0077] Furthermore, based on the enhanced training sample set and defect label dataset, a pre-set deep learning model is trained in the three-dimensional coordinate system of the vehicle body; after training, a painting defect segmentation model is obtained.

[0078] By using a standardized image and vehicle body coordinate mask for spraying, spraying defect identification is performed, generating a probability distribution map of defect areas and candidate defect areas.

[0079] It should be noted that the defect labeling dataset is a collection of defect area labeling data formed during the historical painting process, and the label coordinates correspond pixel-by-pixel with the enhanced training sample set in the vehicle body's three-dimensional coordinate system.

[0080] The pre-defined deep learning model is known in advance, containing only the basic deep learning model framework and parameters, and needs to be trained using an augmented training sample set. The example uses a convolutional neural network (CNN) segmentation model, which receives spray-enhanced images from the augmented training sample set as input and outputs a probability distribution map of defect regions of the same size as the input. The CNN segmentation model structure includes convolutional layers, downsampling layers, upsampling layers, and jumper layers, with initial parameters being the weight and bias parameters of the CNN segmentation model. During the training phase, the parameters of the CNN segmentation model are updated through supervised learning backpropagation and gradient descent algorithms, gradually satisfying the convergence condition for the optimization objective of the CNN segmentation model. After training, a spray-defect segmentation model is obtained.

[0081] Furthermore, during the training phase, the sprayed enhanced image is used as the input sample for the convolutional neural network segmentation model, and the defect label dataset is used as the supervision signal. The convolutional neural network segmentation model obtains the local texture, reflection differences, and fine particle features of the sprayed surface through convolution and feature extraction operations.

[0082] The backpropagation algorithm is used to calculate the propagation relationship of the error between the output of the convolutional neural network segmentation model and the defect labeling data in each network layer. The partial derivatives of the convolutional kernel weight parameters and bias parameters of the convolutional neural network segmentation model with respect to the loss function are obtained layer by layer, thereby obtaining the gradient information of the parameters of the convolutional neural network segmentation model.

[0083] The gradient descent algorithm controls the direction and magnitude of parameter updates for the convolutional neural network segmentation model based on gradient information. It continuously adjusts the weight and bias parameters of the convolutional neural network segmentation model along the descent direction of the loss function, so that the loss function value gradually decreases and tends to stabilize until the convolutional neural network segmentation model meets the convergence condition.

[0084] During training, conventional algorithms such as stochastic gradient descent or adaptive moment estimation can be used to optimize the weight and bias parameters of the convolutional neural network segmentation model in order to improve the stability and convergence speed of training.

[0085] To enable the spraying defect segmentation model to more stably identify spraying defects under conditions of uniform illumination and consistent boundaries, a loss function based on reflection contrast and boundary consistency is used during the training phase, expressed as:

[0086] ;

[0087] in, This represents the loss function of the spray coating defect segmentation model. This represents the area of ​​the vehicle body surface defined by the vehicle body coordinate mask. This represents the horizontal coordinate of the plane representing the uniform standard image of the spray painting. This represents the longitudinal coordinate of the plane representing the uniform standard image of the spraying process. This indicates that the convolutional neural network segmentation model is in coordinates The output defect probability, This indicates that the defect labeling dataset is in coordinates The defect indication (1 for defects, 0 for non-defects). Factors representing reflection contrast and boundary consistency. This represents a very small positive constant, used to avoid the denominator being zero.

[0088] The factors for reflection contrast and boundary consistency are calculated jointly by brightness contrast deviation and boundary uncertainty, and are expressed as follows:

[0089] ;

[0090] ;

[0091] ;

[0092] in, This indicates the coordinate system between the uniform standard image for spraying and the reference surface image. The normalized amount of brightness contrast deviation. This indicates that the uniform standard image for spraying is in coordinates. The normalized measure of the boundary uncertainty, This indicates that the uniform standard image for spraying is in coordinates. brightness value, Indicates the reference surface image in coordinates brightness value, This indicates the average brightness of the uniform standard image applied to the vehicle body surface area. This indicates that the uniform standard image for spraying is in coordinates. gradient magnitude, This represents the standard deviation of the gradient amplitude of the uniform standard image applied to the vehicle body surface area.

[0093] By employing loss functions that combine reflection contrast and boundary consistency, the convolutional neural network segmentation model simultaneously optimizes defect prediction accuracy and illumination / edge consistency during training, making the learning process more aligned with the visual characteristics of the sprayed surface.

[0094] Furthermore, when the change in the loss function of the convolutional neural network segmentation model tends to stabilize after several consecutive training rounds, the training process terminates, and the spraying defect segmentation model is obtained.

[0095] The convergence state is determined by calculating the fluctuation range of the loss function across consecutive training rounds, and is expressed as:

[0096] ;

[0097] in, This represents a measure of the volatility of the loss function during the training phase. Indicates the first The loss function value of the wheel, This represents the average loss function value over the most recent rounds. Indicates the length of the statistical window.

[0098] when When the convolutional neural network segmentation model remains stable and no longer changes significantly over several consecutive training rounds, it is determined that the model has reached a stable convergence state, and the spraying defect segmentation model is then output.

[0099] Furthermore, during the inference phase, a standardized image of the spray coating is input into the spray coating defect segmentation model.

[0100] The spraying defect segmentation model extracts multi-scale feature maps from the unified standard image of spraying through convolutional layers, and generates an output feature map with the same size as the unified standard image of spraying through feature fusion operations of downsampling and upsampling layers.

[0101] In the output feature map, the output value at each pixel location is mapped to an interval using the Sigmoid activation function. , representing the probability of a coating defect at the corresponding location; the defect probabilities of all pixel locations are reconstructed into a two-dimensional probability matrix according to spatial coordinates, and the two-dimensional probability matrix is ​​the probability distribution map of the defect area.

[0102] Within the vehicle surface area defined by the vehicle coordinate mask, probability threshold calculation and defect area extraction operations are performed.

[0103] Probability threshold calculation refers to calculating the probability threshold within the interval The algorithm iterates through different probability levels. For each probability level, it calculates the difference modulus between the predicted defect area and the brightness contrast deviation area within the vehicle body coordinate mask region. It records the difference modulus for each probability level and takes the probability level with the smallest difference modulus as the probability threshold, expressed as:

[0104] ;

[0105] in, Indicates the probability threshold. This represents the probability level, and the probability level value is represented by a range. A continuously varying candidate probability threshold is used to determine whether a pixel output by the spraying defect segmentation model belongs to a defect pixel. This indicates an indicator function that takes the value 1 when the condition within the parentheses is true, and 0 otherwise.

[0106] The predicted defect area represents the probability distribution of all defects in the defect region output by the coating defect segmentation model at different probability levels. The result obtained by statistically analyzing the area of ​​pixels with a probability level greater than or equal to the probability level.

[0107] Brightness contrast deviation area refers to the integral of the brightness contrast deviation between the sprayed uniform standard image and the reference surface image.

[0108] The defect region extraction operation refers to performing binarization processing on the probability distribution map of the defect region according to the probability threshold to generate an initial binary image. In the initial binary image, when the defect probability value of a pixel is greater than or equal to the probability threshold, the pixel value is set to one; when the defect probability value of a pixel is less than the probability threshold, the pixel value is set to zero. In the initial binary image, the pixel with a value of one is called a defect pixel.

[0109] Within the vehicle surface area defined by the vehicle coordinate mask, connectivity analysis is performed on the initial binary image. Specifically, when two defect pixels are adjacent in the horizontal, vertical, or diagonal directions, it is determined that these two defect pixels belong to the same connected pixel set. By traversing all defect pixels in the initial binary image, all independent connected pixel sets are identified in turn, and each connected pixel set is denoted as a connected region.

[0110] For each connected region, calculate the area of ​​the connected region and the coordinates of the geometric center of the defect. The area of ​​the connected region is determined by counting the number of defective pixels in the connected region, and the coordinates of the geometric center of the defect are obtained by averaging the coordinates of all defective pixels in the connected region. For connected regions whose area is less than a preset number of pixels (e.g., 50 pixels), they are identified as noise regions and deleted.

[0111] After connectivity analysis and noise region removal, all remaining connected regions form a set of defect candidate regions, and the geometric center coordinates of the defect corresponding to each defect candidate region are retained.

[0112] The set of candidate defect regions and the geometric center coordinates of the defects are used together to characterize the spatial location of possible painting defects on the vehicle body surface.

[0113] S3. Calculate the geometric and optical features of the defect candidate region set, determine the type of spraying defect, and generate a spraying defect feature table.

[0114] Furthermore, in the three-dimensional coordinate system of the vehicle body, geometric and optical features are calculated for each candidate defect region.

[0115] Geometric features include defect area, defect boundary length, and shape ratio.

[0116] The defect area represents the effective connected region area of ​​the defect candidate region, used to characterize the actual proportion of spraying defects in the unified standard image of spraying; the defect boundary length is determined by the number of outer boundary pixels of the detected defect candidate region; the shape ratio is obtained by the aspect ratio of the outer rectangle of the defect candidate region, the width of the outer rectangle is the difference between the maximum and minimum horizontal coordinates of the defect pixels in the defect candidate region, and the length of the outer rectangle is the difference between the maximum and minimum vertical coordinates of the defects in the defect candidate region.

[0117] Optical characteristics include brightness deviation and brightness fluctuation.

[0118] Brightness deviation represents the average brightness difference between the uniform standard image and the reference surface image within the defect candidate region, expressed as:

[0119] ;

[0120] in, Indicates brightness deviation. Indicates the candidate region for defects. This indicates the number of defective pixels within the defect candidate region.

[0121] Brightness fluctuations are determined by calculating the standard deviation of brightness within the defect candidate region of the uniform standard image of the sprayed coating, expressed as:

[0122] ;

[0123] in, Indicates brightness fluctuation. This represents the average brightness of the uniform standard image during spraying within the defect candidate area.

[0124] The defect area, defect boundary length, shape ratio, brightness deviation, brightness fluctuation, and geometric center coordinates of each defect candidate region are recorded as a set of feature data.

[0125] All feature data are summarized to generate a spraying defect feature table.

[0126] Furthermore, the spraying defect type is determined based on the multi-dimensional feature combination in the spraying defect feature table, and the spraying defect type corresponding to each defect candidate area is recorded in the spraying defect feature table.

[0127] When the defect area is the minimum value in the defect candidate region set, and the defect boundary length is the minimum value in the defect candidate region set, it is identified as a pinhole type spraying defect.

[0128] When the shape ratio reaches its maximum value in the defect candidate region set, and the defect boundary length reaches its maximum value in the defect candidate region set, it is identified as a sagging spray defect.

[0129] When the brightness fluctuation reaches its maximum value in the defect candidate region set, and the brightness deviation reaches its maximum value in the defect candidate region set, it is identified as a bubble-type spraying defect.

[0130] It should also be noted that when the same candidate defect area meets the judgment conditions of multiple spraying defect types at the same time, the priority judgment shall be made in the order of pinhole spraying defects, sagging spraying defects, and bubble spraying defects.

[0131] Among pinhole-type spray defects, the candidate region with the smallest defect area is selected as the pinhole-type spray defect; among sagging-type spray defects, the candidate region with the largest shape ratio is selected as the sagging-type spray defect; among bubble-type spray defects, the candidate region with the largest brightness fluctuation is selected as the bubble-type spray defect.

[0132] S4. Based on the calibration parameter set, convert the geometric center coordinates of the defects in the paint defect feature table into the spatial coordinates of the defects in the three-dimensional coordinate system of the vehicle body, and generate the spatial description data of the paint defects.

[0133] Furthermore, in the vehicle body's three-dimensional coordinate system, spatial mapping is performed on the geometric center coordinates of the defects in each candidate region of the paint defect feature table.

[0134] Based on the calibration parameter set, the geometric center coordinates of each defect candidate region in the paint defect feature table are transformed from the pixel coordinate system of the unified standard image of the paint to the three-dimensional coordinate system of the vehicle body to obtain the defect spatial coordinates. The defect spatial coordinates are used to represent the actual spatial position of the defect on the outer surface of the vehicle body.

[0135] During the spatial mapping process, a one-to-one correspondence is established from the internal parameters of the industrial camera, the external parameters of the industrial camera, and the pose parameters from the spraying inspection station to the vehicle body in the calibration parameter set. Then, the three-dimensional positioning of the geometric center coordinates of the defect is completed through spatial back projection calculation.

[0136] Among them, spatial back-projection calculation is a conventional technique in this field. It is used to reverse-map the coordinates of pixels in the unified standard image of the spraying to the spatial coordinates of defects in the three-dimensional coordinate system of the vehicle body based on the intrinsic parameters of the industrial camera, the extrinsic parameters of the industrial camera, and the pose parameters from the spraying inspection station to the vehicle body, so as to restore the actual spatial position of the defects on the outer surface of the vehicle body. Its implementation method is a conventional technique in this field and will not be described in detail.

[0137] For example, the inverse transformation method of perspective projection matrix or the inverse projection algorithm of pinhole imaging model can be used. That is, the direction of pixel ray is calculated based on the internal and external parameters of the industrial camera, and the equation of the outer surface of the car body is determined by combining the pose parameters from the spraying inspection station to the car body. The intersection point of the pixel ray and the outer surface of the car body is then used as the spatial coordinates of the defect.

[0138] Furthermore, the defect area, defect boundary length, shape ratio, brightness deviation, brightness fluctuation, and spraying defect type recorded in the spraying defect feature table are mapped to the defect spatial coordinates to generate spraying defect spatial description data.

[0139] The spatial description data of spraying defects includes the spatial coordinates of the defects, the type of spraying defects, and geometric and optical features.

[0140] Spatial description data of spraying defects is recorded in the data server of the spraying inspection station, serving as the basic data for spraying quality assessment and spraying defect repair and location.

[0141] The terminal interface of the painting inspection station displays spatial description data of painting defects, and shows the correspondence between the outer surface of the vehicle body and the spatial location of the painting defects through the three-dimensional coordinate system of the vehicle body.

[0142] The display interface uses the three-dimensional outline of the vehicle's outer surface as a background. Different colors are used to display the spatial coordinates of the defects according to the type of paint defect. For example, pinhole defects are marked in red, sagging defects are marked in yellow, and bubble defects are marked in blue.

[0143] In the display interface, by selecting the coordinate points of the defect space, the defect area, shape ratio, brightness deviation and brightness fluctuation can be read, and a defect space description data file can be output for subsequent spraying defect repair or quality statistical analysis.

[0144] In summary, this invention optimizes the response accuracy of the spraying defect segmentation model in terms of brightness balance and edge recognition by using a loss function based on reflection contrast and boundary consistency constraints; and achieves accurate judgment of spraying defect types and quantitative expression of spraying defect features by combining the geometric and optical features of the defect area for multi-dimensional analysis.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting defects in vehicle exterior coating based on deep learning, characterized in that: include, Based on the calibration parameter set, coordinate mapping is performed on the detected image to generate a standardized image. Combined with the vehicle body coordinate mask, lighting and color unification processing is performed to generate a unified standard image for spraying. Data augmentation processing is then performed to generate an enhanced training sample set. A spraying defect segmentation model is trained based on an enhanced training sample set and a defect labeling dataset. The model takes a standardized spraying image and a vehicle body coordinate mask as input to generate a probability distribution map of defect regions and a set of defect candidate regions. Calculate the geometric and optical features of the defect candidate region set, determine the type of spraying defect, and generate a spraying defect feature table; Based on the calibration parameter set, the geometric center coordinates of the defects in the paint defect feature table are converted into spatial coordinates of the defects in the vehicle body three-dimensional coordinate system, generating spatial description data of paint defects.

2. The method for detecting defects in vehicle exterior painting based on deep learning as described in claim 1, characterized in that: The generation of standardized images includes acquiring inspection images of the car body after the painting is completed using an industrial camera installed on top of the painting inspection station; The calibration procedure is used to obtain the intrinsic and extrinsic parameters of the industrial camera, as well as the pose parameters from the spraying inspection station to the car body, and these parameters are combined to form a calibration parameter set. Based on the calibration parameter set, coordinate mapping is performed on the detection images acquired by the industrial camera. The position of each pixel in the detection images acquired by the industrial camera in the industrial camera coordinate system is converted into the spatial position in the vehicle body three-dimensional coordinate system, generating a standardized image in the vehicle body three-dimensional coordinate system.

3. The method for detecting defects in vehicle exterior painting based on deep learning as described in claim 2, characterized in that: The step of combining the vehicle body coordinate mask to perform unified lighting and color processing to generate a unified standard image for spraying includes determining the spatial distribution range of the outer surface of the vehicle body based on the standard surface coordinate set of the vehicle body, and generating a vehicle body coordinate mask within the spatial distribution range. Within the vehicle body surface area defined by the vehicle body coordinate mask, brightness alignment and tone correction are performed on the standardized image to generate a uniform standard image for spraying.

4. The method for detecting defects in vehicle exterior painting based on deep learning as described in claim 3, characterized in that: The process of generating the enhanced training sample set includes scaling and translating the paint uniform standard image in the vehicle body three-dimensional coordinate system to maintain the same spatial orientation as the paint uniform standard image, generating an enhanced paint image, and then using all the enhanced paint images to form the enhanced training sample set.

5. The deep learning-based method for detecting defects in vehicle exterior painting as described in claim 4, characterized in that: The training of the spraying defect segmentation model based on the enhanced training sample set and the defect label dataset includes supervising the training of a preset deep learning model in the three-dimensional coordinate system of the vehicle body based on the enhanced training sample set and the defect label dataset. The weight and bias parameters of the convolutional neural network segmentation model are optimized by backpropagation and gradient descent algorithms, and the training process is constrained by reflection contrast and boundary consistency loss functions to obtain the spraying defect segmentation model.

6. The method for detecting defects in vehicle exterior painting based on deep learning as described in claim 5, characterized in that: The generation of the defect region probability distribution map and the defect candidate region set includes inputting the unified standard image of spraying and the vehicle body coordinate mask into the spraying defect segmentation model to generate a defect region probability distribution map with the same size as the unified standard image of spraying. Within the vehicle surface area defined by the vehicle coordinate mask, the probability levels are traversed and the difference modulus between the predicted defect area and the brightness contrast deviation area is calculated. The probability level corresponding to the smallest difference modulus is taken as the probability threshold. Binarization is performed on the probability distribution map of the defect area based on the probability threshold to generate an initial binary image; Within the vehicle body surface area defined by the vehicle body coordinate mask, connectivity analysis is performed on the initial binary image to identify connected regions, calculate the area of ​​connected regions and the geometric center coordinates of defects, delete small noise regions, obtain a set of defect candidate regions, and record the geometric center coordinates of each defect candidate region.

7. The deep learning-based method for detecting defects in vehicle exterior painting as described in claim 6, characterized in that: The calculation of geometric and optical features of the defect candidate region set includes calculating geometric features for each defect candidate region in the defect candidate region set based on the defect candidate region set, to obtain the defect area, defect boundary length and shape ratio. Based on the unified standard image of the spray coating and the reference surface image, the optical characteristics of each defect candidate region in the defect candidate region set are calculated to obtain the brightness deviation and brightness fluctuation.

8. The method for detecting defects in vehicle exterior painting based on deep learning as described in claim 7, characterized in that: The determination of the coating defect type includes determining the coating defect type for each candidate region based on a multi-dimensional feature combination of defect area, defect boundary length, shape ratio, brightness deviation, and brightness fluctuation. When the defect area is the minimum value in the defect candidate region set and the defect boundary length is the minimum value in the defect candidate region set, it is identified as a pinhole type spraying defect. When the shape ratio reaches its maximum value in the defect candidate region set, and the defect boundary length reaches its maximum value in the defect candidate region set, it is identified as a sagging spray defect. When the brightness fluctuation reaches its maximum value in the defect candidate region set, and the brightness deviation reaches its maximum value in the defect candidate region set, it is identified as a bubble-type spraying defect. When the same defect candidate area meets the judgment conditions of multiple spraying defect types at the same time, it shall be judged in the order of pinhole spraying defects, sagging spraying defects, and bubble spraying defects. Associate the spraying defect type with the corresponding defect candidate area and record it.

9. The method for detecting defects in vehicle exterior painting based on deep learning as described in claim 8, characterized in that: The generation of the spraying defect feature table includes recording the geometric features, optical features, geometric center coordinates of the defect and the type of spraying defect of each defect candidate region as a set of feature data. All feature data are summarized to generate a spraying defect feature table.

10. The method for detecting defects in vehicle exterior painting based on deep learning as described in claim 9, characterized in that: The generation of spraying defect spatial description data includes, based on the industrial camera internal parameters, industrial camera external parameters and the pose parameters from the spraying inspection station to the vehicle body in the calibration parameter set, the defect geometric center coordinates of each defect candidate area in the spraying defect feature table are converted from the pixel coordinate system of the unified standard image of spraying to the defect spatial coordinates in the three-dimensional coordinate system of the vehicle body through spatial back projection calculation. By mapping the spatial coordinates of the defects to the geometric features, optical features, and types of spraying defects recorded in the spraying defect feature table, spatial description data of the spraying defects is generated.

Images