Motorcycle paint surface bubble sagging defect real-time detection system based on image analysis

Abstract

The invention relates to the technical field of image analysis, in particular to a motorcycle paint surface bubble sagging defect real-time detection system based on image analysis. The method comprises the following steps: firstly, acquiring a plurality of images of a to-be-detected paint surface under different illumination conditions by an image acquisition and processing unit, and preprocessing the images to obtain a preprocessed multi-illumination image; then, a three-dimensional information analysis unit analyzes and calculates a surface normal vector of each pixel point of the paint surface according to brightness information of the same pixel point in the preprocessed multi-illumination image, and generates a surface normal graph for representing three-dimensional direction information of the paint surface; then, a candidate area positioning unit analyzes a gradient field of the surface normal diagram and positions bubble and sagging defect candidate areas respectively; and finally, the feature extraction and classification unit extracts fusion features of the defect candidate region, and outputs a detection result obtained by recognition of the classifier through an output unit. According to the invention, the accuracy of paint bubble sagging defect detection can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a real-time detection system for motorcycle paint bubble and sagging defects based on image analysis. Background Art

[0002] In motorcycle manufacturing, paint bubbles and sagging are key defects that affect product grade. Currently, automated inspection in the industry relies primarily on traditional two-dimensional image analysis technology, but it faces analytical bottlenecks when processing highly reflective curved surfaces on motorcycles. Specifically, 1) Existing systems primarily use conventional image analysis algorithms based on fixed thresholds, edge detection, and spot analysis. However, the unique high light reflections and complex curvature of motorcycle paint surfaces create artifacts on two-dimensional images that closely resemble defect features. This makes it impossible to effectively distinguish between real bubbles and separate the surface's inherent light and shadow streaks from linear sagging defects, resulting in a high false alarm rate.

[0003] 2) Traditional image analysis methods rely primarily on low-dimensional information such as area, aspect ratio, and grayscale value when extracting features from two-dimensional images. This feature description is severely inadequate for defects such as bubbles and sags, which are essentially three-dimensional morphological anomalies. It also fails to quantify the degree of convexity or drape of the defect, making it difficult to accurately identify tiny defects with diverse morphologies and low contrast.

[0004] Therefore, a real-time detection system for air bubble and sagging defects on motorcycle paint surfaces based on image analysis is proposed. Summary of the Invention

[0005] The object of the present invention is to provide a real-time detection system for motorcycle paint bubble sag defects based on image analysis, so as to improve the accuracy of paint bubble sag defect detection.

[0006] To achieve the above object, the present invention provides the following technical solutions: The real-time detection system for motorcycle paint bubble and sagging defects based on image analysis includes: An image acquisition and processing unit is used to acquire multiple images of the paint surface to be inspected under different lighting conditions and preprocess them to obtain preprocessed multi-lighting images; a three-dimensional information analysis unit, configured to analyze and calculate the surface normal vector of each pixel of the paint surface based on the brightness information of the same pixel in the preprocessed multi-illumination image, thereby generating a surface normal map for representing the three-dimensional directional information of the paint surface; a candidate region positioning unit, configured to analyze the gradient field of the surface normal map and locate regions where the normal vector direction undergoes a local mutation as defect candidate regions; A feature extraction and classification unit is used to extract the fusion features of the defect candidate area and identify it using a classifier to obtain a detection result of the defect candidate area; An output unit is used to output the detection result.

[0007] Furthermore, the process of preprocessing the multiple paint surface images under different lighting conditions by the image acquisition and processing unit includes: Performing image registration on a plurality of paint surface images acquired under different lighting conditions to obtain a first image; performing image denoising on the first image to obtain a second image; Perform ROI extraction on the second image to obtain the preprocessed multi-illumination image.

[0008] Furthermore, the three-dimensional information parsing unit parses and calculates the surface normal vector of each pixel of the paint surface based on the brightness information of the same pixel in the preprocessed multi-illumination image, thereby generating a surface normal map for representing the three-dimensional directional information of the paint surface. The process includes: For each pixel in the paint surface area in the preprocessed multi-illumination image, a system of linear equations is established based on a Lambertian reflectance model, relating the brightness value of the pixel under multiple illumination directions to the surface normal vector and reflectivity; The linear equations are expressed as a matrix, and the matrix equations are solved using the least squares method to calculate the vector corresponding to each pixel; The solved vector is normalized to obtain the unit surface normal vector of all pixels, and its three components are stored in a three-channel data matrix corresponding to the image size to generate the final surface normal map.

[0009] Furthermore, the process of positioning the candidate area of the bubble defect candidate area by the candidate area positioning unit includes: Calculate the partial derivatives of the first two channels of the surface normal map in the x and y directions respectively, and then synthesize the gradient amplitude of each pixel to generate the corresponding gradient amplitude map; Applying a Gaussian-Laplacian operator to perform convolution processing on the gradient magnitude map to obtain a response map; The response image processed by the Gaussian-Laplacian operator is subjected to binary threshold segmentation to identify the areas where the response value is higher than the preset threshold, forming multiple independent connected domains. These connected domains are identified as candidate areas for bubble defects.

[0010] Furthermore, the process of the candidate area positioning unit positioning the candidate area of the sagging defect includes: Calculating the directional gradient of the surface normal map along a preset direction to generate a corresponding directional gradient map; Performing Hessian matrix analysis on the directional gradient map, and describing the local curvature information of the image through the second-order partial derivatives; By screening pixels that meet the eigenvalue conditions, areas with continuous and linear features are identified; the screening results are then optimized using morphological operations, and the connected areas finally extracted are identified as candidate areas for sagging defects.

[0011] Furthermore, the feature extraction and classification unit extracts the fusion features of the defect candidate area and uses a classifier to identify it, and the process of obtaining the detection result of the defect candidate area includes: Extracting 2D geometric features and 3D topographic features from the binary mask used for ROI extraction and the surface normal map used for defect candidate region location respectively; Inputting the features obtained by fusing the two-dimensional geometric features with the three-dimensional topographic features into a pre-trained classifier for processing to obtain a corresponding confidence score; A grading process is performed according to the confidence scores to obtain a detection result.

[0012] Furthermore, the two-dimensional geometric features include: area, aspect ratio, roundness and principal axis direction; the three-dimensional morphological features include: average normal deviation, surface curvature and gradient energy.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. Based on the Lambertian reflectance model, a set of linear equations is established using the brightness changes of pixels under multiple lighting conditions, which can accurately analyze the surface normal vector of each pixel on the paint surface. By reconstructing the three-dimensional direction of the surface using the principle of photometric stereo vision, the microscopic morphology of the paint surface can be more accurately characterized. The unit normal map generated by normalization not only reduces the data complexity of subsequent processing, but also facilitates image analysis under different lighting conditions. The surface normal map provides a reliable three-dimensional geometric information basis for subsequent defect location, which can effectively distinguish paint defects from normal surface fluctuations, thereby improving the accuracy of bubble and sagging defect detection on motorcycle paint surfaces.

[0014] 2. By calculating the gradient amplitude of the surface normal map and combining it with the Gaussian-Laplacian operator for convolution processing and binary threshold segmentation, candidate areas for bubble defects are located. By calculating the directional gradient of the surface normal map and combining it with Hessian matrix analysis and morphological operations, candidate areas for sag defects are located. This not only enhances the edge response of the defect area and ensures the accurate division of the candidate area, but also meets the high efficiency requirements of real-time detection on the motorcycle paint production line, thereby improving the accuracy of motorcycle paint bubble and sag defect detection.

[0015] 3. By extracting two-dimensional geometric features and three-dimensional morphological features from the binary mask and surface normal map respectively, the fused features are input into the pre-trained classifier, and the detection results are output based on the confidence score level. By fusing two-dimensional geometric features and three-dimensional morphological features, the morphological and spatial characteristics of the defects can be fully characterized, thereby improving the classifier's ability to distinguish between bubbles and sagging defects. At the same time, the pre-trained classifier has optimized parameters during the training stage, and the inference process has low computational complexity, making it suitable for real-time detection. The fusion of two-dimensional and three-dimensional features can improve the accuracy of bubble and sagging defect detection on motorcycle paint surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the structure of the real-time detection system for motorcycle paint bubble and sagging defects based on image analysis of the present invention; Figure 2 Schematic diagram of the process of obtaining defect candidate area detection results of the present invention; Figure 3 The present invention is a flowchart of a method for implementing a real-time detection system for motorcycle paint bubble and sagging defects based on image analysis. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] See also Figures 1 to 3 The present invention provides a real-time detection system for motorcycle paint bubble and sagging defects based on image analysis. The technical solution is as follows: Example 1:

[0019] In order to improve the accuracy and efficiency of detecting air bubble and sagging defects on motorcycle paint surfaces, a company used the real-time detection system for air bubble and sagging defects on motorcycle paint surfaces based on image analysis proposed by the present invention. The structure of the system is as follows: Figure 1 Shown, including: An image acquisition and processing unit is used to acquire multiple images of the paint surface to be inspected under different lighting conditions and preprocess them to obtain preprocessed multi-lighting images; Furthermore, the shooting conditions for multi-lighting images were set as follows: Four high-brightness white LED light sources were symmetrically arranged in a circular array at 0°, 90°, 180°, and 270°, centered around the fixed industrial camera lens. Each light source was angled 45° from the camera's optical axis, ensuring that light could illuminate the curved surface from different angles, creating distinct light and shadow variations. Furthermore, the process of pre-processing the multiple paint surface images under different lighting conditions by the image acquisition and processing unit includes: Performing image registration on a plurality of paint surface images acquired under different lighting conditions to obtain a first image; Performing image denoising on the first image to obtain a second image; Performing ROI extraction on the second image to obtain a preprocessed multi-illumination image; Furthermore, because slight vibrations in the production line or slight movement of the workpiece can cause sub-pixel spatial misalignment between multiple images, the Enhanced Correlation Coefficient (ECC) algorithm uses the first captured image as a fixed reference benchmark to align the remaining images. Furthermore, image denoising can use bilateral filtering or non-local mean denoising algorithm to remove random noise generated during image acquisition and improve the signal-to-noise ratio of the image; Furthermore, in order to meet the needs of real-time processing and improve processing speed, ROI extraction technology is used to concentrate computing resources on the effective motorcycle paint area, eliminating interference from the background and non-detection areas; the process includes: using the pre-trained Mask R-CNN model to segment the image and obtain the probability value of each pixel belonging to the ROI; using the Otsu algorithm to automatically find the optimal global threshold and compare it with the probability value to generate a binary mask image to identify the paint area, that is, 1 represents the ROI and 0 represents the background and non-detection area; the binary mask image is applied to each image to obtain a preprocessed multi-light image.

[0020] By performing image preprocessing before 3D information analysis, including image registration, image denoising, and ROI extraction, various physical and noise interferences can be avoided, ensuring that the data is as close as possible to the ideal model, thereby ensuring the accuracy and stability of the 3D information analysis results and further improving the accuracy of motorcycle paint bubble and sag defect detection.

[0021] A three-dimensional information analysis unit is used to analyze and calculate the surface normal vector of each pixel of the paint surface based on the brightness information of the same pixel in the preprocessed multi-illumination image, thereby generating a surface normal map for representing the three-dimensional directional information of the paint surface; Furthermore, the three-dimensional information analysis unit analyzes and calculates the surface normal vector of each pixel of the paint surface based on the brightness information of the same pixel in the preprocessed multi-illumination image, thereby generating a surface normal map for representing the three-dimensional directional information of the paint surface. The process includes: For each pixel within the paint surface area in the preprocessed multi-illumination image, a set of linear equations is established based on the Lambertian reflectance model, relating the brightness value of the pixel under multiple illumination directions to the surface normal vector and reflectivity. The linear equations are expressed as a matrix, and the least squares method is used to solve the matrix equations to calculate the vector corresponding to each pixel; Normalize the solved vector to obtain the unit surface normal vector of all pixels, and store its three components in a three-channel data matrix corresponding to the image size to generate the final surface normal map; Furthermore, the Lambertian reflectance model can be described as follows: the brightness value received by the camera sensor at a pixel is related to the surface reflectivity, light source direction vector, and surface normal vector of that pixel; the brightness value and light source direction vector are known, while the surface reflectivity and surface normal vector are unknown; the surface reflectivity is a scalar that represents the brightness of the material itself; Furthermore, the preprocessed multi-illumination image is substituted into the Lambertian reflectance model for calculation. For any pixel point within the ROI, due to the presence of four light sources in different directions, four equations can be listed according to the Lambertian model to construct a linear equation system. An intermediate vector is defined, and the normalized result of the intermediate vector is the surface normal vector, and the modulus of the intermediate vector is the reflectivity. The linear equation system is rewritten into a matrix form using the intermediate vector, that is, a 4×1 known brightness value vector can be expressed as the product of a 4×3 known light source direction vector matrix and a 3×1 intermediate vector to be solved. The optimal intermediate vector is obtained using the least squares method, and it is normalized to obtain the unit surface normal vector. Furthermore, in order to further optimize the surface normal map and avoid the reduced sensitivity of subsequent defect detection due to errors introduced by noise or uneven lighting, a convolutional neural network was used to post-process the surface normal map; the collected annotated paint image dataset, including the surface normal map and known defect areas, was used to train the convolutional neural network. The model input was the generated initial surface normal map, and the output was the optimized normal map.

[0022] By combining the Lambertian reflectance model and the principle of photometric stereo vision to generate a surface normal map, a reliable three-dimensional geometric information basis can be provided for subsequent defect location, effectively distinguishing paint defects from normal surface fluctuations, thereby improving the accuracy of motorcycle paint bubble and sagging defect detection.

[0023] The candidate area positioning unit is used to analyze the gradient field of the surface normal map and locate the area where the normal vector direction has a local mutation as the defect candidate area; Furthermore, the candidate area positioning unit can perform corresponding defect candidate area positioning operations according to user requirements, including: performing defect candidate area positioning for bubble sag at the same time, or performing any type of defect candidate area positioning; Furthermore, the process of locating the candidate area of the bubble defect by the candidate area positioning unit includes: Calculate the partial derivatives of the first two channels of the surface normal map in the x and y directions respectively, and then synthesize the gradient amplitude of each pixel to generate the corresponding gradient amplitude map; Apply the Gaussian-Laplacian operator to the gradient magnitude map for convolution to obtain the response map; The response image processed by the Gaussian-Laplacian operator is subjected to binary threshold segmentation to identify the areas where the response value is higher than the preset threshold, forming multiple independent connected domains. These connected domains are then identified as candidate areas for bubble defects. Furthermore, the Sobel operator is used to calculate the partial derivatives of the first two channels of the surface normal map in the x and y directions, respectively, which reflects the intensity of changes in the horizontal and vertical directions of different components of the surface normal map; Furthermore, the comprehensive gradient magnitude of each pixel is obtained by combining the partial derivatives of different components in the horizontal and vertical directions, thereby generating the corresponding gradient magnitude map; Furthermore, the gradient amplitude map is smoothed using a Gaussian filter. The degree of smoothing is controlled by the standard deviation of the Gaussian kernel, which can be flexibly adjusted according to the roughness of the paint surface. The smoothed image is then Laplace processed to generate a response map. Positive values in the response map represent raised areas, such as bubbles, while negative values represent sunken areas. Furthermore, the preset threshold is an adaptive threshold determined dynamically based on the global characteristics of the response graph by utilizing the Otsu algorithm; By calculating the gradient amplitude of the normal map, the local normal direction mutation caused by bubble defects can be captured; the convolution processing of the gradient amplitude map by the Gaussian-Laplacian operator further enhances the edge response of the defect area and highlights the circular or quasi-circular characteristics of bubble-type defects. At the same time, the Gaussian-Laplacian operator has the function of smoothing noise, which can reduce the false detection rate in complex industrial environments.

[0024] Furthermore, the process of the candidate area positioning unit positioning the candidate area of the sagging defect includes: Calculate the directional gradient of the surface normal map along the preset direction and generate the corresponding directional gradient map; Perform Hessian matrix analysis on the directional gradient map and describe the local curvature information of the image through the second-order partial derivative; By screening pixels that meet the eigenvalue conditions, regions with continuous and linear features are identified. Morphological operations are then used to optimize the screening results, and the connected regions finally extracted are identified as candidate regions for sagging defects. Furthermore, the directional gradient of the first two components of the surface normal map is calculated along a preset direction; and a corresponding directional gradient map is generated according to the directional gradient value of each pixel point; the preset direction can be the y direction or other directions; Furthermore, for each pixel of the directional gradient map, the second-order partial derivative is calculated and the Hessian matrix is constructed. The second-order partial derivative is calculated using the central difference operator. The Hessian matrix is composed of three elements: the rate of change in the horizontal direction, the rate of change in the vertical direction, and the cross-rate of change between the horizontal and vertical directions. For each pixel of the Hessian matrix, its two eigenvalues are calculated. Furthermore, the linear features of sagging defects are characterized by one feature with a large eigenvalue and another with a small value approximately equal to 0; therefore, this is used as a screening condition. Dilation and erosion operations are applied to the image after eigenvalue screening to enhance connectivity and smooth boundaries. Then, the region connection method is used to extract optimized independent regions, each of which is considered a candidate region for sagging defects. Furthermore, in a complex paint surface environment, single eigenvalue screening may be affected by noise interference. In order to further improve the ability to distinguish candidate areas of sag defects, a machine learning model, such as a support vector machine or a random forest, is used. The auxiliary features and eigenvalues extracted from the directional gradient map are used as model input to output the classification results. The classification results of the model are used as the final identified candidate areas. Among them, the auxiliary features extracted from the directional gradient map can be local gradient energy or curvature distribution.

[0025] By calculating the directional gradient of the surface normal map and combining Hessian matrix analysis and morphological operations, candidate areas for sag defects are located. Hessian matrix analysis describes the local curvature through second-order partial derivatives, accurately capturing the linear features of sag defects. Morphological operations optimize the boundaries of connected areas, reduce interference in fragmented areas, and improve positioning accuracy.

[0026] The feature extraction and classification unit is used to extract the fusion features of the defect candidate area and use the classifier to identify it to obtain the detection results of the defect candidate area; Furthermore, the feature extraction and classification unit extracts the fusion features of the defect candidate area and uses the classifier to identify it, and the process of obtaining the detection result of the defect candidate area is as follows: Figure 2 As shown, the details are as follows: Extracting 2D geometric features and 3D topographic features from the binary mask used for ROI extraction and the surface normal map used for defect candidate region location respectively; The features obtained by fusing the two-dimensional geometric features with the three-dimensional morphological features are input into a pre-trained classifier for processing to obtain the corresponding confidence score; Perform grading processing according to the confidence score to obtain the test results; Furthermore, the two-dimensional geometric features include: area, aspect ratio, roundness and principal axis direction; the three-dimensional topographic features include: average normal deviation, surface curvature and gradient energy; Furthermore, the binary mask used for ROI extraction is further optimized into a binary mask of the defect candidate area. The binary mask of the defect candidate area is then analyzed by calling functions in the OpenCV library to obtain two-dimensional geometric features. The binary mask of the defect candidate area and the three-channel surface normal map are combined to calculate the average normal deviation based on the angle between the average normal vector of all pixels in the candidate area and the normal vector of the pixel itself. The surface normal map is then operated to obtain the surface curvature and gradient energy respectively. Furthermore, the classifier uses a pre-trained LightGBM model, takes the fused feature vector obtained by splicing the 2D geometric features and the 3D topographic features as the model input, outputs a confidence score vector to represent the probability that the candidate region belongs to each category, and selects the highest probability value as the confidence score for this judgment; Furthermore, the high confidence threshold is set to 0.95 and the medium confidence threshold is set to 0.75. The confidence score is compared with the high confidence threshold and the medium confidence threshold. If it is higher than the high confidence threshold, it is confirmed as a defect. If it is within the range of the high confidence threshold and the medium confidence threshold, it is determined to be a suspected defect. If it is lower than the medium confidence threshold, it is determined to be a non-defect. Furthermore, in order to improve the classification accuracy and generalization of the LightGBM model, on the one hand, additional texture features and statistical features are introduced into the model input, combined with the existing two-dimensional geometric features and three-dimensional morphological features, to enrich the information content of the fusion feature vector and enhance the distinguishing ability of the classifier; on the other hand, the transfer learning technology is used to pre-train the LightGBM model based on the public industrial defect detection dataset to learn the common defect features, and then a motorcycle paint defect dataset is collected to fine-tune the model parameters and optimize the classification performance of bubbles and sagging defects.

[0027] The following are the test results for three different samples to be tested, designated as Sample 1, Sample 2, and Sample 3. The motorcycle paint samples to be tested come from the same manufacturer but have different production dates. The image acquisition and processing unit of the present invention is used to obtain preprocessed multi-lighting images of each sample. The three-dimensional information analysis unit and the candidate area positioning unit are then used in sequence to obtain a surface normal map of each sample in the defect candidate area. The two-dimensional geometric features and three-dimensional morphological features extracted from the binary mask and the surface normal map are then input into the feature extraction and classification unit for detection, resulting in the defect detection results for each group, as shown in Table 1.

[0028] Table 1 Defect detection results

[0029] By fusing two-dimensional geometric features with three-dimensional morphological features, the morphological and spatial characteristics of defects can be fully characterized; multi-dimensional feature fusion can improve the classifier's ability to distinguish between bubbles and sagging defects; the pre-trained classifier has optimized parameters during the training phase, and the inference process has low computational complexity, making it suitable for real-time detection; at the same time, grading processing is implemented according to the confidence score output by the classifier, which can achieve refined management in motorcycle paint defect detection; the fusion of two-dimensional and three-dimensional features can improve the accuracy of motorcycle paint bubble and sagging defect detection.

[0030] The output unit is used to output the detection results.

[0031] Furthermore, the output unit can output the defect detection results through charts, text boxes, voice, etc.

[0032] This embodiment proposes a real-time detection system for motorcycle paint bubble and sag defects based on image analysis. First, the image acquisition and processing unit acquires multiple images of the paint surface to be inspected under different lighting conditions and preprocesses them to obtain preprocessed multi-lighting images. Then, the three-dimensional information analysis unit analyzes and calculates the surface normal vector of each pixel of the paint surface based on the brightness information of the same pixel point in the preprocessed multi-lighting images, generating a surface normal map for characterizing the three-dimensional directional information of the paint surface. Next, the candidate area positioning unit analyzes the gradient field of the surface normal map and locates the candidate defect areas of bubbles and sags respectively. Finally, the feature extraction and classification unit extracts the fusion features of the defect candidate areas and outputs the detection results obtained by the classifier through the output unit. The present invention can improve the accuracy of paint bubble and sag defect detection.

[0033] Example 2: As an embodiment of the present invention, refer to Figure 3 The method for implementing a real-time detection system for motorcycle paint bubble and sagging defects based on image analysis includes: Acquire multiple images of the paint surface to be inspected under different lighting conditions, and preprocess them to obtain preprocessed multi-lighting images; Based on the brightness information of the same pixel in the preprocessed multi-illumination image, the surface normal vector of each pixel on the paint surface is analyzed and calculated, thereby generating a surface normal map used to represent the three-dimensional directional information of the paint surface; Furthermore, the process of generating the surface normal map includes: For each pixel within the paint surface area in the preprocessed multi-illumination image, a set of linear equations is established based on the Lambertian reflectance model, relating the brightness value of the pixel under multiple illumination directions to the surface normal vector and reflectivity. The linear equations are expressed as a matrix, and the least squares method is used to solve the matrix equations to calculate the vector corresponding to each pixel; The solved vector is normalized to obtain the unit surface normal vector of all pixels, and its three components are stored in a three-channel data matrix corresponding to the image size to generate the final surface normal map.

[0034] Analyzing the gradient field of the surface normal map and locating the region where the normal vector direction has a local mutation as a defect candidate region; Furthermore, the process of locating the candidate area of bubble defects includes: Calculate the partial derivatives of the first two channels of the surface normal map in the x and y directions respectively, and then synthesize the gradient amplitude of each pixel to generate the corresponding gradient amplitude map; Apply the Gaussian-Laplacian operator to the gradient magnitude map for convolution to obtain the response map; The response image processed by the Gaussian-Laplacian operator is subjected to binary threshold segmentation to identify the areas where the response value is higher than the preset threshold, forming multiple independent connected domains. These connected domains are then identified as candidate areas for bubble defects. Furthermore, the process of locating the candidate area of sag defects includes: Calculate the directional gradient of the surface normal map along the preset direction and generate the corresponding directional gradient map; Perform Hessian matrix analysis on the directional gradient map and describe the local curvature information of the image through the second-order partial derivative; By screening pixels that meet the eigenvalue conditions, areas with continuous and linear features are identified; the screening results are then optimized using morphological operations, and the connected areas finally extracted are identified as candidate areas for sagging defects.

[0035] The fusion features of the defect candidate area are extracted and identified using a classifier to obtain the detection results of the defect candidate area.

[0036] Furthermore, the process of obtaining the defect candidate area detection results includes: Extracting the two-dimensional geometric features and three-dimensional morphological features of the defect candidate area from the binary mask used for extraction and the surface normal map used for locating the defect candidate area respectively; The features obtained by fusing the two-dimensional geometric features with the three-dimensional morphological features are input into a pre-trained classifier for processing to obtain the corresponding confidence score; Grading processing is performed according to the confidence scores to obtain the detection results.

[0037] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A real-time detection system for motorcycle paint bubble and sagging defects based on image analysis, characterized by: include: An image acquisition and processing unit is used to acquire multiple images of the paint surface to be inspected under different lighting conditions and preprocess them to obtain preprocessed multi-lighting images; a three-dimensional information analysis unit, configured to analyze and calculate the surface normal vector of each pixel of the paint surface based on the brightness information of the same pixel in the preprocessed multi-illumination image, thereby generating a surface normal map for representing the three-dimensional directional information of the paint surface; a candidate region positioning unit, configured to analyze the gradient field of the surface normal map and locate regions where the normal vector direction undergoes a local mutation as defect candidate regions; A feature extraction and classification unit is used to extract the fusion features of the defect candidate area and identify it using a classifier to obtain a detection result of the defect candidate area; An output unit is used to output the detection result.

2. The real-time detection system for motorcycle paint bubble and sagging defects based on image analysis according to claim 1 is characterized in that: The process of preprocessing the multiple paint surface images under different lighting conditions by the image acquisition and processing unit includes: Performing image registration on a plurality of paint surface images acquired under different lighting conditions to obtain a first image; performing image denoising on the first image to obtain a second image; Perform ROI extraction on the second image to obtain the preprocessed multi-illumination image.

3. The real-time detection system for motorcycle paint bubble and sagging defects based on image analysis according to claim 1 is characterized in that: The three-dimensional information parsing unit parses and calculates the surface normal vector of each pixel of the paint surface based on the brightness information of the same pixel in the preprocessed multi-illumination image, thereby generating a surface normal map for representing the three-dimensional directional information of the paint surface. The process includes: For each pixel in the paint surface area in the preprocessed multi-illumination image, a system of linear equations is established based on a Lambertian reflectance model, relating the brightness value of the pixel under multiple illumination directions to the surface normal vector and reflectivity; The linear equations are expressed as a matrix, and the matrix equations are solved using the least squares method to calculate the vector corresponding to each pixel; The solved vector is normalized to obtain the unit surface normal vector of all pixels, and its three components are stored in a three-channel data matrix corresponding to the image size to generate the final surface normal map.

4. The real-time detection system for motorcycle paint bubble and sagging defects based on image analysis according to claim 1 is characterized in that: The process of positioning the candidate area of the bubble defect candidate area by the candidate area positioning unit includes: Calculate the partial derivatives of the first two channels of the surface normal map in the x and y directions respectively, and then synthesize the gradient amplitude of each pixel to generate the corresponding gradient amplitude map; Applying a Gaussian-Laplacian operator to perform convolution processing on the gradient magnitude map to obtain a response map; The response image processed by the Gaussian-Laplacian operator is subjected to binary threshold segmentation to identify the areas where the response value is higher than the preset threshold, forming multiple independent connected domains. These connected domains are identified as candidate areas for bubble defects.

5. The real-time detection system for motorcycle paint bubble and sagging defects based on image analysis according to claim 1 is characterized in that: The process of positioning the candidate area of the sag defect candidate area by the candidate area positioning unit includes: Calculating the directional gradient of the surface normal map along a preset direction to generate a corresponding directional gradient map; Performing Hessian matrix analysis on the directional gradient map, and describing the local curvature information of the image through the second-order partial derivatives; By screening pixels that meet the eigenvalue conditions, areas with continuous and linear features are identified; the screening results are then optimized using morphological operations, and the connected areas finally extracted are identified as candidate areas for sagging defects.

6. The real-time detection system for motorcycle paint bubble and sagging defects based on image analysis according to claim 1 is characterized in that: The feature extraction and classification unit extracts the fusion features of the defect candidate area and uses a classifier to identify it, and the process of obtaining the detection result of the defect candidate area includes: Extracting 2D geometric features and 3D topographic features from the binary mask used for extraction and the surface normal map used for defect candidate area location, respectively; Inputting the features obtained by fusing the two-dimensional geometric features with the three-dimensional topographic features into a pre-trained classifier for processing to obtain a corresponding confidence score; A grading process is performed according to the confidence scores to obtain a detection result.

7. The real-time detection system for motorcycle paint bubble and sagging defects based on image analysis according to claim 6 is characterized in that: The two-dimensional geometric features include: area, aspect ratio, roundness and main axis direction; the three-dimensional morphological features include: average normal deviation, surface curvature and gradient energy.

Images