Feature Point Quality Defect Recognition Method for Multi-Objective Hierarchical Image Processing

Abstract

The present invention discloses a method for identifying quality defects of feature points in multi-object hierarchical image processing, which relates to the technical field of vehicle painting processing. The method includes collecting image data of the vehicle paint surface from different angles and light source conditions using a multi-modal imaging device, performing hierarchical processing on the collected images according to the resolution and target scale, fusing the image data with different resolutions to form a multi-scale image data set; performing hierarchical feature extraction based on the multi-scale image data set, classifying the extracted features according to the target scale to form a multi-object hierarchical feature library; through multi-modal imaging, multi-scale data fusion, hierarchical feature extraction, hierarchical classification, as well as three-dimensional positioning and annotation, comprehensively solving the complexity problem of defect detection in the vehicle painting process, being able to effectively handle problems such as diverse defect types, large feature differences, wide size ranges, and complex distributions, and significantly improving the detection accuracy, classification accuracy, and work efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle painting processing, and more specifically, to a method for identifying quality defects at feature points through multi-object hierarchical image processing. Background Art

[0002] With the rapid development of the automotive industry, the monitoring of the appearance quality of vehicles, especially defects on the paint surface, has become a crucial link in the automotive manufacturing process. Vehicle painting is a key link in automotive manufacturing, and the painting quality directly affects the appearance and corrosion resistance of vehicles. Existing vehicle painting quality defect identification technologies mainly rely on machine vision and artificial intelligence technologies, and have realized the transformation from traditional manual inspection to automated and intelligent inspection;

[0003] Machine vision technology uses industrial cameras, light sources, and image processing algorithms to achieve automatic detection of defects on the paint surface. Generally, a gantry frame is used to arrange the light source and camera to cover the main surfaces of the vehicle body, and common defects such as particles, sags, and scratches can be detected;

[0004] However, in actual use, various types of defects may occur during the vehicle painting process. These defects have significant differences in shape, size, and distribution, increasing the complexity of detection. Common defects include particles, sags, scratches, shrinkage holes, bubbles, color differences, etc. These defects have large characteristic differences, and it is difficult to cover them with a single detection method. Many defects (such as pinholes and fibers) are extremely small in size and account for a very low proportion in the image (for example, the proportion of defective pixels may be only 0.01%), and are easily misjudged or missed;

[0005] Moreover, scratches may show irregular linear distributions, while keyholes may be circular or oval and may be densely distributed, making it difficult to detect and classify. Summary of the Invention

[0006] To solve the above problems, the present invention provides a method for identifying quality defects at feature points through multi-object hierarchical image processing.

[0007] The present invention provides a method for identifying quality defects at feature points through multi-object hierarchical image processing, including the following steps:

[0008] Step 1: Use a multi-modal imaging device to collect image data of the paint surface under different angles and light source conditions. Classify the collected images according to resolution and target scale, and fuse image data with different resolutions to form a multi-scale image dataset;

[0009] Step 2: According to the multi-scale image dataset, perform hierarchical feature extraction, classify the extracted features according to the target scale, and form a multi-object hierarchical feature library;

[0010] Step 3: Classify and grade the detected defects according to the multi-objective hierarchical feature library, and integrate the detection results to form hierarchical defect detection information;

[0011] Step 4: Map the detected defect location information to the vehicle body three-dimensional model according to the hierarchical defect detection information to achieve three-dimensional positioning of the defects, and perform multi-objective hierarchical annotation according to the scale of the defects, and transmit it to the mobile terminal of the staff.

[0012] Preferably, the specific steps of using the multi-modal imaging device to collect the paint surface image data from different angles and light source conditions include:

[0013] First, select a device with multi-modal imaging capabilities, specifically including a high-definition camera, a fringe light projector, a white light source, and a polarized light source, and install them;

[0014] Subsequently, configure the fringe light, white light, and polarized light sources respectively;

[0015] During the acquisition process, turn on the fringe light projector, white light source, and polarized light source in sequence, collect horizontal and vertical fringe images, uniform illumination images, and images under polarized light conditions respectively, and obtain multi-angle images of the paint surface from different angles to obtain the paint surface image data.

[0016] Preferably, the specific steps of Step 1 further include:

[0017] Classify the collected image data into high-resolution images and low-resolution images according to the resolution;

[0018] According to the scale of the detection target, divide the images into two groups. For the detection of tiny defects, high-resolution images are used, and for the detection of large-area defects, low-resolution images are used;

[0019] Perform multi-scale decomposition on the high-resolution and low-resolution image data respectively to obtain low-frequency components and high-frequency components. The specific calculation formula is , calculate and obtain the low-frequency component ;

[0020] According to the calculation formula ;

[0021] ;

[0022] , calculate and obtain the high-frequency components , and ;

[0023] where is the low-frequency component of the previous decomposition, H is the low-pass filter, G is the high-pass filter, and are low - pass filters in the row and column directions respectively, and are high - pass filters in the row and column directions respectively;

[0024] Fuse the decomposed low - frequency and high - frequency components according to the resolution and the target scale;

[0025] For the low - frequency and high - frequency components, perform fusion. Then, perform inverse wavelet transform on the fused low - frequency and high - frequency components to restore the fused multi - scale image, and use the fused multi - scale image as the multi - scale image dataset.

[0026] Preferably, the specific steps for fusing the decomposed low - frequency and high - frequency components include:

[0027] Low - frequency component fusion. For the low - frequency component of the high - resolution image , which respectively includes the high - resolution low - frequency component and the low - resolution low - frequency component According to the formulas and calculate the energy of the low - frequency component of the high - resolution image and the energy of the low - frequency component of the low - resolution image ;

[0028] According to the formula , calculate and obtain the fused low - frequency component;

[0029] where ;

[0030] For high - frequency component fusion, for the high - frequency component of the high - resolution image, specifically including the horizontal, vertical, and diagonal directions, calculate their absolute values respectively. Similarly, for the high - frequency component of the low - resolution image, also calculate their absolute values respectively. Compare the absolute values of the horizontal high - frequency components of the high - resolution and low - resolution images, and select the part with the larger absolute value as the fused horizontal high - frequency component; compare the absolute values of the vertical high - frequency components of the high - resolution and low - resolution images, and select the part with the larger absolute value as the fused vertical high - frequency component; compare the absolute values of the diagonal high - frequency components of the high - resolution and low - resolution images, and select the part with the larger absolute value as the fused diagonal high - frequency component.

[0031] Preferably, the specific working steps for performing inverse wavelet transform on the fused low - frequency and high - frequency components to restore the fused multi - scale image are as follows:

[0032] Perform convolution operation on the fused low - frequency component with the low - pass synthesis filters in the row and column directions to obtain the first convolution result and restore the low - frequency part of the image;

[0033] Perform a convolution operation on the fused horizontal high-frequency component with the high-pass synthesis filter in the row direction and the low-pass synthesis filter in the column direction to obtain a second convolution result and restore the horizontal high-frequency part of the image;

[0034] Perform a convolution operation on the fused vertical high-frequency component with the low-pass synthesis filter in the row direction and the high-pass synthesis filter in the column direction to obtain a third convolution result and restore the vertical high-frequency part of the image;

[0035] Perform a convolution operation on the fused diagonal high-frequency component with the high-pass synthesis filters in the row and column directions to obtain a fourth convolution result and restore the diagonal high-frequency part of the image;

[0036] Add the first convolution result, the second convolution result, the third convolution result, and the fourth convolution result to obtain the fused multi-scale image.

[0037] Preferably, the specific working steps of step two are as follows:

[0038] First, extract the texture features of the tiny defects from the fused multi-scale image, and the extracted texture features and the tiny defect feature library;

[0039] Then, extract the shape features of the large-area defects from the fused multi-scale image, and classify the extracted shape features into the large-area defect feature library;

[0040] Create a hierarchical feature library, which is divided into two parts: the tiny defect feature library and the large-area defect feature library. Each feature library contains the following information: feature vector, image identifier corresponding to the feature, defect type corresponding to the feature, and scale information corresponding to the feature, forming a multi-target hierarchical feature library.

[0041] Preferably, the specific steps for extracting the texture features of the tiny defects and the shape features of the large-area defects are as follows:

[0042] First, extract the texture features for the tiny defects. By analyzing the gray-scale distribution and gradient information of the image, obtain the feature vector reflecting the texture details, specifically including contrast Y1, correlation Y2, energy Y3, and homogeneity Y4;

[0043] Subsequently, extract the shape features for the large-area defects. By calculating the geometric parameters of the contour, specifically including area U1, perimeter U2, aspect ratio U3, and circularity U4, obtain the feature vector reflecting the shape features.

[0044] Preferably, the specific working steps of step three are as follows:

[0045] For tiny defects, according to the formula;

[0046] , calculate and obtain the texture feature score Y5 of the micro defect. A threshold range for the micro defect is set in advance: T1 < T2. If the texture feature score Y5 of the micro defect is lower than T1, the level of the micro defect is a low level. If the texture feature score Y5 of the micro defect is greater than or equal to T1 and less than T2, the level of the micro defect is a medium level. If the texture feature score Y5 of the micro defect is greater than or equal to T2, the level of the micro defect is a high level, and the defect level is obtained.

[0047] Preferably, the specific working steps of step three further include:

[0048] For large-area defects, according to the formula;

[0049] , calculate and obtain the texture feature score U5 of the large-area defect. A threshold range for the large-area defect is set in advance (T3 < T4). If the texture feature score U5 of the large-area defect is lower than T3, the level of the large-area defect is a low level. If the texture feature score U5 of the large-area defect is greater than or equal to T3 and less than T4, the level of the large-area defect is a medium level. If the texture feature score U5 of the large-area defect is greater than or equal to T4, the level of the large-area defect is a high level, and the defect level is obtained;

[0050] Integrate the classified defect information into a structured format, specifically (defect type, defect location, defect size, defect level), and use this structured format as the classified defect detection information.

[0051] Preferably, the specific working steps of step four are:

[0052] Extract the position information of the defect in the two-dimensional image from the defect detection information, including the center coordinates (X1, Y1) and size of the defect;

[0053] According to the internal and external parameters of the camera, map the two-dimensional defect position to the three-dimensional model of the vehicle body. The specific steps include;

[0054] Obtain the internal parameter matrix K1 of the camera, , where and are the focal lengths of the camera, and are the image center coordinates;

[0055] Obtain the depth information of the defect position through a depth sensor or a binocular camera;

[0056] According to the formula Calculate and obtain the three-dimensional coordinate information of the defect , where is the external matrix of the camera, is the translation vector, representing the displacement of the camera;

[0057] Integrate the type, location, size, severity of the defect, and the three-dimensional coordinate information of the defect into annotation information to form complete defect annotation data, and perform unified formatting. Finally, transmit the formatted data to the mobile terminal of the staff through the wireless network.

[0058] Beneficial effects: Through multi-modal imaging, multi-scale data fusion, hierarchical feature extraction, hierarchical classification, and three-dimensional positioning and annotation, the complexity problem of defect detection in the vehicle painting process is comprehensively solved. It can effectively cope with problems such as diverse defect types, large feature differences, wide size ranges, and complex distributions, significantly improving the detection accuracy, classification accuracy, and work efficiency. Description of the Drawings

[0059] Figure 1 is the flowchart of the method of the present invention. Detailed Embodiments

[0060] Application scenarios: In the actual use process, various types of defects may occur during the vehicle painting process. These defects have significant differences in shape, size, and distribution, increasing the complexity of detection. Common defects include particles, sags, scratches, shrinkage pores, bubbles, color differences, etc. These defects have large feature differences, and it is difficult for a single detection method to cover them. Many defects (such as pinholes, fibers) are extremely small in size and account for a very low proportion in the image (such as the proportion of defect pixels may be only 0.01%), and are easily misjudged or missed;

[0061] Moreover, scratches may show irregular linear distributions, while keyholes may show circular or oval shapes and may be densely distributed, making it difficult to detect and classify.

[0062] Such as Figure 1 shown: The feature point quality defect recognition method for multi-object hierarchical image processing includes the following steps:

[0063] Step 1: Use a multi-modal imaging device to collect image data of the car paint surface from different angles and light source conditions. Grade the collected images according to the resolution and target scale, and fuse the image data with different resolutions to form a multi-scale image dataset. It should be noted that by combining multiple light sources (such as fringe light, white light, polarized light) and imaging angles, the problems of high reflectivity and low contrast on the car paint surface can be effectively solved, enhancing the visibility of defect features. Fusing the image data with different resolutions can cover both tiny defects (such as pinholes, fibers) and large-area defects (such as sags, shrinkage pores) at the same time, avoiding the limitations of single-scale image data. Grading the images according to the resolution and target scale provides a basis for subsequent hierarchical feature extraction and classification, improving the processing efficiency;

[0064] Step 2: According to the multi-scale image dataset, perform hierarchical feature extraction, classify the extracted features according to the target scale, and form a multi-target hierarchical feature library. It should be noted that for image data of different scales, features suitable for that scale are extracted (such as texture features of tiny defects and shape features of large-area defects), which improves the pertinence and effectiveness of the features. Classifying the features according to the target scale provides rich feature support for subsequent hierarchical classification and improves the accuracy of classification.

[0065] Step 3: According to the multi-target hierarchical feature library, perform hierarchical classification on the detected defects, integrate the detection results, and form hierarchical defect detection information. It should be noted that according to the multi-scale features in the feature library, hierarchical classification of defects can effectively distinguish defects of different sizes and types, improve the accuracy of classification, and integrate the results of hierarchical classification to form complete detection information, which is convenient for subsequent analysis and processing.

[0066] Step 4: According to the hierarchical defect detection information, map the detected defect location information to the vehicle body three-dimensional model to achieve three-dimensional positioning of the defects, and perform multi-target hierarchical annotation according to the scale of the defects, and transmit it to the mobile terminal of the staff. It should be noted that mapping the defect location information to the vehicle body three-dimensional model can achieve high-precision defect positioning and provide intuitive guidance for the repair work. Hierarchical annotation according to the scale of the defects can clearly display defects of different sizes and types, which is convenient for the staff to quickly identify and process.

[0067] As an optional embodiment: The specific steps of using the multi-modal imaging device to collect the paint surface image data from different angles and light source conditions include:

[0068] First, select a device with multi-modal imaging capabilities, specifically including a high-definition camera, a fringe light projector, a white light source, and a polarized light source, and install them.

[0069] Subsequently, configure the fringe light, white light, and polarized light sources respectively.

[0070] During the collection process, turn on the fringe light projector, white light source, and polarized light source in sequence, collect horizontal and vertical fringe images, uniform illumination images, and images under polarized light conditions respectively, and obtain multi-angle images of the paint surface from different angles to obtain the paint surface image data.

[0071] It should be noted that the fringe light projector needs to support adjustable fringe spacing and direction to meet the detection requirements of different scale defects. The spacing of the fringe light can be adjusted according to the paint surface characteristics. For example, when detecting tiny defects, the fringe spacing can be set to 1-2 mm, and when detecting large-area defects, the fringe spacing can be set to 5-10 mm.

[0072] The white light source should provide uniform diffused light to reduce reflections and highlights on the car paint surface. A laser-induced high-brightness white light source is used, with a spectral output range of 170 - 2100 nm, a radiant brightness of 40 mW / (mm²·sr·nm) @ 400 nm, and having the characteristics of an ultra-wide spectrum and high brightness;

[0073] The polarized light source needs to be equipped with an adjustable polarization filter to reduce the interference of reflected light. For example, a high-polarization light source PLS-100 is used, with a polarization extinction ratio (PER) of up to 45 dB and a wavelength range of 1310 ± 20 nm or 1550 ± 20 nm. The polarization axis angle can be adjusted within the range of 0 to 180 degrees to meet different detection requirements;

[0074] To fully cover the car paint surface, images need to be acquired from different angles. The robotic arm or manual adjustment device should ensure that the distance between the camera and the car paint surface is maintained between 30 - 50 cm.

[0075] As an optional embodiment: The specific steps of the first step further include:

[0076] The acquired image data is divided into high-resolution images and low-resolution images according to the resolution; it should be noted that the resolution is divided according to the pixel size or sampling rate of the image. For high-resolution images, the pixel size is greater than 1200×1200, and for low-resolution images, the pixel size is less than 600×600;

[0077] According to the scale of the detection target, the images are divided into two groups. For micro defect detection, high-resolution images are used, and for large-area defect detection, low-resolution images are used;

[0078] The high-resolution and low-resolution image data are respectively subjected to multi-scale decomposition to obtain low-frequency components and high-frequency components. The specific calculation formula is , and the low-frequency component is calculated and obtained;

[0079] According to the calculation formula ;

[0080] ;

[0081] , and the high-frequency components , and are calculated and obtained;

[0082] Where is the low-frequency component of the previous decomposition, H is the low-pass filter, G is the high-pass filter, and They are low-pass filters in the row and column directions respectively. and They are high-pass filters in the row and column directions respectively.

[0083] Fuse the decomposed low-frequency and high-frequency components according to the resolution and the target scale.

[0084] For the low-frequency and high-frequency components, perform fusion. Then, perform inverse wavelet transform on the fused low-frequency and high-frequency components to recover the fused multi-scale image, and use the fused multi-scale image as the multi-scale image dataset.

[0085] As an optional embodiment: The specific steps for fusing the decomposed low-frequency and high-frequency components include:

[0086] Low-frequency component fusion. For the low-frequency component of the high-resolution image , which respectively includes the high-resolution low-frequency component and the low-resolution low-frequency component According to the formulas and calculate the energy of the low-frequency component of the high-resolution image and the energy of the low-frequency component of the low-resolution image ;

[0087] According to the formula , calculate and obtain the fused low-frequency component;

[0088] where ;

[0089] For high-frequency component fusion, for the high-frequency component of the high-resolution image, specifically including the horizontal, vertical, and diagonal directions, calculate their absolute values respectively. Similarly, for the high-frequency component of the low-resolution image, also calculate their absolute values respectively. Compare the absolute values of the horizontal high-frequency components of the high-resolution and low-resolution images, and select the part with the larger absolute value as the fused horizontal high-frequency component; compare the absolute values of the vertical high-frequency components of the high-resolution and low-resolution images, and select the part with the larger absolute value as the fused vertical high-frequency component; compare the absolute values of the diagonal high-frequency components of the high-resolution and low-resolution images, and select the part with the larger absolute value as the fused diagonal high-frequency component.

[0090] As an optional embodiment: The specific working steps for performing inverse wavelet transform on the fused low-frequency and high-frequency components to recover the fused multi-scale image are as follows:

[0091] Perform convolution operation on the fused low-frequency component with the low-pass synthesis filters in the row and column directions to obtain the first convolution result and recover the low-frequency part of the image;

[0092] Perform a convolution operation on the fused horizontal high-frequency component with the high-pass synthesis filter in the row direction and the low-pass synthesis filter in the column direction to obtain a second convolution result, restoring the horizontal high-frequency part of the image;

[0093] Perform a convolution operation on the fused vertical high-frequency component with the low-pass synthesis filter in the row direction and the high-pass synthesis filter in the column direction to obtain a third convolution result, restoring the vertical high-frequency part of the image;

[0094] Perform a convolution operation on the fused diagonal high-frequency component with the high-pass synthesis filters in the row and column directions to obtain a fourth convolution result, restoring the diagonal high-frequency part of the image; It should be noted that the low-pass synthesis filters in the row and column directions are used to restore the low-frequency part of the image, and the high-pass synthesis filters in the row and column directions are used to restore the high-frequency part of the image;

[0095] Add the first convolution result, the second convolution result, the third convolution result, and the fourth convolution result to obtain the fused multi-scale image.

[0096] As an optional embodiment: The specific working steps of the second step are as follows:

[0097] First, extract the texture features of the tiny defects from the fused multi-scale image, and classify the extracted texture features into the tiny defect feature library;

[0098] Then, extract the shape features of the large-area defects from the fused multi-scale image, and classify the extracted shape features into the large-area defect feature library;

[0099] Create a hierarchical feature library, which is divided into two parts: the tiny defect feature library and the large-area defect feature library. Each feature library contains the following information: feature vector, image identifier corresponding to the feature, defect type corresponding to the feature, and scale information corresponding to the feature, forming a multi-object hierarchical feature library.

[0100] As an optional embodiment: The specific steps for extracting the texture features of the tiny defects and the shape features of the large-area defects are as follows:

[0101] First, extract the texture features for the tiny defects. By analyzing the gray-scale distribution and gradient information of the image, obtain the feature vectors reflecting the texture details, specifically including contrast Y1, correlation Y2, energy Y3, and homogeneity Y4; It should be noted that in this embodiment, specifically by selecting appropriate distance and direction parameters (such as a distance of 1 pixel and directions of 0°, 45°, 90°, 135°), calculate the gray-level co-occurrence matrix, and extract texture features from the gray-level co-occurrence matrix, including contrast, correlation, energy, homogeneity, and ASM;

[0102] Subsequently, shape features are extracted for large-area defects. By calculating the geometric parameters of the contour, specifically including the area U1, perimeter U2, aspect ratio U3, and circularity U4, a feature vector reflecting the shape features is obtained. It should be noted that the calculation formulas for area, perimeter, aspect ratio, and circularity are existing formulas.

[0103] As an optional embodiment: The specific working steps of step three are as follows:

[0104] For micro-defects, according to the formula;

[0105] , the texture feature score Y5 of the micro-defect is calculated and obtained. A threshold range for micro-defects is set in advance: T1 < T2. If the texture feature score Y5 of the micro-defect is lower than T1, the level of the micro-defect is a low level. If the texture feature score Y5 of the micro-defect is greater than or equal to T1 and less than T2, the level of the micro-defect is a medium level. If the texture feature score Y5 of the micro-defect is greater than or equal to T2, the level of the micro-defect is a high level, and the defect level is obtained.

[0106] As an optional embodiment: The specific working steps of step three further include:

[0107] For large-area defects, according to the formula;

[0108] , the texture feature score U5 of the large-area defect is calculated and obtained. A threshold range for large-area defects is set in advance (T3 < T4). If the texture feature score U5 of the large-area defect is lower than T3, the level of the large-area defect is a low level. If the texture feature score U5 of the large-area defect is greater than or equal to T3 and less than T4, the level of the large-area defect is a medium level. If the texture feature score U5 of the large-area defect is greater than or equal to T4, the level of the large-area defect is a high level, and the defect level is obtained;

[0109] The classified defect information is integrated into a structured format, specifically (defect type, defect location, defect size, defect level), and this structured format is used as the classified defect detection information. It should be noted that the defect type can be "micro-defect" or "large-area defect", the defect location and size are determined according to the actual detection results, and the defect level is calculated according to the above classification formula.

[0110] As an optional embodiment: The specific working steps of step four are as follows:

[0111] Extract the position information of the defect in the two-dimensional image from the defect detection information, including the center coordinates (X1, Y1) and size of the defect;

[0112] Map the two-dimensional defect position onto the three-dimensional model of the vehicle body according to the internal and external parameters of the camera. The specific steps are as follows;

[0113] Obtain the internal parameter matrix K1 of the camera, , where and are the focal lengths of the camera, and are the image center coordinates;

[0114] Obtain the depth information of the defect position through a depth sensor or a binocular camera; it should be noted that the depth information provides the depth distance of the defect position relative to the camera;

[0115] According to the formula Calculate and obtain the three-dimensional coordinate information of the defect , where is the external matrix of the camera, is the translation vector, representing the displacement of the camera;

[0116] Integrate the type, position, size, severity of the defect and the three-dimensional coordinate information of the defect into annotation information to form complete defect annotation data, and perform unified formatting. Finally, transmit the formatted data to the mobile terminal of the staff through a wireless network.

[0117] Working principle:

[0118] Use a multi-modal imaging device to collect paint surface image data under different angles and light source conditions. Classify the collected images according to the resolution and target scale, and fuse the image data of different resolutions to form a multi-scale image dataset; it should be noted that by combining multiple light sources (such as stripe light, white light, polarized light) and imaging angles, problems such as high reflectivity and low contrast on the paint surface can be effectively solved, enhancing the visibility of defect features. Fusing the image data of different resolutions can cover both tiny defects (such as pinholes, fibers) and large-area defects (such as sags, shrinkage holes) at the same time, avoiding the limitations of single-scale image data. Classifying the images according to the resolution and target scale provides a basis for subsequent hierarchical feature extraction and classification, improving the processing efficiency;

[0119] According to the multi-scale image dataset, perform hierarchical feature extraction, classify the extracted features according to the target scale, and form a multi-target hierarchical feature library; it should be noted that for image data of different scales, extract features suitable for that scale (such as texture features of tiny defects, shape features of large-area defects), improving the pertinence and effectiveness of the features. Classifying the features according to the target scale provides rich feature support for subsequent hierarchical classification, improving the accuracy of classification;

[0120] According to the multi-objective hierarchical feature library, the detected defects are classified hierarchically, and the detection results are integrated to form hierarchical defect detection information. It should be noted that classifying and grading the defects according to the multi-scale features in the feature library can effectively distinguish defects of different sizes and types, improve the accuracy of classification, and integrate the results of hierarchical classification to form complete detection information for subsequent analysis and processing.

[0121] Step 4: According to the hierarchical defect detection information, map the detected defect location information to the vehicle body three-dimensional model to achieve three-dimensional positioning of the defects, and perform multi-objective hierarchical annotation according to the scale of the defects, and transmit it to the mobile terminal of the staff. It should be noted that mapping the defect location information to the vehicle body three-dimensional model can achieve high-precision defect positioning and provide intuitive guidance for the repair work. Hierarchical annotation according to the scale of the defects can clearly display defects of different sizes and types, facilitating the staff to quickly identify and process them.

[0122] The above is only the preferred implementation mode of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be pointed out that for those of ordinary skill in the art of this technology, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of this template.

Claims

1. A method for identifying quality defects at characteristic points by multi-objective hierarchical image processing, characterized in that: The following steps are involved: Step 1: Use multimodal imaging equipment to collect car paint surface image data from different angles and light source conditions, hierarchically process the collected images according to resolution and target scale, and fuse image data of different resolutions to form a multi-scale image dataset; Step 2: Based on the multi-scale image data set, perform hierarchical feature extraction, classify the extracted features according to the target scale, and form a multi-target hierarchical feature library; Step 3: Classify the detected defects according to the multi-objective classification feature library, integrate the detection results, and form the classification defect detection information; Step 4: Based on the graded defect detection information, the detected defect location information is mapped with the three-dimensional model of the vehicle body to achieve three-dimensional positioning of the defect, and multi-target graded labeling is performed according to the scale of the defect, and transmitted to the staff's mobile terminal; The specific steps of step one also include: The collected image data is divided into high-resolution images and low-resolution images according to the resolution; According to the scale of the inspection target, the images are divided into two groups, where high-resolution images are used for tiny defect detection and low-resolution images are used for large-area defect detection; Perform multi-scale decomposition on high-resolution and low-resolution image data to obtain low-frequency components and high-frequency components. The specific calculation formula is: , calculate and obtain the low-frequency component ; According to the calculation formula ; ; , calculate and obtain the high-frequency component , and ; in is the low-frequency component of the last decomposition, H is a low-pass filter, G is a high-pass filter, and are low-pass filters in row and column directions, respectively. and They are high-pass filters in row and column directions respectively; According to the resolution and target scale, the decomposed low-frequency and high-frequency components are fused; The low-frequency component and the high-frequency component are fused, and the fused low-frequency and high-frequency components are subjected to inverse wavelet transform to restore the fused multi-scale image, which is used as a multi-scale image dataset.

2. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1 is characterized in that: The specific steps of using a multimodal imaging device to collect vehicle paint surface image data from different angles and light source conditions include: First, select equipment with multimodal imaging capabilities, including high-definition cameras, streak light projectors, white light sources, and polarized light sources, and install them; Then, stripe light, white light and polarized light sources are configured respectively; During the acquisition process, the stripe light projector, white light source and polarized light source are turned on in sequence to collect horizontal and vertical stripe images, uniformly illuminated images and images under polarized light conditions, respectively, and multi-angle images of the vehicle paint surface are obtained from different angles to obtain vehicle paint surface image data.

3. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1 is characterized in that: The specific steps of fusing the decomposed low-frequency and high-frequency components include: Low-frequency component fusion, for low-frequency components of high-resolution images , including high-resolution low-frequency components and low-resolution low-frequency components According to the formula and Calculate the energy of the low-frequency component of the high-resolution image and the energy of the low-frequency components of the low-resolution image ; According to the formula , calculate and obtain the fused low-frequency component; in ; For high-frequency component fusion, the absolute values ​​of the high-frequency components of the high-resolution image, specifically in the horizontal, vertical and diagonal directions, are calculated respectively. Similarly, the absolute values ​​of the high-frequency components of the low-resolution image are also calculated respectively. The absolute values ​​of the horizontal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused horizontal high-frequency component; the absolute values ​​of the vertical high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused vertical high-frequency component. The absolute values ​​of the diagonal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the combined diagonal high-frequency component.

4. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1, characterized in that: The specific working steps of performing inverse wavelet transform on the fused low-frequency and high-frequency components to restore the fused multi-scale image are as follows: The fused low-frequency component is convolved with the low-pass comprehensive filter in the row direction and the column direction to obtain the first convolution result and restore the low-frequency part of the image; The fused horizontal high-frequency component is convolved with the high-pass synthesis filter in the row direction and the low-pass synthesis filter in the column direction to obtain a second convolution result, thereby restoring the horizontal high-frequency part of the image; The fused vertical high-frequency component is convolved with the low-pass comprehensive filter in the row direction and the high-pass comprehensive filter in the column direction to obtain the third convolution result, thereby restoring the vertical high-frequency part of the image; The fused diagonal high-frequency components are convolved with high-pass integrated filters in the row and column directions to obtain a fourth convolution result, thereby restoring the diagonal high-frequency part of the image; The first convolution result, the second convolution result, the third convolution result and the fourth convolution result are added together to obtain a fused multi-scale image.

5. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1, characterized in that: The specific working steps of step 2 are as follows: Firstly, the texture features of tiny defects are extracted from the fused multi-scale image, and the extracted texture features are used as a tiny defect feature library; Then, the shape features of large-area defects are extracted from the fused multi-scale images, and the extracted shape features are classified into a large-area defect feature library; Create a hierarchical feature library, which is divided into two parts: a micro-defect feature library and a large-area defect feature library. Each feature library contains the following information: feature vector, image identifier corresponding to the feature, defect type corresponding to the feature, and scale information corresponding to the feature, forming a multi-target hierarchical feature library.

6. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 5 is characterized in that: The specific steps of extracting texture features of tiny defects and shape features of large area defects are as follows: First, texture features are extracted for tiny defects. By analyzing the grayscale distribution and gradient information of the image, feature vectors reflecting texture details are obtained, including contrast Y1, correlation Y2, energy Y3, and homogeneity Y4. Then, shape features are extracted for large-area defects, and feature vectors reflecting shape features are obtained by calculating the geometric parameters of the contour, including area U1, perimeter U2, aspect ratio U3, and circularity U4.

7. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1 is characterized in that: The specific working steps of step three are: For minor defects, according to the formula; , calculate and obtain the texture feature score Y5 of the tiny defect, and set a threshold range of tiny defects in advance: T1<T2, if the texture feature score Y5 of the tiny defect is lower than T1, the grade of the tiny defect is low, if the texture feature score Y5 of the tiny defect is greater than or equal to T1 and less than T2, the grade of the tiny defect is medium, if the texture feature score Y5 of the tiny defect is greater than or equal to T2, the grade of the tiny defect is high, and the defect grade is obtained.

8. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 7 is characterized in that: The specific working steps of step three also include: For large area defects, according to the formula; , calculate and obtain the texture feature score U5 of the large area defect, and set a threshold range of a large area defect in advance: T3<T4, if the texture feature score U5 of the large area defect is lower than T3, the level of the large area defect is low, if the texture feature score U5 of the large area defect is greater than or equal to T3 and less than T4, the level of the large area defect is medium, if the texture feature score U5 of the large area defect is greater than or equal to T4, the level of the large area defect is high, and the defect level is obtained; The classified defect information is integrated into a structured format, specifically: defect type, defect location, defect size, defect level, and this structured format is used as the classified defect detection information.

9. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 7, characterized in that: The specific working steps of step 4 are: Extract the position information of the defect in the two-dimensional image from the defect detection information, including the center coordinates (X1, Y1) and size of the defect; According to the intrinsic and extrinsic parameters of the camera, the two-dimensional defect position is mapped to the three-dimensional model of the vehicle body. The specific steps include: Get the camera's intrinsic parameter matrix K1, ,in and is the focal length of the camera, and are the coordinates of the image center; Obtain the depth information of the defect location through a depth sensor or a binocular camera; According to the formula Calculate and obtain the three-dimensional coordinate information of the defect ,in is the camera's appearance matrix, is the translation vector, which represents the displacement of the camera; The defect type, location, size, severity and three-dimensional coordinate information of the defect are integrated into annotation information to form complete defect annotation data, and then formatted uniformly. Finally, the formatted data is transmitted to the staff's mobile phone terminal via a wireless network.

Images