Characteristic point location quality defect identification method for multi-target grading image processing

Abstract

The invention discloses a feature point location quality defect identification method for multi-target grading image processing, and relates to the technical field of vehicle paint spraying processing, and the method comprises the steps: collecting vehicle paint surface image data from different angles and under different light source conditions through a multi-mode imaging device, carrying out the grading processing of the collected image according to the resolution and the target scale, and carrying out the recognition of the vehicle paint surface image data; fusing the image data with different resolutions to form a multi-scale image data set; performing hierarchical feature extraction according to the multi-scale image data set, and classifying the extracted features according to target scales to form a multi-target hierarchical feature library; through multi-modal imaging, multi-scale data fusion, hierarchical feature extraction, hierarchical classification and three-dimensional positioning and labeling, the problem of complexity of defect detection in the vehicle paint spraying process is comprehensively solved, and the problems of various defect types, large feature difference, wide size range, complex distribution and the like can be effectively solved; and the detection precision, the classification accuracy and the working efficiency are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle painting, and more specifically to a method for identifying quality defects of characteristic points by multi-target hierarchical image processing. Background Art

[0002] With the rapid development of the automobile industry, the appearance quality of vehicles, especially the defect monitoring of paint surface, has become a crucial link in the automobile manufacturing process. Vehicle painting is a key link in automobile manufacturing. The quality of paint directly affects the appearance and corrosion resistance of the vehicle. The existing vehicle paint quality defect recognition technology is mainly based on machine vision and artificial intelligence technology, and has achieved the transformation from traditional manual inspection to automated and intelligent inspection. Machine vision technology uses industrial cameras, light sources and image processing algorithms to automatically detect surface defects of car paint. Generally, the light source and camera are arranged in a door arch frame to cover the main surface of the car body. It can detect common defects such as particles, sagging and scratches. However, in actual use, many types of defects may appear during the vehicle painting process. These defects vary significantly in shape, size and distribution, which increases the complexity of detection. Common defects include particles, sagging, scratches, shrinkage holes, bubbles, color difference, etc. The characteristics of these defects vary greatly and are difficult to cover with a single detection method. Many defects (such as pinholes and fibers) are extremely small in size and account for a very low proportion in the image (for example, defective pixels may account for only 0.01%), which can easily be misjudged or missed. Moreover, scratches may be distributed in irregular lines, while keyholes may be circular or oval and may be densely distributed, making them difficult to detect and classify. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a method for identifying quality defects of feature points by multi-target hierarchical image processing.

[0004] The present invention provides a method for identifying quality defects of characteristic points by multi-objective hierarchical image processing, comprising the following steps: Step 1: Use multimodal imaging equipment to collect car paint surface image data from different angles and light source conditions, hierarchically process the collected images according to resolution and target scale, and fuse image data of different resolutions to form a multi-scale image dataset; Step 2: Based on the multi-scale image data set, perform hierarchical feature extraction, classify the extracted features according to the target scale, and form a multi-target hierarchical feature library; Step 3: Classify the detected defects according to the multi-objective classification feature library, integrate the detection results, and form the classification defect detection information; Step 4: Based on the graded defect detection information, the detected defect location information is mapped with the three-dimensional model of the vehicle body to achieve three-dimensional positioning of the defect, and multi-target graded labeling is performed according to the scale of the defect and transmitted to the staff's mobile terminal.

[0005] Preferably, the specific steps of using a multimodal imaging device to collect vehicle paint surface image data from different angles and light source conditions include: First, select equipment with multimodal imaging capabilities, including high-definition cameras, streak light projectors, white light sources, and polarized light sources, and install them; Then, stripe light, white light and polarized light sources are configured respectively; During the acquisition process, the stripe light projector, white light source and polarized light source are turned on in sequence to collect horizontal and vertical stripe images, uniformly illuminated images and images under polarized light conditions, respectively, and multi-angle images of the vehicle paint surface are obtained from different angles to obtain vehicle paint surface image data.

[0006] Preferably, the specific steps of step one also include: The collected image data is divided into high-resolution images and low-resolution images according to the resolution; According to the scale of the inspection target, the images are divided into two groups, where high-resolution images are used for tiny defect detection and low-resolution images are used for large-area defect detection; Perform multi-scale decomposition on high-resolution and low-resolution image data to obtain low-frequency components and high-frequency components. The specific calculation formula is: , calculate and obtain the low-frequency component ; According to the calculation formula ; ; , calculate and obtain the high-frequency component , and ; in is the low-frequency component of the last decomposition, H is a low-pass filter, G is a high-pass filter, and are low-pass filters in row and column directions, respectively. and They are high-pass filters in row and column directions respectively; According to the resolution and target scale, the decomposed low-frequency and high-frequency components are fused; The low-frequency component and the high-frequency component are fused, and the fused low-frequency and high-frequency components are subjected to inverse wavelet transform to restore the fused multi-scale image, which is used as a multi-scale image dataset.

[0007] Preferably, the specific step of fusing the decomposed low-frequency and high-frequency components comprises: Low-frequency component fusion, for low-frequency components of high-resolution images , including high-resolution low-frequency components and low-resolution low-frequency components According to the formula and Calculate the energy of the low-frequency component of the high-resolution image and the energy of the low-frequency components of the low-resolution image ; According to the formula , calculate and obtain the fused low-frequency component; in ; For high-frequency component fusion, the absolute values ​​of the high-frequency components of the high-resolution image, specifically in the horizontal, vertical and diagonal directions, are calculated respectively. Similarly, the absolute values ​​of the high-frequency components of the low-resolution image are also calculated respectively. The absolute values ​​of the horizontal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused horizontal high-frequency component; the absolute values ​​of the vertical high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused vertical high-frequency component. The absolute values ​​of the diagonal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the combined diagonal high-frequency component.

[0008] Preferably, the specific working steps of performing inverse wavelet transform on the fused low-frequency and high-frequency components to restore the fused multi-scale image are as follows: The fused low-frequency component is convolved with the low-pass comprehensive filter in the row direction and the column direction to obtain the first convolution result and restore the low-frequency part of the image; The fused horizontal high-frequency component is convolved with the high-pass synthesis filter in the row direction and the low-pass synthesis filter in the column direction to obtain a second convolution result, thereby restoring the horizontal high-frequency part of the image; The fused vertical high-frequency component is convolved with the low-pass comprehensive filter in the row direction and the high-pass comprehensive filter in the column direction to obtain the third convolution result, thereby restoring the vertical high-frequency part of the image; The fused diagonal high-frequency components are convolved with high-pass integrated filters in the row and column directions to obtain a fourth convolution result, thereby restoring the diagonal high-frequency part of the image; The first convolution result, the second convolution result, the third convolution result and the fourth convolution result are added together to obtain a fused multi-scale image.

[0009] Preferably, the specific working steps of step 2 are as follows: Firstly, the texture features of tiny defects are extracted from the fused multi-scale image, and the extracted texture features and tiny defect feature library are constructed; Then, the shape features of large-area defects are extracted from the fused multi-scale images, and the extracted shape features are classified into a large-area defect feature library; Create a hierarchical feature library, which is divided into two parts: a micro-defect feature library and a large-area defect feature library. Each feature library contains the following information: feature vector, image identifier corresponding to the feature, defect type corresponding to the feature, and scale information corresponding to the feature, forming a multi-target hierarchical feature library.

[0010] Preferably, the specific steps of extracting texture features of tiny defects and extracting shape features of large area defects are: First, texture features are extracted for tiny defects. By analyzing the grayscale distribution and gradient information of the image, feature vectors reflecting texture details are obtained, including contrast Y1, correlation Y2, energy Y3HE homogeneity Y4; Then, shape features are extracted for large-area defects, and feature vectors reflecting shape features are obtained by calculating the geometric parameters of the contour, including area U1, perimeter U2, aspect ratio U3, and circularity U4.

[0011] Preferably, the specific working steps of step three are: For minor defects, according to the formula; , calculate and obtain the texture feature score Y5 of the tiny defect, set a threshold range of tiny defects in advance (T1 <T2), if the texture feature score Y5 of the tiny defect is lower than T1, the grade of the tiny defect is low, if the texture feature score Y5 of the tiny defect is greater than or equal to T1 and less than T2, the grade of the tiny defect is medium, if the texture feature score Y5 of the tiny defect is greater than or equal to T2, the grade of the tiny defect is high, and the defect grade is obtained.

[0012] Preferably, the specific working steps of step three also include: Located in a large area defect, according to the formula; , calculate and obtain the texture feature score U5 of the large area defect, set a threshold range of a large area defect in advance (T3 <T4), if the texture feature score U5 of the large area defect is lower than T3, the level of the large area defect is low, if the texture feature score U5 of the large area defect is greater than or equal to T3 and less than T4, the level of the large area defect is medium, if the texture feature score U5 of the large area defect is greater than or equal to T4, the level of the large area defect is high, and the defect level is obtained; The classified defect information is integrated into a structured format, specifically defect type, defect location, defect size, and defect registration, and this structured format is used as classified defect detection information.

[0013] Preferably, the specific working steps of step 4 are: Extract the position information of the defect in the two-dimensional image from the defect detection information, including the center coordinates (X1, Y1) and size of the defect; According to the intrinsic and extrinsic parameters of the camera, the two-dimensional defect position is mapped to the three-dimensional model of the vehicle body. The specific steps include: Get the camera's intrinsic parameter matrix K1, ,in and is the focal length of the camera, and are the coordinates of the image center; Obtain the depth information of the defect location through a depth sensor or a binocular camera; According to the formula Calculate and obtain the three-dimensional coordinate information of the defect ,in is the camera's appearance matrix, is the translation vector, which represents the displacement of the camera; The defect type, location, size, severity and three-dimensional coordinate information of the defect are integrated into annotation information to form complete defect annotation data, and then formatted uniformly. Finally, the formatted data is transmitted to the staff's mobile phone terminal via a wireless network.

[0014] Beneficial effects: Through multimodal imaging, multi-scale data fusion, hierarchical feature extraction, hierarchical classification, and three-dimensional positioning and annotation, the complexity of defect detection in the vehicle painting process is comprehensively solved. It can effectively deal with problems such as diverse defect types, large feature differences, wide size ranges, and complex distribution, and significantly improve detection accuracy, classification accuracy, and work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0016] Application scenarios: In actual use, various types of defects may appear in the vehicle painting process. These defects vary significantly in shape, size and distribution, which increases the complexity of detection. Common defects include particles, sagging, scratches, shrinkage holes, bubbles, color difference, etc. The characteristics of these defects vary greatly and are difficult to cover with a single detection method. Many defects (such as pinholes and fibers) are extremely small in size and account for a very low proportion in the image (for example, defective pixels may account for only 0.01%), which can easily be misjudged or missed. Moreover, scratches may be distributed in irregular lines, while keyholes may be circular or oval and may be densely distributed, making them difficult to detect and classify.

[0017] like Figure 1 As shown: The feature point quality defect recognition method of multi-target hierarchical image processing includes the following steps: Step 1: Use multimodal imaging equipment to collect image data of the paint surface from different angles and light source conditions, grade the collected images according to resolution and target scale, and fuse image data of different resolutions to form a multi-scale image data set; It should be noted that by combining multiple light sources (such as streak light, white light, polarized light) and imaging angles, it is possible to effectively solve problems such as high reflection and low contrast on the paint surface, enhance the visibility of defect features, and fuse image data of different resolutions to simultaneously cover tiny defects (such as pinholes and fibers) and large-area defects (such as sagging and shrinkage holes), avoiding the limitations of single-scale image data, and grading images according to resolution and target scale, which provides a basis for subsequent graded feature extraction and classification, and improves processing efficiency; Step 2: Based on the multi-scale image data set, hierarchical feature extraction is performed, and the extracted features are classified according to the target scale to form a multi-target hierarchical feature library; it should be noted that for image data of different scales, features suitable for the scale (such as texture features of tiny defects, shape features of large-area defects) are extracted to improve the pertinence and effectiveness of the features, and the features are classified according to the target scale, which provides rich feature support for subsequent hierarchical classification and improves the accuracy of classification; Step 3: According to the multi-target hierarchical feature library, the detected defects are classified and classified, and the detection results are integrated to form hierarchical defect detection information; it should be noted that the defects are classified and classified according to the multi-scale features in the feature library, which can effectively distinguish defects of different sizes and types, improve the accuracy of classification, and integrate the results of classification to form complete detection information, which is convenient for subsequent analysis and processing; Step 4: Based on the graded defect detection information, the detected defect location information is mapped to the three-dimensional model of the vehicle body to achieve three-dimensional positioning of the defect, and multi-target graded labeling is performed according to the scale of the defect, and transmitted to the staff's mobile terminal. It should be noted that mapping the defect location information to the three-dimensional model of the vehicle body can achieve high-precision defect positioning, provide intuitive guidance for repair work, and graded labeling according to the scale of the defect can clearly display defects of different sizes and types, which is convenient for staff to quickly identify and handle.

[0018] As an optional embodiment: the specific steps of using a multimodal imaging device to collect vehicle paint surface image data from different angles and light source conditions include: First, select equipment with multimodal imaging capabilities, including high-definition cameras, streak light projectors, white light sources, and polarized light sources, and install them; Then, stripe light, white light and polarized light sources are configured respectively; During the acquisition process, the stripe light projector, white light source and polarized light source are turned on in sequence to collect horizontal and vertical stripe images, uniformly illuminated images and images under polarized light conditions, respectively, and multi-angle images of the vehicle paint surface are obtained from different angles to obtain vehicle paint surface image data.

[0019] It should be noted that the stripe light projector needs to support adjustable stripe spacing and direction to meet the detection needs of defects of different scales. The stripe light spacing can be adjusted according to the surface characteristics of the paint. For example, the stripe spacing can be set to 1-2 mm for tiny defect detection and 5-10 mm for large area defect detection. The white light source should provide uniform diffuse light to reduce reflection and highlights on the paint surface. A laser-induced high-brightness white light source is used, with a spectral output range of 170-2100 nanometers and a radiant brightness of 40mW / (mm²·sr·nm)@400nm, which has ultra-wide spectrum and high brightness characteristics; Polarized light sources need to be equipped with adjustable polarization filters to reduce interference from reflected light. For example, using a highly polarized light source PLS-100, its polarization extinction ratio (PER) can reach 45dB, and the wavelength range is 1310±20nm or 1550±20nm. The polarization axis angle can be adjusted from 0 to 180 degrees to meet different detection needs; To fully cover the paint surface, images need to be collected from different angles. A robotic arm or manual adjustment device should ensure that the distance between the camera and the paint surface remains between 30 and 50 cm.

[0020] As an optional embodiment: the specific steps of step one also include: The collected image data is divided into high-resolution images and low-resolution images according to the resolution. It should be noted that the resolution is divided according to the pixel size or sampling rate of the image. The pixel size of the high-resolution image is greater than 1200×1200, and the pixel size of the low-resolution image is less than 600×600. According to the scale of the inspection target, the images are divided into two groups, where high-resolution images are used for tiny defect detection and low-resolution images are used for large-area defect detection; Perform multi-scale decomposition on high-resolution and low-resolution image data to obtain low-frequency components and high-frequency components. The specific calculation formula is: , calculate and obtain the low-frequency component ; According to the calculation formula ; ; , calculate and obtain the high-frequency component , and ; in is the low-frequency component of the last decomposition, H is a low-pass filter, G is a high-pass filter, and are low-pass filters in row and column directions, respectively. and They are high-pass filters in row and column directions respectively; According to the resolution and target scale, the decomposed low-frequency and high-frequency components are fused; The low-frequency component and the high-frequency component are fused, and the fused low-frequency and high-frequency components are subjected to inverse wavelet transform to restore the fused multi-scale image, which is used as a multi-scale image dataset.

[0021] As an optional embodiment: the specific steps of fusing the decomposed low-frequency and high-frequency components include: Low-frequency component fusion, for low-frequency components of high-resolution images , including high-resolution low-frequency components and low-resolution low-frequency components According to the formula and Calculate the energy of the low-frequency component of the high-resolution image and the energy of the low-frequency components of the low-resolution image ; According to the formula , calculate and obtain the fused low-frequency component; in ; For high-frequency component fusion, the absolute values ​​of the high-frequency components of the high-resolution image, specifically in the horizontal, vertical and diagonal directions, are calculated respectively. Similarly, the absolute values ​​of the high-frequency components of the low-resolution image are also calculated respectively. The absolute values ​​of the horizontal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused horizontal high-frequency component; the absolute values ​​of the vertical high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused vertical high-frequency component. The absolute values ​​of the diagonal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the combined diagonal high-frequency component.

[0022] As an optional embodiment: the specific working steps of performing inverse wavelet transform on the fused low-frequency and high-frequency components to restore the fused multi-scale image are as follows: The fused low-frequency component is convolved with the low-pass comprehensive filter in the row direction and the column direction to obtain the first convolution result and restore the low-frequency part of the image; The fused horizontal high-frequency component is convolved with the high-pass synthesis filter in the row direction and the low-pass synthesis filter in the column direction to obtain a second convolution result, thereby restoring the horizontal high-frequency part of the image; The fused vertical high-frequency component is convolved with the low-pass comprehensive filter in the row direction and the high-pass comprehensive filter in the column direction to obtain the third convolution result, thereby restoring the vertical high-frequency part of the image; The fused diagonal high-frequency components are convolved with the high-pass comprehensive filters in the row and column directions to obtain a fourth convolution result, thereby restoring the diagonal high-frequency part of the image; it should be noted that the low-pass comprehensive filters in the row and column directions are used to restore the low-frequency part of the image, and the high-pass comprehensive filters in the row and column directions are used to restore the high-frequency part of the image; The first convolution result, the second convolution result, the third convolution result and the fourth convolution result are added together to obtain a fused multi-scale image.

[0023] As an optional embodiment: the specific working steps of step 2 are as follows: Firstly, the texture features of tiny defects are extracted from the fused multi-scale image, and the extracted texture features and tiny defect feature library are constructed; Then, the shape features of large-area defects are extracted from the fused multi-scale images, and the extracted shape features are classified into a large-area defect feature library; Create a hierarchical feature library, which is divided into two parts: a micro-defect feature library and a large-area defect feature library. Each feature library contains the following information: feature vector, image identifier corresponding to the feature, defect type corresponding to the feature, and scale information corresponding to the feature, forming a multi-target hierarchical feature library.

[0024] As an optional embodiment: The specific steps of extracting texture features of tiny defects and shape features of large area defects are: First, texture features are extracted for tiny defects, and feature vectors reflecting texture details are obtained by analyzing the grayscale distribution and gradient information of the image, including contrast Y1, correlation Y2, energy Y3HE and homogeneity Y4. It should be noted that, in this embodiment, by selecting appropriate distance and direction parameters (such as distance of 1 pixel, direction of 0°, 45°, 90°, 135°), the grayscale co-occurrence matrix is ​​calculated, and texture features are extracted from the grayscale co-occurrence matrix, including contrast, correlation, energy, homogeneity and ASM. Then, the shape features of large-area defects are extracted, and the feature vector reflecting the shape features is obtained by calculating the geometric parameters of the contour, including the area U1, perimeter U2, aspect ratio U3 and circularity U4. It should be noted that the calculation formulas for area, perimeter, aspect ratio and circularity are existing formulas.

[0025] As an optional embodiment: the specific working steps of step three are: For minor defects, according to the formula; , calculate and obtain the texture feature score Y5 of the tiny defect, set a threshold range of tiny defects in advance (T1<T2), if the texture feature score Y5 of the tiny defect is lower than T1, the grade of the tiny defect is low, if the texture feature score Y5 of the tiny defect is greater than or equal to T1 and less than T2, the grade of the tiny defect is medium, if the texture feature score Y5 of the tiny defect is greater than or equal to T2, the grade of the tiny defect is high, and the defect grade is obtained.

[0026] As an optional embodiment: the specific working steps of step three also include: Located in a large area defect, according to the formula; , calculate and obtain the texture feature score U5 of the large area defect, set a threshold range of a large area defect in advance (T3<T4), if the texture feature score U5 of the large area defect is lower than T3, the level of the large area defect is low, if the texture feature score U5 of the large area defect is greater than or equal to T3 and less than T4, the level of the large area defect is medium, if the texture feature score U5 of the large area defect is greater than or equal to T4, the level of the large area defect is high, and the defect level is obtained; The classified defect information is integrated into a structured format, specifically defect type, defect location, defect size, and defect registration, and this structured format is used as the classified defect detection information. It should be noted that the defect type can be "minor defect" or "large area defect", the defect location and size are determined according to the actual detection results, and the defect level is calculated according to the above classification formula.

[0027] As an optional embodiment: the specific working steps of step 4 are: Extract the position information of the defect in the two-dimensional image from the defect detection information, including the center coordinates (X1, Y1) and size of the defect; According to the intrinsic and extrinsic parameters of the camera, the two-dimensional defect position is mapped to the three-dimensional model of the vehicle body. The specific steps include: Get the camera's intrinsic parameter matrix K1, ,in and is the focal length of the camera, and are the coordinates of the image center; The depth information of the defect position is obtained through a depth sensor or a binocular camera. It should be noted that the depth information provides the depth distance of the defect position relative to the camera. According to the formula Calculate and obtain the three-dimensional coordinate information of the defect ,in is the camera's appearance matrix, is the translation vector, which represents the displacement of the camera; The defect type, location, size, severity and three-dimensional coordinate information of the defect are integrated into annotation information to form complete defect annotation data, and then formatted uniformly. Finally, the formatted data is transmitted to the staff's mobile phone terminal via a wireless network.

[0028] Working principle: Use multimodal imaging equipment to collect image data of the paint surface from different angles and light source conditions, grade the collected images according to resolution and target scale, and fuse image data of different resolutions to form a multi-scale image data set; It should be noted that by combining multiple light sources (such as streak light, white light, polarized light) and imaging angles, it is possible to effectively solve problems such as high reflection and low contrast on the paint surface, enhance the visibility of defect features, and fuse image data of different resolutions to simultaneously cover tiny defects (such as pinholes and fibers) and large-area defects (such as sagging and shrinkage holes), avoiding the limitations of single-scale image data, and grading images according to resolution and target scale, which provides a basis for subsequent graded feature extraction and classification, and improves processing efficiency; According to the multi-scale image data set, hierarchical feature extraction is performed, and the extracted features are classified according to the target scale to form a multi-target hierarchical feature library; it should be noted that for image data of different scales, features suitable for the scale (such as texture features of tiny defects, shape features of large-area defects) are extracted to improve the pertinence and effectiveness of the features, and the features are classified according to the target scale, which provides rich feature support for subsequent hierarchical classification and improves the accuracy of classification; According to the multi-target hierarchical feature library, the detected defects are classified and classified, and the detection results are integrated to form hierarchical defect detection information; it should be noted that the defects are classified and classified according to the multi-scale features in the feature library, which can effectively distinguish defects of different sizes and types, improve the accuracy of classification, and integrate the results of classification to form complete detection information, which is convenient for subsequent analysis and processing; Step 4: Based on the graded defect detection information, the detected defect location information is mapped to the three-dimensional model of the vehicle body to achieve three-dimensional positioning of the defect, and multi-target graded labeling is performed according to the scale of the defect, and transmitted to the staff's mobile terminal. It should be noted that mapping the defect location information to the three-dimensional model of the vehicle body can achieve high-precision defect positioning, provide intuitive guidance for repair work, and graded labeling according to the scale of the defect can clearly display defects of different sizes and types, which is convenient for staff to quickly identify and handle.

[0029] The above are only preferred implementations of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical staff in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of this template.

Claims

1. A method for identifying quality defects at characteristic points by multi-objective hierarchical image processing, characterized in that: The following steps are involved: Step 1: Use multimodal imaging equipment to collect car paint surface image data from different angles and light source conditions, hierarchically process the collected images according to resolution and target scale, and fuse image data of different resolutions to form a multi-scale image dataset; Step 2: Based on the multi-scale image data set, perform hierarchical feature extraction, classify the extracted features according to the target scale, and form a multi-target hierarchical feature library; Step 3: Classify the detected defects according to the multi-objective classification feature library, integrate the detection results, and form the classification defect detection information; Step 4: Based on the graded defect detection information, the detected defect location information is mapped with the three-dimensional model of the vehicle body to achieve three-dimensional positioning of the defect, and multi-target graded labeling is performed according to the scale of the defect and transmitted to the staff's mobile terminal.

2. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1 is characterized in that: The specific steps of using a multimodal imaging device to collect vehicle paint surface image data from different angles and light source conditions include: First, select equipment with multimodal imaging capabilities, including high-definition cameras, streak light projectors, white light sources, and polarized light sources, and install them; Then, stripe light, white light and polarized light sources are configured respectively; During the acquisition process, the stripe light projector, white light source and polarized light source are turned on in sequence to collect horizontal and vertical stripe images, uniformly illuminated images and images under polarized light conditions, respectively, and multi-angle images of the vehicle paint surface are obtained from different angles to obtain vehicle paint surface image data.

3. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 2 is characterized in that: The specific steps of step one also include: The collected image data is divided into high-resolution images and low-resolution images according to the resolution; According to the scale of the inspection target, the images are divided into two groups, where high-resolution images are used for tiny defect detection and low-resolution images are used for large-area defect detection; Perform multi-scale decomposition on high-resolution and low-resolution image data to obtain low-frequency components and high-frequency components. The specific calculation formula is: , calculate and obtain the low-frequency component ; According to the calculation formula ; ; , calculate and obtain the high-frequency component , and ; in is the low-frequency component of the last decomposition, H is a low-pass filter, G is a high-pass filter, and are low-pass filters in row and column directions, respectively. and They are high-pass filters in row and column directions respectively; According to the resolution and target scale, the decomposed low-frequency and high-frequency components are fused; The low-frequency component and the high-frequency component are fused, and the fused low-frequency and high-frequency components are subjected to inverse wavelet transform to restore the fused multi-scale image, which is used as a multi-scale image dataset.

4. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 3 is characterized in that: The specific steps of fusing the decomposed low-frequency and high-frequency components include: Low-frequency component fusion, for low-frequency components of high-resolution images , including high-resolution low-frequency components and low-resolution low-frequency components According to the formula and Calculate the energy of the low-frequency component of the high-resolution image and the energy of the low-frequency components of the low-resolution image ; According to the formula , calculate and obtain the fused low-frequency component; in ; For high-frequency component fusion, the absolute values ​​of the high-frequency components of the high-resolution image, specifically in the horizontal, vertical and diagonal directions, are calculated respectively. Similarly, the absolute values ​​of the high-frequency components of the low-resolution image are also calculated respectively. The absolute values ​​of the horizontal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused horizontal high-frequency component; the absolute values ​​of the vertical high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the fused vertical high-frequency component. The absolute values ​​of the diagonal high-frequency components of the high-resolution and low-resolution images are compared, and the part with the larger absolute value is selected as the combined diagonal high-frequency component.

5. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 3 is characterized in that: The specific working steps of performing inverse wavelet transform on the fused low-frequency and high-frequency components to restore the fused multi-scale image are as follows: The fused low-frequency component is convolved with the low-pass comprehensive filter in the row direction and the column direction to obtain the first convolution result and restore the low-frequency part of the image; The fused horizontal high-frequency component is convolved with the high-pass synthesis filter in the row direction and the low-pass synthesis filter in the column direction to obtain a second convolution result, thereby restoring the horizontal high-frequency part of the image; The fused vertical high-frequency component is convolved with the low-pass comprehensive filter in the row direction and the high-pass comprehensive filter in the column direction to obtain the third convolution result, thereby restoring the vertical high-frequency part of the image; The fused diagonal high-frequency components are convolved with high-pass integrated filters in the row and column directions to obtain a fourth convolution result, thereby restoring the diagonal high-frequency part of the image; The first convolution result, the second convolution result, the third convolution result and the fourth convolution result are added together to obtain a fused multi-scale image.

6. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1, characterized in that: The specific working steps of step 2 are as follows: Firstly, the texture features of tiny defects are extracted from the fused multi-scale image, and the extracted texture features are used as a tiny defect feature library; Then, the shape features of large-area defects are extracted from the fused multi-scale images, and the extracted shape features are classified into a large-area defect feature library; Create a hierarchical feature library, which is divided into two parts: a micro-defect feature library and a large-area defect feature library. Each feature library contains the following information: feature vector, image identifier corresponding to the feature, defect type corresponding to the feature, and scale information corresponding to the feature, forming a multi-target hierarchical feature library.

7. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 6 is characterized in that: The specific steps of extracting texture features of tiny defects and shape features of large area defects are as follows: First, texture features are extracted for tiny defects. By analyzing the grayscale distribution and gradient information of the image, feature vectors reflecting texture details are obtained, including contrast Y1, correlation Y2, energy Y3HE homogeneity Y4; Then, shape features are extracted for large-area defects, and feature vectors reflecting shape features are obtained by calculating the geometric parameters of the contour, including area U1, perimeter U2, aspect ratio U3, and circularity U4.

8. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 1, characterized in that: The specific working steps of step three are: For minor defects, according to the formula; , calculate and obtain the texture feature score Y5 of the tiny defect, set a threshold range of tiny defects in advance (T1<T2), if the texture feature score Y5 of the tiny defect is lower than T1, the grade of the tiny defect is low, if the texture feature score Y5 of the tiny defect is greater than or equal to T1 and less than T2, the grade of the tiny defect is medium, if the texture feature score Y5 of the tiny defect is greater than or equal to T2, the grade of the tiny defect is high, and the defect grade is obtained.

9. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 8, characterized in that: The specific working steps of step three also include: Located in a large area defect, according to the formula; , calculate and obtain the texture feature score U5 of the large area defect, set a threshold range of a large area defect in advance (T3<T4), if the texture feature score U5 of the large area defect is lower than T3, the level of the large area defect is low, if the texture feature score U5 of the large area defect is greater than or equal to T3 and less than T4, the level of the large area defect is medium, if the texture feature score U5 of the large area defect is greater than or equal to T4, the level of the large area defect is high, and the defect level is obtained; The classified defect information is integrated into a structured format, specifically defect type, defect location, defect size, and defect registration, and this structured format is used as the classified defect detection information.

10. The method for identifying quality defects of characteristic points by multi-objective hierarchical image processing according to claim 8, characterized in that: The specific working steps of step 4 are: Extract the position information of the defect in the two-dimensional image from the defect detection information, including the center coordinates (X1, Y1) and size of the defect; According to the intrinsic and extrinsic parameters of the camera, the two-dimensional defect position is mapped to the three-dimensional model of the vehicle body. The specific steps include: Get the camera's intrinsic parameter matrix K1, ,in and is the focal length of the camera, and are the coordinates of the image center; Obtain the depth information of the defect location through a depth sensor or a binocular camera; According to the formula Calculate and obtain the three-dimensional coordinate information of the defect ,in is the camera's appearance matrix, is the translation vector, which represents the displacement of the camera; The defect type, location, size, severity and three-dimensional coordinate information of the defect are integrated into annotation information to form complete defect annotation data, and then formatted uniformly. Finally, the formatted data is transmitted to the staff's mobile phone terminal via a wireless network.

Images