Machine learning based online monitoring system for automotive parts spray color shade

Abstract

The application relates to the technical field of image processing, and discloses an online monitoring system for the color phase of automobile parts based on machine learning, which comprises the following steps: collecting a spraying image of a sprayed automobile part, dividing a connected domain, marking a target connected domain and a target edge pixel point, respectively determining a gradient amplitude difference, a gradient direction difference and a bubble probability of the target edge pixel point, respectively determining a direction gradient amplitude difference and a sagging probability of the target edge pixel point, establishing a feature vector of the target edge pixel point, and obtaining a spraying color phase monitoring result corresponding to the connected domain according to the feature vectors of all edge pixel points of the connected domain. The application can improve the accuracy of online monitoring of the color phase of automobile parts.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an online monitoring system for automobile accessory spraying color phase based on machine learning. BACKGROUND

[0002] The online monitoring system for automobile accessory spraying color phase can detect the spraying color of each automobile accessory in real time, discover and correct color difference problems in time, ensure that different accessories on the same vehicle or the same accessories produced in different batches have consistent colors, avoid defective and waste products caused by color phase problems, and improve the appearance and quality of the vehicle. Machine learning algorithms can learn and analyze a large amount of historical data, identify the characteristics and patterns of automobile accessory spraying color phase, and realize online monitoring of automobile accessory spraying color phase.

[0003] In the process of using machine learning algorithms to realize online monitoring of automobile accessory spraying color phase, the monitoring is generally based on the color characteristics of automobile accessories. However, when bubbles, sagging and other defects appear on the surface of automobile accessories, the gradient and brightness characteristics of the surface of automobile accessories are ignored, and only the color characteristics cannot accurately identify bubbles, sagging and other defects, often leading to inaccurate online monitoring of automobile accessory spraying color phase. SUMMARY

[0004] The present application provides an online monitoring system for automobile accessory spraying color phase based on machine learning to solve the problem of inaccurate monitoring results caused by ignoring gradient and brightness characteristics and only using color characteristics in the process of online monitoring of automobile accessory spraying color phase. The technical solution adopted is as follows:

[0005] One embodiment of the present application provides an online monitoring system for automobile accessory spraying color phase based on machine learning, which comprises the following modules:

[0006] A data acquisition module is used to acquire spraying images of automobile accessories after spraying, and divide the spraying images into multiple connected domains;

[0007] A bubble defect analysis module is used to record any one connected domain as a target connected domain, record any one edge pixel point of the target connected domain as a target edge pixel point, and determine the gradient amplitude difference, gradient direction difference and bubble probability of the target edge pixel point according to the difference between the gradient amplitude and the difference between the gradient directions of the target edge pixel point and all pixel points in the eight adjacent domains;

[0008] A sagging defect analysis module is used to determine the directional gradient amplitude difference and sagging probability of the target edge pixel point according to the difference between the gradient amplitudes in the vertical and horizontal directions of the target edge pixel point, and the difference between the gradient amplitudes of the target edge pixel point and all pixel points in the eight adjacent domains;

[0009] The spraying color phase online monitoring module is used to establish a feature vector of a target edge pixel point, and obtain a spraying color phase monitoring result corresponding to the connected domain according to the feature vectors of all edge pixel points of the connected domain.

[0010] Further, the gradient amplitude difference of the target edge pixel point is determined by:

[0011] The mean value of the absolute value of the difference between the gradient amplitude of the target edge pixel point and the gradient amplitude of all pixel points in the eight-neighbor domain is denoted as the gradient amplitude difference of the target edge pixel point.

[0012] Further, the gradient direction difference is determined by:

[0013] The mean value of the absolute value of the difference between the gradient direction of the target edge pixel point and the gradient direction of all pixel points in the eight-neighbor domain is denoted as the gradient direction difference of the target edge pixel point.

[0014] Further, the bubble probability is determined by:

[0015] The first preset weight is taken as the weight value of the normalized value of the gradient amplitude difference of the target edge pixel point, the second preset weight is taken as the weight value of the normalized value of the gradient direction difference of the target edge pixel point, and weighted summation is performed to obtain the bubble probability of the target edge pixel point.

[0016] Further, the sum of the first weight and the second weight is 1.

[0017] Further, the direction gradient amplitude difference of the target edge pixel point is determined by:

[0018] The difference between the gradient amplitude in the vertical direction and the gradient amplitude in the horizontal direction of the target edge pixel point is denoted as the direction gradient amplitude difference of the target edge pixel point.

[0019] Further, the sagging probability is determined by:

[0020] The mean value of the gradient amplitude of all pixel points in the eight-neighbor domain of the target edge pixel point is denoted as the adjacent gradient average amplitude of the target edge pixel point, and the square of the difference between the gradient amplitude of the target edge pixel point and the adjacent gradient average amplitude is denoted as the gradient amplitude variation degree of the target edge pixel point.

[0021] The sagging probability of the target edge pixel point is obtained according to the gradient amplitude variation degree and the direction gradient amplitude difference of the target edge pixel point.

[0022] Further, the specific steps of obtaining the sagging probability of the target edge pixel point according to the gradient amplitude variation degree and the direction gradient amplitude difference of the target edge pixel point include:

[0023] The preset third weight is used as the weight of the gradient magnitude change of the target edge pixel, and the preset fourth weight is used as the weight of the difference in the directional gradient magnitude of the target edge pixel. The weighted sum is then used to obtain the sag probability of the target edge pixel. The sum of the third weight and the fourth weight is 1.

[0024] Furthermore, the specific steps for establishing the feature vector of the target edge pixels are as follows:

[0025] The feature vector of the target edge pixel is obtained by sequentially arranging the pixel values ​​of the R, G, and B channels, gradient magnitude difference, gradient direction difference, bubble probability, directional gradient magnitude difference, and drip probability into a row vector.

[0026] Furthermore, the specific steps for obtaining the spraying hue monitoring result corresponding to the connected region based on the feature vectors of all edge pixels of the connected region are as follows:

[0027] The feature vectors of all edge pixels of a connected component are input into an SVM classifier to obtain the spray hue monitoring results corresponding to the connected component. The spray hue monitoring results include three types: no defects, sagging defects, and bubble defects.

[0028] The beneficial effects of this invention are:

[0029] This application evaluates the features of bubble defects on the surface of automotive parts based on the fact that the bubble's surface is raised relative to the entire painted surface by extracting the features of high-intensity reflection at the edge and center of the bubble. This determines the gradient amplitude difference, gradient direction difference, and bubble probability of the target edge pixels. Furthermore, it extracts features of run defects. Runs refer to the state where paint flows down the surface of automotive parts due to gravity during painting on a vertical surface, forming flow marks. Therefore, this application determines the directional gradient amplitude of the target edge pixels based on the feature that the gradient amplitude of the edge pixels in the corresponding run area is significantly greater in the vertical direction than in the horizontal direction. The difference in values ​​is analyzed, and then, combined with the characteristic that the edge reflectivity of the drip defect is relatively strong, causing the gradient assignment near the edge pixel of the corresponding drip region to change significantly, the drip probability of the target edge pixel is determined. Finally, based on the features of bubble defects and drip defects of each edge pixel in the extracted connected region, the feature vector of the target edge pixel is established, and based on the feature vectors of all edge pixels in the connected region, the spray hue monitoring result corresponding to the connected region is obtained. This solves the problem that only color features are used in the online monitoring of the spray hue of automotive parts, ignoring gradient and brightness features, which leads to inaccurate monitoring results, and improves the accuracy of online monitoring of the spray hue of automotive parts. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic flowchart of an online monitoring system for the hue of automotive parts spraying based on machine learning, provided in one embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram of the structure of an online monitoring system for the hue of automotive parts spraying based on machine learning, provided in one embodiment of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figure 1 The diagram illustrates a flowchart of an online monitoring system for the hue of automotive parts paint based on machine learning, according to an embodiment of the present invention. Figure 2 The diagram illustrates the structure of an online monitoring system for the hue of automotive parts spraying based on machine learning, according to an embodiment of the present invention. The system includes: a data acquisition module, a bubble defect analysis module, a sagging defect analysis module, and an online monitoring module for spraying hue.

[0035] The data acquisition module collects images of the painted automotive parts and divides the painted images into multiple connected regions.

[0036] Using a high-resolution color industrial camera with compensated direct and diffuse light sources, images of the finished automotive parts are acquired to obtain the painting images. Noise reduction is then applied to these images to minimize the interference of shadows caused by the lighting.

[0037] The role of the light source is to reduce color shift on the surface of automotive parts caused by highlights; the sprayed image is an RGB image; denoising the sprayed image is a well-known technique and will not be elaborated further. Preferably, as an embodiment of this application, Gaussian filtering is used to denoise the sprayed image. In practical applications, as other implementation methods, while achieving the goal of denoising the sprayed image, the implementer may use other existing methods such as bilateral filtering or mean filtering to denoise the sprayed image. This application does not impose any special limitations.

[0038] Connectivity analysis is performed on the spraying image to divide it into multiple connected components and obtain the connected components.

[0039] At this point, the connected components have been obtained.

[0040] The bubble defect analysis module designates any connected component as the target connected component and any edge pixel of the target connected component as the target edge pixel. Based on the differences in gradient magnitude and gradient direction between the target edge pixel and all pixels in its eight neighboring regions, it determines the gradient magnitude difference, gradient direction difference, and bubble probability of the target edge pixel.

[0041] First, the bubble defects on the surface of the car parts are analyzed. The bubbles themselves are round or nearly round, and the surface of the bubble is raised compared to the entire painted surface. Therefore, the center of the connected area corresponding to the bubble shows obvious and stronger reflection compared to other areas.

[0042] Let any connected component be designated as the target connected component. The Sobel operator is used to process the sprayed image to obtain the gradient magnitude and gradient direction of all pixels within the image. Let any edge pixel of the target connected component be designated as the target edge pixel. The average of the absolute values ​​of the differences between the gradient magnitudes of the target edge pixel and all pixels in its eight-neighborhood is designated as the gradient magnitude difference of the target edge pixel. The average of the absolute values ​​of the differences between the gradient directions of the target edge pixel and all pixels in its eight-neighborhood is designated as the gradient direction difference of the target edge pixel.

[0043] The use of the Sobel operator to obtain the gradient magnitude and gradient direction of a pixel is a well-known technique and will not be elaborated further.

[0044] The bubble probability of the target edge pixels is determined based on the differences in gradient magnitude and gradient direction of the target edge pixels.

[0045] Preferably, as an embodiment of this application, a preset first weight is used as the weight of the normalized value of the gradient magnitude difference of the target edge pixels, and a preset second weight is used as the weight of the normalized value of the gradient direction difference of the target edge pixels. The gradient magnitude difference and gradient direction difference of the target edge pixels are weighted and summed to obtain the bubble probability of the target edge pixels.

[0046] In this embodiment, the first weight and the second weight are both preset parameters, and the sum of the first weight and the second weight is 1. The values ​​of the first weight and the second weight are 0.6 and 0.4, respectively.

[0047] It should be noted that this embodiment uses the Z-Score standard normalization method to calculate the normalized value. In practical applications, implementers may use other methods of existing technology, such as the maximum-minimum normalization method or the sigmoid function, to calculate the normalized value, and no limitation is made here.

[0048] When the gradient magnitude difference of the target edge pixels is greater, the target edge pixels are more likely to correspond to the center of the bubble on the surface of the car parts. At the same time, when the gradient direction difference of the target edge pixels is greater, the target edge pixels are more likely to correspond to the edge of the bubble on the surface of the car parts. In this case, the probability of the target edge pixels being a bubble is greater, and the target edge pixels are more likely to correspond to the bubble position on the surface of the car parts.

[0049] The gradient magnitude difference, gradient direction difference, and bubble probability of each edge pixel in each connected region can be obtained using the same method.

[0050] At this point, the gradient magnitude difference, gradient direction difference, and bubble probability of each edge pixel in each connected region are obtained.

[0051] The drip defect analysis module determines the directional gradient magnitude difference and drip probability of the target edge pixel based on the difference in gradient magnitude between the vertical and horizontal directions of the target edge pixel, and the difference in gradient magnitude between the target edge pixel and all pixels in its eight neighborhoods.

[0052] This study analyzes the run defects on the surface of automotive parts. Runs refer to the condition where paint, during painting on a vertical surface, flows down the surface due to gravity, forming run marks. When a connected component corresponds to a run area on an automotive part, the gradient amplitude of the edge pixels of the connected component is significantly greater in the vertical direction than in the horizontal direction. The gradient direction of the edge pixels of the connected component is relatively concentrated. Simultaneously, the edge reflectivity of run defects is strong, and the gradient values ​​near the edge pixels of the connected component change considerably.

[0053] The Sobel operator is used to obtain the gradient magnitudes of all pixels in the sprayed image in the horizontal and vertical directions.

[0054] The sag probability of the target edge pixel is determined based on the difference in gradient magnitude between the vertical and horizontal directions of the target edge pixel, and the difference in gradient magnitude between the target edge pixel and all pixels in its eight-neighborhood.

[0055] The difference between the gradient magnitudes of the target edge pixel in the vertical and horizontal directions is denoted as the directional gradient magnitude difference of the target edge pixel. The mean of the gradient magnitudes of all pixels within the eight neighborhoods of the target edge pixel is denoted as the average magnitude of the neighboring gradients of the target edge pixel. The square of the difference between the gradient magnitude of the target edge pixel and the average magnitude of the neighboring gradients is denoted as the gradient magnitude change of the target edge pixel. A preset third weight is used as the weight of the gradient magnitude change of the target edge pixel, and a preset fourth weight is used as the weight of the directional gradient magnitude difference of the target edge pixel. The gradient magnitude change and the directional gradient magnitude difference of the target edge pixel are weighted and summed to obtain the sag probability of the target edge pixel.

[0056] The third and fourth weights are preset parameters, and the sum of the third and fourth weights is 1. In this embodiment, the values ​​of the third and fourth weights are 0.5 and 0.5, respectively.

[0057] When the difference in the directional gradient magnitude of the target edge pixels is greater, the target pixel will more significantly exhibit the characteristic that the gradient magnitude in the vertical direction is significantly greater than that in the horizontal direction. At the same time, when the gradient magnitude of the target edge pixels changes more, the target edge pixels will exhibit the characteristic of stronger reflectivity, and the target pixel will be more likely to correspond to the edge of the drip defect. In this case, the drip probability of the target edge pixels is greater.

[0058] The same method can be used to obtain the difference in directional gradient magnitude and the probability of sag for each edge pixel in each connected region.

[0059] At this point, the difference in directional gradient magnitude and the probability of sag are obtained for each edge pixel in each connected region.

[0060] The online monitoring module for spray coating hue establishes the feature vectors of target edge pixels and obtains the spray coating hue monitoring results corresponding to the connected region based on the feature vectors of all edge pixels of the connected region.

[0061] The feature vector of the target edge pixel is obtained by sequentially arranging the pixel values ​​of the R, G, and B channels, gradient magnitude difference, gradient direction difference, bubble probability, directional gradient magnitude difference, and drip probability into a row vector.

[0062] The same method can be used to obtain the difference in directional gradient magnitude and the probability of sag for each edge pixel in each connected region.

[0063] Following the same method, 1000 images of painted automotive parts were collected, including those without defects, those with sagging defects, and those with bubble defects. Feature vectors of all edge pixels within the connected components of each image were obtained. The painting hue monitoring results corresponding to each connected component were manually labeled, including three categories: no defects, sagging defects, and bubble defects. The feature vectors of all edge pixels within the connected components of 700 collected images were used as the training set, and the feature vectors of all edge pixels within the connected components of 300 images were used as the test set to train an SVM classifier, resulting in the trained SVM classifier.

[0064] Training the SVM classifier and obtaining the trained SVM classifier are well-known techniques and will not be elaborated further.

[0065] Input the feature vectors of all edge pixels of a connected component into a trained SVM classifier to obtain the spray hue monitoring results corresponding to the connected component.

[0066] This enables online monitoring of the hue of paint applied to automotive parts.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An online monitoring system for automotive parts spray color phase based on machine learning, characterized by, The system comprises the following modules: A data acquisition module is configured to acquire a spraying image of a sprayed automobile part, and divide the spraying image into a plurality of connected domains; A bubble defect analysis module is configured to record any one connected domain as a target connected domain, record any one edge pixel point of the target connected domain as a target edge pixel point, determine a gradient amplitude difference, a gradient direction difference and a bubble probability of the target edge pixel point according to differences between the gradient amplitude and the gradient direction of the target edge pixel point and all pixel points in an eight-neighbor domain, including: recording a mean value of absolute values of differences between the gradient direction of the target edge pixel point and all pixel points in the eight-neighbor domain as the gradient direction difference of the target edge pixel point; a first preset weight is taken as a weight value of a normalized value of the gradient amplitude difference of the target edge pixel point, a second preset weight is taken as a weight value of a normalized value of the gradient direction difference of the target edge pixel point, and a weighted sum is performed to obtain the bubble probability of the target edge pixel point; A sagging defect analysis module is configured to determine a directional gradient amplitude difference and a sagging probability of the target edge pixel point according to a difference between gradient amplitudes of the target edge pixel point in a vertical direction and a horizontal direction and a difference between the gradient amplitude of the target edge pixel point and all pixel points in the eight-neighbor domain, including: recording a difference between the gradient amplitudes of the target edge pixel point in the vertical direction and the horizontal direction as the directional gradient amplitude difference of the target edge pixel point; recording a mean value of gradient amplitudes of all pixel points in the eight-neighbor domain of the target edge pixel point as a neighboring gradient average amplitude, recording a square of a difference between the gradient amplitude of the target edge pixel point and the neighboring gradient average amplitude as a gradient amplitude variation degree of the target edge pixel point; and obtaining the sagging probability of the target edge pixel point according to the gradient amplitude variation degree and the directional gradient amplitude difference of the target edge pixel point. A spraying color phase online monitoring module is configured to establish a feature vector of the target edge pixel point, and obtain a spraying color phase monitoring result corresponding to the connected domain according to the feature vectors of all edge pixel points of the connected domain.

2. The machine learning based online monitoring system for automobile parts spray color shade as claimed in claim 1 wherein, The determination method of the gradient amplitude difference of the target edge pixel point comprises the following steps: The mean value of absolute values of differences between the gradient amplitude of the target edge pixel point and all pixel points in the eight-neighbor domain is recorded as the gradient amplitude difference of the target edge pixel point.

3. The machine learning based online monitoring system for automobile parts spray color shade as claimed in claim 1 wherein, The sum of the first weight and the second weight is 1.

4. The machine learning based online monitoring system for automobile parts spray color shade as claimed in claim 1 wherein, The specific steps of obtaining the sagging probability of the target edge pixel point according to the gradient amplitude variation degree and the directional gradient amplitude difference of the target edge pixel point comprise the following steps: A third preset weight is taken as a weight value of the gradient amplitude variation degree of the target edge pixel point, a fourth preset weight is taken as a weight value of the directional gradient amplitude difference of the target edge pixel point, a weighted sum is performed to obtain the sagging probability of the target edge pixel point, and the sum of the third weight and the fourth weight is 1.

5. The machine learning based online monitoring system for automobile parts painting hue according to claim 1, wherein, The specific steps of establishing the feature vector of the target edge pixel point comprise the following steps: The pixel values of the target edge pixel point in R, G and B channels, the gradient amplitude difference, the gradient direction difference, the bubble probability, the direction gradient amplitude difference and the sagging probability are sequentially arranged as a row vector to obtain a feature vector of the target edge pixel point.

6. The machine learning based online monitoring system for automobile parts spray color shade as claimed in claim 1 wherein, The spraying color phase monitoring result corresponding to the connected domain is obtained according to the feature vectors of all edge pixel points of the connected domain, and the specific steps include the following steps: The feature vectors of all edge pixel points of a connected domain are input into an SVM classifier to obtain a spraying color phase monitoring result corresponding to the connected domain, and the spraying color phase monitoring result includes three types of no defect, sagging defect and bubble defect.

Images