Intelligent segmentation method and system of spraying area based on visual radar

Abstract

The present invention relates to the field of image processing technology, and specifically to a method and system for intelligent segmentation of a spraying area based on a visual radar, including: obtaining a paint image to be detected; obtaining suspected paint defect pixels, obtaining paint defect pixels, and obtaining a paint defect area; obtaining the sum of the paint defect area and the line segment according to the gravity direction, and obtaining the sagging characteristics of the paint defect area; obtaining multiple key straight lines; using the key straight lines to pass through all the pixels in each paint defect area, and obtaining the bubble expression of the paint defect area; obtaining the possibility that the paint defect area is a sagging area, and obtaining the paint defect area and the sagging area. The present invention aims to solve the problem that when the sagging area and the bubble area in the paint image are obtained by an edge detection algorithm, a normal area will be obtained, and it cannot be determined whether the detected defect area is a sagging area or a bubble area.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for intelligently segmenting a spraying area based on a visual radar. Background Art

[0002] Visual radar is a sensor system that combines lidar and cameras. It includes sensors such as lidar and cameras. Lidar is used to produce high-precision three-dimensional maps, detect and track surrounding obstacles and objects; cameras are used to capture images and identify objects through image processing and analysis. Car paint can protect the car body from corrosion, so the quality of the car paint after spraying is tested. When painting a car, there is a high possibility of sagging or bubbling. Because the sagging area or bubble area in the car paint reflects different problems in the spraying process, and the processing methods of the sagging area and the bubble area are also different. Therefore, after obtaining the paint defect area containing the sagging area and the bubble area in the car paint, the sagging area is distinguished from the bubble area.

[0003] Since the grayscale values ​​of the pixels in the sagging area and bubble area are significantly different from those in the normal area, the edge detection algorithm is used to obtain the sagging area and bubble area in the paint. However, due to the structural problems of the vehicle, there are normal protrusions on the vehicle, which causes the grayscale values ​​of some normal area pixels in the vehicle to be detected to be different from the grayscale values ​​of other normal area pixels. When the sagging area and bubble area in the paint are obtained by the edge detection algorithm, the normal area may be regarded as a paint defect area, and the existing algorithm cannot distinguish the bubble area from the sagging area; therefore, the edge detection algorithm needs to be optimized. Summary of the invention

[0004] The present invention provides a method and system for intelligently segmenting a spraying area based on a visual radar to solve the existing problems.

[0005] The method and system for intelligent segmentation of spraying area based on visual radar of the present invention adopt the following technical solutions:

[0006] The present invention proposes a method for intelligent segmentation of a spraying area based on a visual radar, the method comprising the following steps:

[0007] Obtain the paint image of the vehicle to be detected, the gravity direction of the vehicle to be detected, and the light source direction;

[0008] According to the gradient of the pixel points, the suspected paint defect pixel points are obtained, and then the suspected paint defect area and the local area of ​​the suspected paint defect pixel points are obtained; according to the distribution of the gradient values ​​of the suspected paint defect pixel points in the suspected paint defect area and the distribution of the gradient values ​​of the pixel points in the local area of ​​the suspected paint defect pixel points, the paint defect characteristics of the suspected paint defect pixel points are obtained, and then the paint defect area is obtained;

[0009] According to the gravity direction, all straight lines passing through each paint defect area are obtained to obtain the sum line segment of each paint defect area; according to the difference in grayscale values ​​of adjacent pixels in the sum line segment of each paint defect area and the difference in the gradient direction of the pixels in each paint defect area, the sagging characteristics of each paint defect area are obtained;

[0010] The direction of the light source is rotated to obtain multiple key directions, and then multiple key rays are obtained; according to the distribution of the grayscale values ​​of the pixels in the same paint defect area on each key ray and the parallel ray, the grayscale expression degree, dividing line, reflective area and light-facing area of ​​each paint defect area in each key direction are obtained; according to the grayscale expression degree of each paint defect area in each key direction, the length of the dividing line and the grayscale distribution of the pixels in the reflective area and the light-facing area, the bubble characteristics of the paint defect area in each key direction are obtained; the bubble expression degree of each paint defect area is obtained by combining the grayscale expression degree of the paint defect area in each key direction;

[0011] According to the sagging characteristics, bubble expression and shape of each paint defect area, the possibility of each paint defect area being a sagging area is obtained, and then the sagging area and the bubble area are obtained.

[0012] Furthermore, the method of obtaining the suspected paint defect pixel point according to the gradient of the pixel point, and then obtaining the suspected paint defect area and the local area of ​​the suspected paint defect pixel point includes the following specific methods:

[0013] Obtain the gradient of each pixel in the paint image to be inspected, and record the pixel with a gradient value greater than 0 as a suspected paint defect pixel;

[0014] All suspected paint defect pixels are used to form a number of suspected paint defect regions; the i-th suspected paint defect pixel and the suspected paint defect pixels in the 8-neighborhood are used as elements in the local region of the i-th suspected paint defect pixel to obtain the local region of the i-th suspected paint defect pixel;

[0015] The information entropy of the gradient values ​​of all suspected paint defect pixels in the suspected paint defect area where the i-th suspected paint defect pixel is located is recorded as the gradient information entropy of the i-th suspected paint defect pixel; the variance of the gradient values ​​of all pixels in the local area of ​​the i-th suspected paint defect pixel is recorded as the gradient variance of the i-th suspected paint defect pixel.

[0016] Further, the paint defect feature of the suspected paint defect pixel is obtained according to the distribution of the gradient value of the suspected paint defect pixel in the suspected paint defect area and the distribution of the gradient value of the pixel in the local area of ​​the suspected paint defect pixel, and then the paint defect area is obtained, including the specific method of:

[0017] J i =sigmoid(H i ×σ i )

[0018] In the formula, J i represents the paint defect feature of the i-th suspected paint defect pixel, H i represents the gradient information entropy of the i-th suspected paint defect pixel, σ i represents the gradient variance of the i-th suspected paint defect pixel, sigmoid() represents the sigmoid function;

[0019] A defect threshold T1 is preset, and suspected paint defect pixels whose paint defect characteristics are greater than the defect threshold T1 are recorded as paint defect pixels;

[0020] All the paint defect pixels are combined to form a number of paint defect areas.

[0021] Furthermore, the method of obtaining all straight lines passing through each paint defect area according to the gravity direction to obtain the sum of line segments of each paint defect area includes the following specific methods:

[0022] According to the direction of gravity, all straight lines passing through each paint defect area are obtained;

[0023] In the fth straight line passing through the bth paint defect area, remove the pixel points that are not in the bth paint defect area, and divide the fth straight line passing through the bth paint defect area into a plurality of line segments;

[0024] In the fth straight line that passes through the bth car paint defect area, each line segment is connected end to end with its adjacent line segment to obtain a sum line segment, which is recorded as the fth sum line segment of the bth car paint defect area;

[0025] The fth line segment of the bth paint defect area is recorded as the target line segment.

[0026] Furthermore, the sagging feature of each paint defect area is obtained according to the difference in grayscale values ​​of adjacent pixels in each paint defect area and line segment and the difference in gradient direction of pixels in each paint defect area, including the specific method of:

[0027]

[0028] In the formula, R b represents the sagging characteristics of the b-th paint defect area, n b represents the sum of the number of straight lines where the target line segment passes through the b-th paint defect area and its parallel straight lines, g f，b Indicates the number of pixels contained in the target line segment, C b，f，d+1 Represents the gray value of the d+1th pixel in the target line segment, C b，f，d Represents the gray value of the dth pixel in the target line segment, σ b represents the variance of the gradient direction of all pixels in the b-th paint defect area, || represents the absolute value function, exp() is the exponential function with a natural constant as the base, and s.gn() represents the sign function.

[0029] Furthermore, the light source direction is rotated to obtain multiple focus directions, and then multiple focus rays, including the specific method of:

[0030] Rotate the light source direction to the left to obtain m3 key directions, and rotate the light source direction to the right to obtain m4 key directions, and record the light source direction as the key direction; wherein m3 represents the number of directions obtained by rotating the preset light source direction to the left, and m4 represents the number of directions obtained by rotating the preset light source direction to the right;

[0031] According to the zth key direction, the zth key ray is obtained.

[0032] Furthermore, the grayscale representation, boundary line, reflective area and light-facing area of ​​each paint defect area in each key direction are obtained according to the distribution of the grayscale values ​​of the pixels in the same paint defect area on each key ray and the parallel ray, including the specific method of:

[0033] The zth key ray is obtained through the zth key direction; the zth key ray passing through the bth paint defect area and the parallel ray are recorded as the important ray in the zth key direction of the bth paint defect area;

[0034] The maximum absolute value of the difference between the grayscale values ​​of two adjacent pixels in the rth important ray in the zth key direction of the bth paint defect area, and the pixel corresponding to the maximum grayscale value of the two corresponding pixels, is recorded as the important pixel of the rth important ray in the zth key direction in the bth paint defect area;

[0035] The absolute value of the difference between the grayscale values ​​of the important pixel point and the adjacent next pixel point in the rth important ray in the zth key direction in the bth paint defect area is recorded as the grayscale representation degree of the rth important ray in the zth key direction in the bth paint defect area; the average value of the grayscale representation degrees of all important rays in the zth key direction in the bth paint defect area is recorded as the grayscale representation degree of the bth paint defect area in the zth key direction;

[0036] Connect the important pixel points in each important ray in the zth key direction of the bth paint defect area with the important pixel points in the adjacent important rays around it to obtain the boundary line of the bth paint defect area in the zth key direction;

[0037] Several area blocks are obtained through the boundary line of the bth paint defect area in the zth focus direction and the edge of the bth paint defect area; the area block with the largest average grayscale value among all the area blocks of the bth paint defect area is recorded as the light-facing area of ​​the bth paint defect area in the zth focus direction, and the other area blocks in the bth paint defect area except the zth light-facing area are recorded as the zth reflective area of ​​the bth paint defect area in the zth focus direction.

[0038] Furthermore, the bubble characteristics of the paint defect area in each key direction are obtained according to the grayscale expression of each paint defect area in each key direction, the length of the dividing line, and the grayscale distribution of the pixels in the reflective area and the light-facing area; the bubble expression of each paint defect area is obtained in combination with the grayscale expression of the paint defect area in each key direction, including the specific method of:

[0039] The ratio of the number of pixels whose grayscale value in the reflective area of ​​the b-th paint defect area in the z-th key direction is greater than the minimum grayscale value in the light-facing area of ​​the b-th paint defect area in the z-th key direction to the number of pixels contained in the reflective area of ​​the b-th paint defect area in the z-th key direction is recorded as the credibility of the b-th paint defect area in the z-th key direction; the ratio of the length of the boundary line of the b-th paint defect area in the z-th key direction to the number of important rays of the b-th paint defect area in the z-th key direction is recorded as the reliability of the b-th paint defect area in the z-th key direction;

[0040] The specific calculation formula for obtaining the bubble characteristics of the b-th paint defect area in the z-th key direction is as follows:

[0041]

[0042] Where W z，b represents the bubble characteristics of the b-th paint defect area in the z-th key direction, It represents the grayscale expression of the b-th paint defect area in the z-th key direction. represents the reliability of the b-th paint defect area in the z-th key direction, α z，b represents the credibility of the b-th paint defect area in the z-th key direction, and exp() is an exponential function with a natural constant as the base;

[0043] According to the bubble characteristics of the b-th paint defect area in all key directions, the specific calculation formula for obtaining the bubble expression degree of the b-th paint defect area is as follows:

[0044]

[0045] Where W b represents the bubble expression degree of the bth paint defect area, m5 represents the number of key directions, W represents the grayscale representation of the b-th paint defect area in the z-th key direction, z，b represents the bubble feature of the b-th paint defect area in the z-th key direction, softmax() represents the weight normalization function, and sigmoid() represents the sigmoid function.

[0046] Furthermore, the possibility of each paint defect area being a sagging area is obtained according to the sagging characteristics, bubble expression and shape of each paint defect area, and then the sagging area and bubble area are obtained, including the specific method of:

[0047] The circularity of each paint defect area is obtained. According to the circularity, sagging characteristics and bubble expression of each paint defect area, the calculation formula for calculating the possibility that each paint defect area is a sagging area is as follows:

[0048]

[0049] Where P b W represents the probability that the b-th paint defect area is a sagging area. b Indicates the bubble performance of the b-th paint defect area, R b represents the sagging characteristics of the b-th paint defect area, S b represents the circularity of the b-th paint defect area, sigmoid() represents the sigmoid function;

[0050] A possibility threshold T2 is preset, and a paint defect area whose possibility of being a sagging area is greater than the possibility threshold T2 is recorded as a sagging area; a paint defect area whose possibility of being a sagging area is less than or equal to the possibility threshold T2 is recorded as a bubble area.

[0051] The present invention also proposes a spray area intelligent segmentation system based on visual radar, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor executes the computer program to implement the steps of the above method.

[0052] The beneficial effects of the technical solution of the present invention are as follows: in the process of obtaining the sagging area and the bubble area in the to-be-detected vehicle paint, the present invention uses the feature that the area formed by the normal pixels whose gradient values ​​are not 0 is intentionally formed by the automobile manufacturer, so that the gradient of the normal pixels whose gradient values ​​are not 0 is relatively similar to that of the surrounding normal pixels, and obtains the paint defect feature of each paint defect pixel through the difference in the gradient value between each paint defect pixel and the surrounding paint defect pixels, so that the paint defect feature of each paint defect pixel reflects the possibility of whether the pixel is intentionally formed by the manufacturer, and then removes the normal pixels detected by the edge detection algorithm to obtain the paint defect area; based on the feature that the sagging area is affected by gravity to become thin at the top and thick at the bottom, so that the grayscale value of the pixels in the sagging area changes in one direction, the sagging feature of each paint defect area is obtained according to the change of the grayscale value of the pixels in each paint defect area along the gravity direction, so that the sagging feature of each paint defect area reflects The distribution of the grayscale values ​​of the pixels in the defect area is similar to the feature that the upper part is thin and the lower part is thick in the sagging area; according to the fact that the bubble is a protrusion on the paint surface, the bubble area is composed of a darker area and a brighter area, resulting in a feature that the grayscale difference of the pixels at the intersection of the darker area and the brighter area in the bubble area is extremely large, and the bubble expression degree of each paint defect area is obtained through the grayscale difference between each pixel point and a pixel point around it in each paint defect area, so that the bubble expression degree of each paint defect area reflects the feature similarity of the grayscale value difference between the area and some pixels in the bubble area and a pixel point around it; the inverse of the circularity of the paint defect area is used as the weight of the sagging feature of the paint defect area, which increases the difference between the possibility of the sagging area and the bubble area being the sagging area, reduces the possibility of incorrectly segmenting the sagging area and the bubble area when the threshold is set unreasonably, and solves the problem that the existing edge detection algorithm cannot obtain the sagging area and the bubble area in the paint to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 The present invention is a flowchart of the steps of the method for intelligent segmentation of the spraying area based on visual radar;

[0055] Figure 2 is the image of the detected normal area;

[0056] Figure 3 is an image of the sagging area;

[0057] Figure 4 is the image of the bubble area;

[0058] Figure 5 An image showing the relationship between the light source and the bubble area. DETAILED DESCRIPTION

[0059] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of the spray area intelligent segmentation method and system based on visual radar proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0060] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0061] The specific scheme of the method and system for intelligent segmentation of spraying areas based on visual radar provided by the present invention is described in detail below with reference to the accompanying drawings.

[0062] See also Figure 1 , which shows a flowchart of a method for intelligent segmentation of a spraying area based on a visual radar provided by an embodiment of the present invention, the method comprising the following steps:

[0063] Step S001: Acquire the paint image of the vehicle to be detected, the gravity direction of the vehicle to be detected, and the light source direction.

[0064] This embodiment is to obtain the bubble area and the sagging area in the paint of the vehicle to be detected, thereby obtaining the paint image of the vehicle to be detected.

[0065] Specifically, the paint image of the vehicle to be tested for sagging and bubble phenomena is obtained through the camera in the visual radar, and is recorded as the paint image of the vehicle to be tested. The paint image of the vehicle to be tested is grayed, and the grayed image is Gaussian filtered. The image after Gaussian filtering is recorded as the paint image to be tested. The gravity direction and the light source direction of the vehicle to be tested are obtained. Among them, if the light bulb is above the vehicle to be tested, the light source direction to be tested is from top to bottom. Among them, the paint of the vehicle to be tested does not contain paint defect areas caused by other factors except the paint defect areas caused by the sagging phenomenon and the paint defect areas caused by the bubble phenomenon.

[0066] Step S002: According to the gradient of the pixel points, the suspected paint defect pixel points are obtained, and then the suspected paint defect area and the local area of ​​the suspected paint defect pixel points are obtained; according to the distribution of the gradient values ​​of the suspected paint defect pixel points in the suspected paint defect area and the distribution of the gradient values ​​of the pixel points in the local area of ​​the suspected paint defect pixel points, the paint defect characteristics of the suspected paint defect pixel points are obtained, and then the paint defect area is obtained.

[0067] It should be noted that since the grayscale values ​​of pixels in the bubble area and sagging area are significantly different from those in the normal area, the canny edge detection algorithm is used to obtain the bubble area and sagging area in the paint image to be detected. Due to the structural problems of the car, the grayscale values ​​of pixels in some normal areas of the paint image to be detected are different from those in other normal areas. Therefore, when the bubble area and sagging area in the paint image to be detected are obtained by the existing canny edge detection algorithm, a normal area will be obtained. Therefore, the canny edge detection algorithm is modified. Among them, the normal areas that can be detected by the edge detection algorithm due to the structural problems of the car are as follows: Figure 2 As shown. The specific steps of the Canny edge detection algorithm to obtain multiple edges in an image are as follows: Gaussian filtering is performed on the paint image to be detected, the gradient of each pixel in the paint image to be detected is obtained, non-maximum suppression is performed, and suspected paint defect pixels are obtained. Through double thresholds, noise points in the suspected paint defect pixels are removed, and isolated weak edges are eliminated to obtain the final edges. This embodiment improves the steps after obtaining the gradient of each pixel in the paint image to be detected in the Canny edge detection algorithm. In this embodiment, the pixels whose gradient values ​​obtained by the Canny operator are not 0 are recorded as suspected paint defect pixels.

[0068] It should be further explained that when the canny edge detection algorithm removes normal pixels from suspected paint defect pixels, since the normal area in the car is intentionally formed by the car manufacturer, the gradient values ​​and gradient directions of the pixels in the normal area composed of suspected paint defect pixels are relatively similar. Therefore, the suspected paint defect area is obtained based on the suspected paint defect pixels, and then the paint defect characteristics of each suspected paint defect pixel are obtained based on the distribution of pixel gradients in the suspected paint defect area where the suspected paint defect pixel is located, and then the normal pixels in the suspected paint defect pixels are removed to obtain paint defect pixels, and then the paint defect area is obtained.

[0069] Specifically, the gradient of each pixel in the vehicle paint image to be detected is obtained through the canny operator. The canny operator is a well-known technology and will not be described in detail in this embodiment. Pixels with gradient values ​​greater than 0 are recorded as suspected vehicle paint defect pixels. All suspected vehicle paint defect pixels are formed into several suspected vehicle paint defect regions, which are obtained by the connected domain analysis method. This is a well-known technology and will not be described in detail in this embodiment. The i-th suspected vehicle paint defect pixel and the suspected vehicle paint defect pixel in the 8-neighborhood are taken as elements in the local region of the i-th suspected vehicle paint defect pixel to obtain the local region of the i-th suspected vehicle paint defect pixel; it should be noted that if the 8-neighborhood of the suspected vehicle paint defect pixel exceeds the boundary of the vehicle paint image to be detected, the actual existing pixels are used for analysis and the local region is obtained.

[0070] Further, the information entropy of the gradient values ​​of all suspected paint defect pixels in the suspected paint defect area where the i-th suspected paint defect pixel is located is recorded as the gradient information entropy of the i-th suspected paint defect pixel. The variance of the gradient values ​​of all pixels in the local area of ​​the i-th suspected paint defect pixel is recorded as the gradient variance of the i-th suspected paint defect pixel. The method for obtaining information entropy is a well-known technology and is not described in this embodiment.

[0071] Furthermore, the specific calculation formula for obtaining the paint defect feature of the i-th suspected paint defect pixel is as follows:

[0072] J i =sigmoid(H i ×σ i )

[0073] In the formula, J i represents the paint defect feature of the i-th suspected paint defect pixel, H i represents the gradient information entropy of the i-th suspected paint defect pixel, σ i represents the gradient variance of the i-th suspected paint defect pixel, and sigmoid() represents the sigmoid function, which is used for normalization processing.

[0074] It should be noted that H i The larger the value of , the more disordered the distribution of the gradient value of the suspected paint defect pixel in the suspected paint defect area where the i-th suspected paint defect pixel is located is, which further indicates that the possibility that the suspected paint defect area where the i-th suspected paint defect pixel is located is intentionally formed by the automobile manufacturer is smaller, that is, the paint defect feature of the i-th suspected paint defect pixel is larger; σ i The larger the value is, the greater the difference in gradient value between each suspected paint defect pixel and other suspected paint defect pixels in the local area of ​​the ith suspected paint defect pixel is, indicating that the possibility that the local area of ​​the ith suspected paint defect pixel is intentionally formed by the automobile manufacturer is smaller, that is, the paint defect feature of the ith suspected paint defect pixel is greater.

[0075] Furthermore, a defect threshold T1 is preset, and the suspected paint defect pixels whose paint defect characteristics are greater than the defect threshold T1 are recorded as paint defect pixels. Among them, the defect threshold T1=0.4 preset in this embodiment is described as an example, and other values ​​can be set in other implementations. All paint defect pixels are formed into several paint defect areas, and the formation of several paint defect areas based on paint defect pixels relies on a connected domain analysis algorithm, which is a prior art and will not be repeated in this embodiment.

[0076] Step S003: According to the direction of gravity, all straight lines passing through each paint defect area are obtained to obtain the sum line segment of each paint defect area; according to the difference in grayscale values ​​of adjacent pixels in the sum line segment of each paint defect area and the difference in gradient direction of pixels in each paint defect area, the sagging characteristics of each paint defect area are obtained.

[0077] It should be noted that the paint defect area in the paint image to be detected may be a paint defect area caused by sagging factors or a paint defect area caused by bubbles, and the two situations reflect different problems in the paint spraying process, and the paint defect areas caused by the two situations are also handled differently. Therefore, the paint defect area is divided into the paint defect area caused by sagging factors and the paint defect area caused by bubbles, that is, the paint defect area is divided into the sagging area and the bubble area.

[0078] It should be further explained that the generation of the sagging area is affected by gravity, making the upper part of the sagging area thinner and the lower part thicker, that is, the gray value of the pixel points in the sagging area gradually increases when looking down along the gravity direction, and at the same time, the gradient direction of the pixel points in the sagging area is relatively similar. Therefore, according to the distribution of the gray value of the pixel points in each paint defect area and the distribution of the gradient direction, the sagging characteristics of each paint defect area are obtained. Among them, the sagging area is as follows: Figure 3 shown.

[0079] Specifically, according to the direction of gravity, all straight lines passing through each paint defect area are obtained. When a straight line passes through a paint defect area, it passes through the paint defect area along the direction of gravity. That is, when the gravity direction in the paint image to be detected is from top to bottom, the straight line first passes through the upper pixel point in the bth paint defect area and then passes through the lower pixel point in the bth paint defect area.

[0080] Furthermore, in the fth straight line passing through the bth paint defect area, the pixels not in the bth paint defect area are removed, and the fth straight line passing through the bth paint defect area is divided into several line segments. In the fth straight line passing through the bth paint defect area, each line segment is connected end to end with its adjacent line segment to obtain a sum line segment, which is recorded as the fth sum line segment of the bth paint defect area, that is, several line segments on the same straight line are spliced ​​into a sum line segment. The fth sum line segment of the bth paint defect area is recorded as the target line segment.

[0081] Furthermore, the specific calculation formula for obtaining the sagging characteristics of the b-th paint defect area is as follows:

[0082]

[0083] In the formula, R b represents the sagging characteristics of the b-th paint defect area, n b represents the sum of the number of straight lines where the target line segment passes through the b-th paint defect area and its parallel straight lines, g f,b Indicates the number of pixels contained in the target line segment, C b，f，d+1 Represents the gray value of the d+1th pixel in the target line segment, C b，f，d Represents the gray value of the dth pixel in the target line segment, σ b represents the variance of the gradient direction of all pixels in the b-th paint defect area, || represents the absolute value function, exp() is an exponential function with a natural constant as the base, and this embodiment adopts the exp(--x) model to present the inverse proportional relationship and normalization processing, x is the input of the model, and the implementer can set the inverse proportional function and normalization function according to the actual situation; sgn() represents the sign function, when the input is greater than 0, the output is 1, when the input is equal to 0, the output is 0, and when the output is less than 0, the output is -1.

[0084] It should be noted that sgn(C b，f，d+1 -C b，f，d )=1, indicating that the gray value of the d+1th pixel in the target line segment is greater than the gray value of the dth pixel, further indicating that the dth pixel in the target line segment meets the characteristic that the gray value of the pixel in the sagging area gradually increases along the direction of gravity, that is, the greater the sagging feature of the bth paint defect area, the greater the gray value of the d+1th pixel in the target line segment. In other cases The larger the value of is, the more pixels in the target line segment have gray values ​​smaller than the gray value of the next pixel, which further indicates that the pixels in the bth paint defect area in the target straight line that passes through the bth paint defect area for the fth time are more consistent with the characteristics of the sagging area with a thin top and a thick bottom, that is, the larger the sagging characteristics of the bth paint defect area are. The larger the value of , the more pixels in the b-th paint defect area are smaller than the grayscale value of the next pixel, which further indicates that the b-th paint defect area is more consistent with the characteristics of the sagging area with a thin top and a thick bottom; b The smaller the value is, the more similar the gradient directions of the pixels in the b-th paint defect area are, which further indicates that the b-th paint defect area is more consistent with the feature that the gradient directions of the pixels in the sagging area are relatively similar, that is, the greater the sagging feature of the b-th paint defect area.

[0085] At this point, the sagging characteristics of each paint defect area are obtained.

[0086] Step S004: Rotate the direction of the light source to obtain multiple key directions, and then obtain multiple key rays; obtain the grayscale expression, dividing line, reflective area and light-facing area of ​​each paint defect area in each key direction according to the distribution of grayscale values ​​of pixels in the same paint defect area on each key ray and the parallel ray; obtain the bubble characteristics of the paint defect area in each key direction according to the grayscale expression, the length of the dividing line and the grayscale distribution of pixels in the reflective area and the light-facing area of ​​each paint defect area in each key direction; obtain the bubble expression of each paint defect area in combination with the grayscale expression of the paint defect area in each key direction.

[0087] It should be noted that when inspecting the paint of the car to be inspected, since the bubbles are raised on the paint of the car to be inspected, a part of the bubble blocks part of the light from another part, so that a part of the bubble affects the grayscale value of another part, which results in a larger grayscale value of pixels in a part of the bubble and a smaller grayscale value of pixels in another part, a larger difference in grayscale value between some pixels in the bubble and the surrounding pixels, and a shorter boundary between the area with a larger grayscale value and the area with a smaller grayscale value in the bubble. The bubble area on the paint of the vehicle is as follows: Figure 4 Therefore, a straight line is drawn according to the light source direction of the bubble to obtain the dividing line in each paint defect area, and the light-facing area and the reflective area of ​​each paint defect area are obtained. According to the number of pixel gray values ​​in the reflective area that are greater than the light-facing area, the difference in gray values ​​between some pixels and surrounding pixels, and the length of the dividing line, the bubble expression degree of each defect is obtained.

[0088] It should be further explained that when taking pictures of the car to be inspected, when the light is above the car, from the perspective of the car, the light source is located above the car. From the perspective of the bubbles on the car paint, the light sources of some bubbles are directly above it, the light sources of some bubbles are located to the upper left, the light sources of some bubbles are located to the upper right, and the light sources of other bubbles may be in other directions, that is, the light source direction of each bubble may be different from the light source directions of other bubbles. Therefore, based on the direction of the light source, multiple directions are obtained, and the bubble characteristics of each paint defect area in each direction are obtained. Based on the bubble characteristics of each paint defect area in multiple directions, the bubble expression degree of each paint defect area is obtained. The relationship between the light source and the bubbles is as follows: Figure 5 shown.

[0089] Specifically, by rotating the light source direction to the left, m3 key directions are obtained, and by rotating the light source direction to the right, m4 key directions are obtained. The light source direction is recorded as the key direction. Among them, the number of directions obtained by rotating the light source direction to the left is m3=4, and the number of directions obtained by rotating the light source direction to the right is m4=4. The angles of rotation to the left are 15 degrees, 30 degrees, 45 degrees, and 55 degrees, and the angles of rotation to the right are 16 degrees, 32 degrees, 48 ​​degrees, and 50 degrees. This is used as an example for description, and other values ​​can be set in other implementations. The light source direction is recorded as the key direction.

[0090] Further, according to the zth key direction, the zth key ray is obtained. The zth key ray passing through the bth paint defect area and its parallel ray are recorded as the important ray in the zth key direction of the bth paint defect area. The pixel corresponding to the maximum grayscale value of the two adjacent pixels in the rth important ray in the zth key direction of the bth paint defect area is recorded as the important pixel of the rth important ray in the zth key direction of the bth paint defect area. The absolute value of the difference between the grayscale value of the important pixel and the next adjacent pixel in the rth important ray in the zth key direction of the bth paint defect area is recorded as the grayscale embodiment of the rth important ray in the zth key direction of the bth paint defect area. The average of the grayscale embodiment of all important rays in the zth key direction of the bth paint defect area is recorded as the grayscale embodiment of the bth paint defect area in the zth key direction.

[0091] Furthermore, the important pixel points in each important ray in the zth key direction in the bth paint defect area are connected with the important pixel points in the important rays adjacent to the surrounding important ray, so as to obtain the boundary line of the bth paint defect area in the zth key direction. According to the boundary line of the bth paint defect area in the zth key direction and the edge of the bth paint defect area, several area blocks are obtained. The area block with the largest average grayscale value among all the area blocks of the bth paint defect area is recorded as the light-facing area of ​​the bth paint defect area in the zth key direction. The other area blocks in the bth paint defect area except the zth light-facing area are recorded as the reflective area of ​​the bth paint defect area in the zth key direction.

[0092] Furthermore, the ratio of the number of pixels whose grayscale value in the reflective area of ​​the b-th paint defect area in the z-th key direction is greater than the minimum grayscale value in the light-facing area of ​​the b-th paint defect area in the z-th key direction to the number of pixels contained in the reflective area of ​​the b-th paint defect area in the z-th key direction is recorded as the credibility of the b-th paint defect area in the z-th key direction. The ratio of the length of the boundary line of the b-th paint defect area in the z-th key direction to the number of important rays of the b-th paint defect area in the z-th key direction is recorded as the reliability of the b-th paint defect area in the z-th key direction.

[0093] Furthermore, the specific calculation formula for obtaining the bubble characteristics of the b-th paint defect area in the z-th key direction is as follows:

[0094]

[0095] Where W z，b represents the bubble characteristics of the b-th paint defect area in the z-th key direction, It represents the grayscale expression of the b-th paint defect area in the z-th key direction. represents the reliability of the b-th paint defect area in the z-th key direction, α z，b It represents the credibility of the b-th paint defect area in the z-th key direction. exp() is an exponential function with a natural constant as the base. This embodiment adopts the exp(-x) model to present the inverse proportional relationship. x is the input of the model. The implementer can set the inverse proportional function according to the actual situation.

[0096] Formula logic, The larger the value of , the greater the difference in grayscale values ​​between some pixels in the b-th paint defect area and its surrounding pixels, which further indicates that the b-th paint defect area is more consistent with the characteristic that the grayscale values ​​of some pixels in the bubble area are relatively different; z，bThe smaller the value is, the more the b-th car paint defect area conforms to the characteristic that the grayscale value of the pixels in one area is large and the grayscale value of the pixels in another area is small, that is, the z-th bubble feature of the b-th car paint defect area is more obvious; The larger the value is, the shorter the boundary line of the b-th paint defect area in the z-th key direction is, which is more consistent with the characteristic that the length of the boundary line between the area with larger grayscale value and the area with smaller grayscale value in the bubble area is smaller, that is, the larger the bubble feature of the b-th paint defect area in the z-th key direction is.

[0097] Furthermore, according to the bubble characteristics of the b-th paint defect area in all key directions, the specific calculation formula for obtaining the bubble expression degree of the b-th paint defect area is as follows:

[0098]

[0099] Where W b represents the bubble expression degree of the bth paint defect area, m5 represents the number of key directions, W represents the grayscale representation of the b-th paint defect area in the z-th key direction, z，b It represents the bubble feature of the b-th paint defect area in the z-th key direction. Softmax() represents the weight normalization function, and the normalization object is the grayscale expression of the b-th paint defect area in each key direction. Sigmoid() represents the sigmoid function, which is used for normalization processing.

[0100] In the formula, The larger the value of , the greater the possibility that the zth key straight line passes through the area with larger grayscale value and the area with smaller grayscale value at the same time when it passes through the bth paint defect area. It further indicates that the difference between the zth key direction and the light source direction of the bth paint defect area is smaller. Therefore, a greater weight is given to the bubble feature of the bth paint defect area in the zth key direction.

[0101] Step S005: According to the sagging characteristics, bubble expression and shape of each paint defect area, the possibility of each paint defect area being a sagging area is obtained, and then the sagging area and the bubble area are obtained.

[0102] It should be noted that when judging whether each paint defect area is a sagging area based on its bubble expression degree and sagging characteristics, since the bubble area is circular, a bubble expression degree is assigned to each paint defect area according to its circularity, thereby increasing the difference between the bubble expression degree and the sagging characteristics of each paint defect area and reducing the possibility of misclassifying some bubble areas as sagging areas due to unreasonable threshold setting when obtaining the sagging area through threshold segmentation.

[0103] Specifically, the circularity of the b-th paint defect area is obtained, wherein the method for obtaining the circularity of each area is a known technology and is not described in detail in this embodiment, and the value range of the circularity is (0, 1].

[0104] Furthermore, the specific calculation formula for obtaining the possibility that the b-th paint defect area is a sagging area is as follows:

[0105]

[0106] Where P b W represents the probability that the b-th paint defect area is a sagging area. b Indicates the bubble performance of the b-th paint defect area, R b represents the sagging characteristics of the b-th paint defect area, S b represents the circularity of the b-th paint defect area, sigmoid() represents the sigmoid function, and this embodiment is used for normalization processing.

[0107] It should be noted that R b The larger the value of , the more features of the sagging area are presented in the b-th paint defect area, which further indicates that the possibility that the b-th paint defect area is a sagging area is greater, that is, the possibility that the b-th paint defect area is a sagging area is greater; W b The larger the value of , the more obvious the characteristics of the bubble area presented by the b-th paint defect area are, which further indicates that the possibility that the b-th paint defect area is a bubble area is greater, that is, the possibility that the b-th paint defect area is a sagging area is smaller; S b The smaller the value of is, the smaller the difference between the shape of the b-th paint defect area and the circle is, which further indicates that the possibility that the b-th paint defect area is a bubble area is smaller. Therefore, As R b The weight of is used to increase the difference between the possibility that the sagging area is the sagging area and the possibility that the bubble area is the sagging area, and the possibility of misclassifying part of the bubble area as the sagging area when the threshold setting is unreasonable is reduced.

[0108] A possibility threshold T2 is preset, and a paint defect area whose possibility of being a sagging area is greater than the possibility threshold T2 is recorded as a sagging area. A paint defect area whose possibility of being a sagging area is less than or equal to the possibility threshold T2 is recorded as a bubble area. In this embodiment, the possibility threshold T2=0.2 is preset, and this is used as an example for description. Other values ​​can be set in other implementations.

[0109] At this point, this embodiment is completed.

[0110] Another embodiment of the present invention provides a spray area intelligent segmentation system based on visual radar, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the above method steps S001 to S005 are implemented.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. The intelligent segmentation method of spraying area based on visual radar is characterized by: The method comprises the following steps: Obtain the paint image of the vehicle to be detected, the gravity direction of the vehicle to be detected, and the light source direction; According to the gradient of the pixel points, the suspected paint defect pixel points are obtained, and then the suspected paint defect area and the local area of ​​the suspected paint defect pixel points are obtained; according to the distribution of the gradient values ​​of the suspected paint defect pixel points in the suspected paint defect area and the distribution of the gradient values ​​of the pixel points in the local area of ​​the suspected paint defect pixel points, the paint defect characteristics of the suspected paint defect pixel points are obtained, and then the paint defect area is obtained; According to the gravity direction, all straight lines passing through each paint defect area are obtained to obtain the sum line segment of each paint defect area; according to the difference in grayscale values ​​of adjacent pixels in the sum line segment of each paint defect area and the difference in the gradient direction of the pixels in each paint defect area, the sagging characteristics of each paint defect area are obtained; The direction of the light source is rotated to obtain multiple key directions, and then multiple key rays are obtained; according to the distribution of the grayscale values ​​of the pixels in the same paint defect area on each key ray and the parallel ray, the grayscale expression degree, dividing line, reflective area and light-facing area of ​​each paint defect area in each key direction are obtained; according to the grayscale expression degree of each paint defect area in each key direction, the length of the dividing line and the grayscale distribution of the pixels in the reflective area and the light-facing area, the bubble characteristics of the paint defect area in each key direction are obtained; the bubble expression degree of each paint defect area is obtained by combining the grayscale expression degree of the paint defect area in each key direction; According to the sagging characteristics, bubble expression and shape of each paint defect area, the possibility of each paint defect area being a sagging area is obtained, and then the sagging area and bubble area are obtained; The method of obtaining the suspected paint defect pixel point according to the gradient of the pixel point, and then obtaining the suspected paint defect area and the local area of ​​the suspected paint defect pixel point includes the following specific methods: Obtain the gradient of each pixel in the paint image to be inspected, and record the pixel with a gradient value greater than 0 as a suspected paint defect pixel; All suspected paint defect pixels are used to form a number of suspected paint defect regions; the i-th suspected paint defect pixel and the suspected paint defect pixels in the 8-neighborhood are used as elements in the local region of the i-th suspected paint defect pixel to obtain the local region of the i-th suspected paint defect pixel; The information entropy of the gradient values ​​of all suspected paint defect pixels in the suspected paint defect area where the i-th suspected paint defect pixel is located is recorded as the gradient information entropy of the i-th suspected paint defect pixel; the variance of the gradient values ​​of all pixels in the local area of ​​the i-th suspected paint defect pixel is recorded as the gradient variance of the i-th suspected paint defect pixel; The method of obtaining the paint defect feature of the suspected paint defect pixel point according to the distribution of the gradient value of the suspected paint defect pixel point in the suspected paint defect area and the distribution of the gradient value of the pixel point in the local area of ​​the suspected paint defect pixel point, and then obtaining the paint defect area includes the following specific methods: J i =sigmoid(H i ×σ i ) In the formula, J i represents the paint defect feature of the i-th suspected paint defect pixel, H i represents the gradient information entropy of the i-th suspected paint defect pixel, σ i represents the gradient variance of the i-th suspected paint defect pixel, sigmoid() represents the sigmoid function; A defect threshold T1 is preset, and suspected paint defect pixels whose paint defect characteristics are greater than the defect threshold T1 are recorded as paint defect pixels; All the paint defect pixels are combined to form a number of paint defect areas.

2. The method for intelligent segmentation of spraying area based on visual radar according to claim 1 is characterized in that: The method of obtaining all straight lines passing through each paint defect area according to the gravity direction to obtain the sum of line segments of each paint defect area includes: According to the direction of gravity, all straight lines passing through each paint defect area are obtained; In the fth straight line passing through the bth paint defect area, remove the pixel points that are not in the bth paint defect area, and divide the fth straight line passing through the bth paint defect area into a plurality of line segments; In the fth straight line that passes through the bth car paint defect area, each line segment is connected end to end with its adjacent line segment to obtain a sum line segment, which is recorded as the fth sum line segment of the bth car paint defect area; The fth line segment of the bth paint defect area is recorded as the target line segment.

3. The method for intelligent segmentation of spraying area based on visual radar according to claim 2 is characterized in that: The method of obtaining the sagging feature of each paint defect area according to the difference in grayscale values ​​of adjacent pixels in each paint defect area and line segment and the difference in gradient direction of pixels in each paint defect area includes: In the formula, R b represents the sagging characteristics of the b-th paint defect area, n b represents the sum of the number of straight lines where the target line segment passes through the b-th paint defect area and its parallel straight lines, g f,b Indicates the number of pixels contained in the target line segment, C b,f,d+1 Represents the gray value of the d+1th pixel in the target line segment, C b,f,d Represents the gray value of the dth pixel in the target line segment, σ b represents the variance of the gradient direction of all pixels in the b-th paint defect area, || represents the absolute value function, exp() is the exponential function with a natural constant as the base, and sgn() represents the sign function.

4. The method for intelligent segmentation of spraying area based on visual radar according to claim 1 is characterized in that: The specific method of rotating the light source direction to obtain multiple focus directions and then obtain multiple focus rays includes: Rotate the light source direction to the left to obtain m3 key directions, and rotate the light source direction to the right to obtain m4 key directions, and record the light source direction as the key direction; wherein m3 represents the number of directions obtained by rotating the preset light source direction to the left, and m4 represents the number of directions obtained by rotating the preset light source direction to the right; According to the zth key direction, the zth key ray is obtained.

5. The method for intelligent segmentation of spraying area based on visual radar according to claim 1 is characterized in that: The grayscale representation, boundary line, reflective area and light-facing area of ​​each paint defect area in each key direction are obtained according to the distribution of the grayscale values ​​of the pixels in the same paint defect area on each key ray and the parallel ray, including the specific method of: The zth key ray is obtained through the zth key direction; the zth key ray passing through the bth paint defect area and the parallel ray are recorded as the important ray in the zth key direction of the bth paint defect area; The maximum absolute value of the difference between the grayscale values ​​of two adjacent pixels in the rth important ray in the zth key direction of the bth paint defect area, and the pixel corresponding to the maximum grayscale value of the two corresponding pixels, is recorded as the important pixel of the rth important ray in the zth key direction in the bth paint defect area; The absolute value of the difference between the grayscale values ​​of the important pixel point and the adjacent next pixel point in the rth important ray in the zth key direction in the bth paint defect area is recorded as the grayscale representation degree of the rth important ray in the zth key direction in the bth paint defect area; the average value of the grayscale representation degrees of all important rays in the zth key direction in the bth paint defect area is recorded as the grayscale representation degree of the bth paint defect area in the zth key direction; Connect the important pixel points in each important ray in the zth key direction of the bth paint defect area with the important pixel points in the adjacent important rays around it to obtain the boundary line of the bth paint defect area in the zth key direction; Several area blocks are obtained through the boundary line of the bth paint defect area in the zth focus direction and the edge of the bth paint defect area; the area block with the largest average grayscale value among all the area blocks of the bth paint defect area is recorded as the light-facing area of ​​the bth paint defect area in the zth focus direction, and the other area blocks in the bth paint defect area except the zth light-facing area are recorded as the zth reflective area of ​​the bth paint defect area in the zth focus direction.

6. The method for intelligent segmentation of spraying area based on visual radar according to claim 1 is characterized in that: The bubble characteristics of the paint defect area in each key direction are obtained according to the grayscale expression of each paint defect area in each key direction, the length of the dividing line, and the grayscale distribution of the pixels in the reflective area and the light-facing area; the bubble expression of each paint defect area is obtained in combination with the grayscale expression of the paint defect area in each key direction, including the specific method of: The ratio of the number of pixels whose grayscale value in the reflective area of ​​the b-th paint defect area in the z-th key direction is greater than the minimum grayscale value in the light-facing area of ​​the b-th paint defect area in the z-th key direction to the number of pixels contained in the reflective area of ​​the b-th paint defect area in the z-th key direction is recorded as the credibility of the b-th paint defect area in the z-th key direction; the ratio of the length of the boundary line of the b-th paint defect area in the z-th key direction to the number of important rays of the b-th paint defect area in the z-th key direction is recorded as the reliability of the b-th paint defect area in the z-th key direction; The specific calculation formula for obtaining the bubble characteristics of the b-th paint defect area in the z-th key direction is as follows: Where W z,b represents the bubble characteristics of the b-th paint defect area in the z-th key direction, It represents the grayscale expression of the b-th paint defect area in the z-th key direction. represents the reliability of the b-th paint defect area in the z-th key direction, α z,b represents the credibility of the b-th paint defect area in the z-th key direction, and exp() is an exponential function with a natural constant as the base; According to the bubble characteristics of the b-th paint defect area in all key directions, the specific calculation formula for obtaining the bubble expression degree of the b-th paint defect area is as follows: Where W b represents the bubble expression degree of the bth paint defect area, m5 represents the number of key directions, W represents the grayscale representation of the b-th paint defect area in the z-th key direction, z,b represents the bubble feature of the b-th paint defect area in the z-th key direction, softmax() represents the weight normalization function, and sigmoid() represents the sigmoid function.

7. The method for intelligent segmentation of spraying area based on visual radar according to claim 1 is characterized in that: The method of obtaining the possibility that each paint defect area is a sagging area according to the sagging characteristics, bubble expression and shape of each paint defect area, and then obtaining the sagging area and bubble area, includes the following specific methods: The circularity of each paint defect area is obtained. According to the circularity, sagging characteristics and bubble expression of each paint defect area, the calculation formula for calculating the possibility that each paint defect area is a sagging area is as follows: Where P b W represents the probability that the b-th paint defect area is a sagging area. b Indicates the bubble performance of the b-th paint defect area, R b represents the sagging characteristics of the b-th paint defect area, S b represents the circularity of the b-th paint defect area, sigmoid() represents the sigmoid function; A possibility threshold T2 is preset, and a paint defect area whose possibility of being a sagging area is greater than the possibility threshold T2 is recorded as a sagging area; a paint defect area whose possibility of being a sagging area is less than or equal to the possibility threshold T2 is recorded as a bubble area.

8. A spraying area intelligent segmentation system based on visual radar, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method for intelligent segmentation of the spraying area based on visual radar as described in any one of claims 1 to 7 are implemented.

Images