Industrial part image segmentation method for uneven illumination

Abstract

The invention discloses an industrial part image segmentation method for uneven illumination, and the method comprises the following steps: 1, traversing all pixel points of a workpiece image, and mapping the pixel points into an RGB color space, thereby obtaining a three-dimensional RGB color data set C; step 2, simplifying the three-dimensional RGB color data set C to obtain a data set C '; step 3, drawing a plane with the ratio of R to G / R, and obtaining color distribution on the plane; step 4, setting alt based on three main areas existing in the plane of the ratio of R to G / R; g / Rlt; b and clt; rlt; d, obtaining a mask area; and step 5, covering the picture masked in the step 4 on the workpiece picture in the step 1 to obtain a color segmentation result of the part image of the workpiece picture. The method has the advantages that the data complexity of the image is reduced, the color information is simplified, and key features in the image can be better highlighted.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an industrial parts image segmentation method for uneven illumination. Background Art

[0002] In the process of industrial parts production, as industrial robots (robotic arms) are put into production in large quantities, they are mainly used in the part grasping stage. The robotic arm determines the position of the object through the visual servo system, that is, based on the relative position of the end effector and the visual sensor. Image processing is an indispensable part of machine vision. It has an important impact on the processing of subsequent algorithms and the application of actual production scenarios such as part grasping.

[0003] However, lighting is the main factor that affects image processing. In outdoor environments, the direction of sunlight is different at different times, resulting in constantly changing lighting conditions. However, considering that most industrial scenes are indoors, there is the influence of lighting, which causes industrial parts (usually metal parts) to reflect light.

[0004] In the robot positioning and grasping based on machine vision, there is a step to identify the parts in the image, so it is necessary to use image segmentation technology to extract the target parts, and then calculate the center of the target parts to achieve the robot positioning and grasping task. If the parts in the captured image have reflective phenomena, it will lead to inaccurate segmentation, thus affecting the subsequent calculation results. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in the positioning and grasping of the robot arm of the known machine vision, the reflection of the parts will lead to inaccurate segmentation, thereby affecting the subsequent calculation results. The present invention proposes a method for image segmentation of industrial parts with uneven illumination.

[0006] Technical solutions adopted: A method for segmenting industrial parts images with uneven illumination includes the following steps: Step 1: Use the opencv visual system combined with the python tool to traverse all the pixels of the workpiece image and map them into the RGB color space to obtain a three-dimensional RGB color data set C of all the pixels of the workpiece; Step 2: Simplify the three-dimensional RGB color data set C in step 1 by using a cube-building method to obtain a data set C'; each side of the cube contains 3 pixels; Step 3: Based on the data set C' in step 2, draw a plane of R and G / R ratio in the python tool to obtain the color distribution on the plane; wherein the horizontal axis is the R value and the vertical axis is the G / R ratio; Step 4: There are three main regions in the plane based on the ratio of R to G / R, namely: a white background region, a gray-blue part region, and a black part region; set the conditions of a < G / R < b and c < R < d, and draw a mask within the three main regions according to the set conditions to obtain a mask region; where a and b are constants of the G / R ratio, and c and d are constants of the R value. Step 5: Use the mask generated in Step 4 to block part of the region in the workpiece image in Step 1, so as to obtain the color segmentation result of the part image of the workpiece image.

[0007] For a further optimization of the technical solution of the present invention, the specific method of Step 2 is to select the pixel at the center point of the cube and use it to represent the surrounding 26 pixel points, and calculate the R, G, and B values of the center of the cube to which each pixel point belongs by using the formula c / / 3*3 + 1, where c is the R, G, and B values of the pixel point; determine the coordinates of the center point of the cube and eliminate duplicate results; obtain the data set C'. The design purpose of this dependent claim is that if the original method is used to calculate the set C, the calculation time is relatively long due to the large number of points; while using this new method can greatly shorten the calculation time.

[0008] For a further optimization of the technical solution of the present invention, in Step 2, the ranges of the R, G, and B values are all 0 - 255; when any one of the R, G, and B values is equal to 255, in the python calculation, use the min function to limit the calculation result of the formula c / / 3*3 + 1; the min function formula is as follows: min((c / / 3) * 3 + 1, 255) Compare the calculation result with 255. If it is less than 255, use the calculation result; if it is greater than 255, take the value of 255.

[0009] The design purpose of this dependent claim is that first, since the R, G, and B values cannot exceed 255, it is necessary to limit their calculation results within 255. Second, the calculation result will exceed 255, and this situation will only occur when there is a value equal to 255 in c. At this time, a complete 3x3x3 cube cannot be formed, and only a 3x3 plane containing 9 pixel points can be formed. By limiting the final calculation result to 255, in fact, one point at the center of the plane is used to replace the surrounding 8 points.

[0010] For a further optimization of the technical solution of the present invention, the condition for eliminating duplicate results is: eliminate those with the same calculated coordinates of the cube center point and keep one. Because in the python calculation, the calculation results are placed in a bag-like structure. Each time a result is calculated, it is placed in the bag. If the new result already exists in the original bag, it will not be placed repeatedly. If it is placed repeatedly, the number of points will not decrease, and the significance of simplifying the calculation will be lost.

[0011] In a further optimization of the technical solution of the present invention, a step of verifying the data set C' in step 2 is set before step 3; the specific method is: randomly select a pixel point in the three-dimensional RGB color data set C, defined as point A, and we look for a point A' in the set C', where each coordinate of A' (that is, R, G, B value) is less than or equal to 1 with each coordinate of A; if so, proceed to step 3; if not, add pixel point A to the data set C', and then proceed to step 3. This method is used to verify the calculation result of step 2 to ensure its integrity and accuracy to avoid missing points.

[0012] In a further preferred embodiment of the technical solution of the present invention, in step 3, if R=0, R=1 is taken to draw the plane. When drawing the plane of the ratio of R and G / R, in the process of processing G / R, if R=0, the result will not exist, so we can assign R=1 at this time, which not only has a small impact on the result, but also does not cause calculation problems.

[0013] In a further preferred embodiment of the technical solution of the present invention, in step 4, a color picker is used to cover the areas in the three main areas that meet the set conditions with red; the specific method is: Step 41, first select constants a, b, c and d on the plane of R and G / R ratio to determine the mask area; Step 42, using a color picker to select the RGB values ​​of some missing points on the plane of the R and G / R ratio, modifying the mask area, and obtaining new constants a, b, c and d; Step 43, manually adjust the values ​​of a, b, c and d in small increments to modify the mask area; Step 44: Repeat step 43 multiple times to obtain the optimal mask area.

[0014] The purpose of designing this weight is to obtain the best result by modifying the value in a small range multiple times, that is, to show the accuracy of the segmentation result.

[0015] In a further preferred embodiment of the technical solution of the present invention, the color picker is obtained by searching in the browser. The color picker is used to absorb the missing points, and can accurately modify the conditions and improve the accuracy.

[0016] Compared with the prior art, this method has the following technical effects: 1. The method of the present invention reduces the data complexity of the image, simplifies the color information, and can better highlight the key features in the image.

[0017] 2. The method of the present invention has strong flexibility, adaptability and pertinence. It allows the operator to adjust according to the specific image content and target, and can adapt to different application scenarios. It can better deal with and improve the accuracy of segmentation in view of the influence of the factor of illumination on the local area. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 For the workpiece picture; Figure 2 This is an unsimplified schematic diagram of color distribution; Figure 3 Schematic diagram of the constructed cube; Figure 4 This is a simplified schematic diagram of color distribution; Figure 5 shows the color distribution on the plane drawn in the Python tool ( Figure 5a For the RG plane, Figure 5b For BR plane, Figure 5c is GB plane); Figure 6 The color distribution diagram of R and G / R ratio on the plane drawn in the python tool; Figure 7 The workpiece image obtained by the method of this embodiment represents the gray-blue part covering most of the part area; Figure 8 The workpiece image obtained by the method of this embodiment represents the black portion covering most of the part area; Fig. 9 This is a workpiece image covering the background area obtained by the method of this embodiment. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0020] To make the content of the present invention more clearly understood, the following Figure 1-9 The specific implementation manner is further described.

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Example 1

[0022] In this embodiment, an industrial part of a flange is taken as an example to further illustrate that an image segmentation method of an industrial part with uneven illumination in this embodiment includes the following steps: Step 1: Use the opencv vision system combined with the python tool to traverse all the pixel points of the workpiece image, map them into the RGB color space, and obtain the three-dimensional RGB color dataset C of all the pixel points of the workpiece; Step 2: Simplify the dataset using the method of constructing a cube for the three-dimensional RGB color dataset C in Step 1 to obtain dataset C'; each edge of the cube contains 3 pixel points; Step 3: Based on dataset C' in Step 2, draw a plane of the R and G / R ratios in the python tool to obtain the color distribution on the plane; where the abscissa is the R value and the ordinate is the G / R ratio; Step 4: Based on the three main regions existing in the plane of the R and G / R ratios, namely: the white background region, the gray-blue part region, and the black part region; set the conditions of a < G / R < b and c < R < d, and draw a mask in the three main regions according to the set conditions to obtain the mask region; where a and b are constants of the G / R ratio, and c and d are constants of the R value; Step 5: Use the mask generated in Step 4 to occlude part of the region in the workpiece image in Step 1, so as to obtain the color segmentation result of the part image of the workpiece image.

[0023] As Figure 1 shown, in Step 1, the workpiece image is a clear image taken by a high-definition camera with the part placed horizontally on the table.

[0024] The opencv vision system in Step 1 is a known vision system, mainly used to solve computer vision tasks, which is known to those skilled in the art. The python tool is a known tool, which is known to those skilled in the art.

[0025] The RGB color space mentioned in Step 1 is a known technology, which is known to those skilled in the art. The R / G / B values in the RGB color space, R, G, and B respectively represent the colors of the red, green, and blue channels; the range of each color channel is between [0, 255].

[0026] Step 1 is specifically: use the opencv vision system to read the image, then use cv.split to split the image into three channels of B / G / R, read the RGB values of each pixel point, and represent them in the RGB coordinate system, that is, its three-dimensional color distribution map; at this time, a very large three-dimensional RGB color dataset C of all the pixel points of the workpiece will be obtained, as Figure 2 shown.

[0027] Among them, cv.split is channel separation, which separates the 3-channel (RGB) color image into 3 single-channel pure color images in the order of (B, G, R). This is a known technology and is known to those skilled in the art.

[0028] For a three-dimensional RGB color data set C of all pixels of a very large workpiece, it is inconvenient to process such a huge data set. In the method of this embodiment, step 2 is proposed to perform simplified calculation on the three-dimensional RGB color data set C to obtain data set C'.

[0029] Specifically, the three-dimensional RGB color distribution consists of cubes with a side length of 3 pixels, such as Figure 3 As shown, the RGB in the figure is similar to the cube coordinate axes XYZ.

[0030] Select the pixel at the center of the cube and use it to represent the surrounding 26 pixels. Use the formula c / / 3*3+1 to calculate the R, G, and B values ​​of the center of the cube to which each pixel belongs, where c is the R / G / B value of the pixel. Determine the coordinates of the center of the cube and remove duplicate results. Get the data set C'. Figure 4 As shown. Figure 2 and Figure 4 It can be clearly seen that this simplified calculation has almost no effect on the color distribution.

[0031] Furthermore, the pixel at the center of the cube is selected to represent the surrounding 26 pixels, which reduces the number of points in the three-dimensional RGB color data set C by 27 times. Figure 3 As shown, the red dot is the center point of the 3x3x3 cube, which is used to represent the surrounding points.

[0032] Take the R axis as an example. For example, if the R value of the selected pixel point is in the range of 0-2, the R value of the cube center is 1; in the range of 3-5, the R value of the cube center is 4, and so on. Therefore, the R value of the cube center should be 1, 4, 7, etc. At this time, the difference of the cube center is 3 Assume that the R value of one pixel is 2. Through the above formula, we can calculate that the R value of the center point of the cube to which it belongs is equal to 2 / / 3*3+1=0*3+1=1, then R=1. The R value of the selected pixel point is in the range of 0-2 as mentioned above, so the R value of the center of the cube is 1.

[0033] The B-axis and G-axis are the same as the R-axis.

[0034] By calculating the R, B, and G values ​​mentioned above, we get the coordinates of the center point of the cube. Then we remove duplicate results. For example, if a point is calculated with the coordinates (1, 1, 1) and put into the data set C', and another point is also calculated with the coordinates (1, 1, 1), then the new (1, 1, 1) will not be put into the data set C' and needs to be removed. Finally, the data set C' is formed.

[0035] In the method of this embodiment, in step 2, the range of R, G, and B values ​​is 0-255; if any one of the R, G, and B values ​​is equal to 255, in the Python calculation, the min function is used to limit the calculation result of the formula c / / 3*3+1; the min function formula is as follows: min((c / / 3) * 3 + 1, 255); the calculation result is compared with 255, if it is less than 255, the calculation result is used; if it is greater than 255, the value 255 is taken.

[0036] In the method of this embodiment, a step of verifying the data set C' in step 2 is also set before step 3; the specific method is: randomly select a pixel point in the three-dimensional RGB color data set C, defined as point A, and we find out whether there is a point A' in the set C', and each coordinate of A' (that is, R, G, B value) is less than or equal to 1 with each coordinate of A; if so, go to step 3; if not, add pixel point A to the data set C', and then go to step 3. The purpose of designing the verification step is to improve its accuracy and avoid color loss.

[0037] In other words, the condition is whether there is a point in the data set C', and the distance from point A to this point is less than or equal to 1 (that is, the difference between the R, G, and B values ​​of point A and this point is less than or equal to 1). If such a point can be found in the data set C', then the data set C' at this time will not be changed; if this point cannot be found in the data set C', then the coordinates of the center of the cube to which point A belongs need to be added to the data set C'.

[0038] Furthermore, R, G, and B represent the colors of the red, green, and blue channels respectively. The range of each color channel is between [0,255]. In the data set C', the centers of these cubes have been found, and their RGB values ​​are known. Now we need to find out whether there is a center of the cube to which point A belongs. For example: the RGB value of pixel A is (1, 1, 0), and the RGB value of the center coordinate of the cube to which it belongs is (1, 1, 1). From this, we can see that the R value and G value are the same, and the B value differs by 1, so it can also be proved that A belongs to this cube.

[0039] In step 3 of this embodiment, further, the RG plane can be drawn within the Python tool: the abscissa is the R value, and the ordinate is the G value; the BR plane: the abscissa is the B value, and the ordinate is the R value; the GB plane: the abscissa is the G value, and the ordinate is the B value. Before simplifying the calculation, the number of points in set C is too large. If we want to observe the color distribution map of set C from various angles, it is almost impossible to achieve in real time. However, after simplification, as the number of points decreases, we can also observe the color distribution maps from various angles in real time. At the same time, through the functions of these three plane graphs, we can also find that taking the RG plane as an example, the coordinates of the points on the R axis and the G axis differ by 3, which also proves the correctness of our simplified calculation from the side.

[0040] In the method of this embodiment, the workpiece image is segmented through the color distribution map on the plane of the ratio of R to G / R.

[0041] Within the Python tool, based on the plane of the ratio of R to G / R, it is known that there are three main regions, namely: the white background region, the gray-blue part region, and the black part region.

[0042] At this time, considering a situation, if R = 0, then take R = 1. Because when using G / R, if R is equal to 0, it is equivalent to the denominator being 0 and thus does not exist.

[0043] Then, it is necessary to manually find the conditions for segmenting the three main regions, set the conditions of a < G / R < b and c < R < d, and draw a mask within the three main regions according to the set conditions to obtain the mask region; where a and b are constants of the G / R ratio set, and c and d are constants of the R value set; a, b, c, and d are manually set ranges, specifically by using tools such as a color picker to determine the thresholds, that is, select the values of a, b, c, and d, and draw a mask to cover the regions that meet the set conditions. Among them, the color picker is obtained by searching in the browser and is a known technology known to those skilled in the art.

[0044] Further, the mask is also called a binary Mask, and it can also be called a baffle. We modify the shape of this baffle according to the set conditions (a < G / R < b and c < R < d), and draw the regions to be retained in red; that is, set the aforementioned three main regions as red images respectively.

[0045] Using a color picker, cover the regions that meet the set conditions within the three main regions with red; the specific method is: Step 41: First, select constants a, b, c, and d on the plane of the ratio of R to G / R to determine the mask region; Step 42: Use a color picker to select the RGB values of some missing points on the plane of the ratio of R to G / R, modify the mask region, and obtain new constants a, b, c, and d; Step 43, manually adjust the values ​​of a, b, c and d in small increments to modify the mask area; Step 44: Repeat step 43 multiple times to obtain the optimal mask area.

[0046] In step 4 of this embodiment, the constants a, b, c and d thresholds are manually selected. Compared with the automatic threshold selection, the manual threshold selection method is more operable and relatively more accurate.

[0047] In step 4 of this embodiment, the size of the mask is the same as the original image, and the mask obtained in step 4 is used to partially block the workpiece image in step 1, and finally the workpiece image in step 1 is obtained. Figure 7 It represents the gray-blue part that covers most of the part area. Figure 8 Indicates the black portion of the covered part area. Fig. 9 Represents the background area in the workpiece image.

[0048] After the segmentation is completed, it is easier for the robot to correctly locate and capture the parts, because the segmented images are easier to identify and analyze.

[0049] Specific application examples We placed the part on the table and took a photo to get a 3105×2005 image, as shown in Figure 1 As shown. First, use 'cv.imread' to read the image, and then convert the BGR channel to RGB format. Convert the colors of all pixels in the image to set C and remove duplicate colors. At this time, each element in set C is a unique color, and we can get 26245 elements in set C. Then use matplotlib to draw a 3D color distribution image, using the R, G, and B values ​​of the pixel as coordinates, and its color is also determined by its RGB value, as shown Figure 2 shown.

[0050] For set C, we use a cube-building method to simplify the complexity of the data set, where each small cube has a side length of 3 pixels. We use the center point of the cube to replace the surrounding 26 pixels, such as Figure 3 Here, we can use (c / / 3) * 3 + 1 to calculate the RGB value of the center point of the small cube to which each element in set C belongs. The RGB value ranges from 0 to 255, so in order to ensure the rationality of the final calculation result, we use min((c / / 3) * 3 + 1, 255) to compare the calculation result with 255. If it is less than 255, use the calculation result; if it is greater than 255, take 255.

[0051] To ensure the integrity of the dataset, we can verify the simplified dataset C'. The specific operation is described as follows: randomly select a point A in the original dataset C and check if there is a point A' in the new dataset C' such that the R, G, and B values of A' are all less than or equal to 1 compared to the corresponding R, G, and B values of A. If such a point A' exists, no transformation is applied to the new set C'. If the point does not exist, the coordinates of the center of the cube to which point A belongs are added to the set C'.

[0052] Finally, we obtain the simplified dataset C' with 7,533 pixel points. It can be clearly seen that the complexity of the data is greatly reduced. Then we plot a three-dimensional scatter plot to visualize the distribution of the dataset C' in the RGB color space, using the R, G, and B values of the pixels as coordinates, and its color is also determined by its RGB values, as Figure 4 shown. By comparing the two color distribution plots, it can be found that this simplification method has little impact on the color distribution. Then three two-dimensional scatter plots are plotted to show the color distribution in the RG, BR, and GB planes, as shown in Figure 5.

[0053] At the same time, a color distribution plot on the plane of R versus G / R ratio is plotted, as Figure 6 shown. At this time, considering that if R = 0, then take R = 1, which not only has little impact on the result but also does not cause calculation problems with a denominator of 0. Through this color distribution plot, we can clearly divide the image into three main component regions: the white background region, the gray-blue part region, and the black part region. Set the conditions a < G / R < b and c < R < d, and draw a red mask within the three main regions according to the set conditions, that is, highlight the regions that meet the specific G / R ratio and R value range.

[0054] We first roughly determine the condition ranges of each region by visual inspection, that is, roughly determine the values of a, b, c, and d. For the black part region, it can be roughly considered that c = 0 and d = 100, that is, the condition is 0 < R < 110. Then further refine the conditions based on the obtained rough results, that is, set the conditions as a = 0, b = 1.109, c = 0, d = 109, that is, 0 ≤ G / R ≤ 1.109, 0 ≤ R ≤ 109, as Figure 7 shown. By analogy, the conditions for the gray-blue region are 1.109 ≤ G / R ≤ 20, 60 ≤ R ≤ 140, as Figure 8 shown, and the conditions for the white background region are 140 ≤ R ≤ 210, as Fig. 9 shown. Thus, the final segmentation result can be obtained.

[0055] The above embodiments are only for illustrating the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for segmenting industrial parts images with uneven illumination, characterized in that: It includes the following steps: Step 1: Use the opencv vision system in combination with the python tool to traverse all the pixel points of the workpiece image, map them into the RGB color space, and obtain the three-dimensional RGB color dataset C of all the pixel points of the workpiece; Step 2: Simplify the dataset using the method of constructing a cube for the three-dimensional RGB color dataset C in Step 1 to obtain dataset C'; each edge of the cube contains 3 pixel points; Step 3: Based on dataset C' in Step 2, draw a plane of the R to G / R ratio in the python tool to obtain the color distribution on the plane; where the abscissa is the R value and the ordinate is the G / R ratio; Step 4: Based on the three main regions existing in the plane of the R to G / R ratio, namely: the white background region, the gray-blue part region, and the black part region; set the conditions of a < G / R < b and c < R < d, and draw a mask in the three main regions according to the set conditions to obtain the mask region; where a and b are constants of the G / R ratio, and c and d are constants of the R value; Step 5: Use the mask generated in Step 4 to block part of the region in the workpiece image in Step 1, so as to obtain the color segmentation result of the part image of the workpiece image.

2. The method according to claim 1, characterized in that: The specific method of Step 2 is to select the pixel at the center point of the cube, use it to represent the surrounding 26 pixel points, and calculate the R, G, and B values of the cube center to which each pixel point belongs using the formula c / / 3*3 + 1, where c is the R / G / B value of the pixel point; determine the coordinates of the center point of the cube and eliminate duplicate results; Obtain dataset C'.

3. The method according to claim 2, characterized in that: In Step 2, the ranges of the R, G, and B values are all 0 - 255; when any one of the R, G, and B values is equal to 255, in the python calculation, use the min function to limit the calculation result of the formula c / / 3*3 + 1; the min function formula is as follows: min((c / / 3) * 3 + 1, 255) Compare the calculation result with 255. If it is less than 255, use the calculation result; if it is greater than 255, then take the value 255.

4. The method according to claim 2, characterized in that: The condition for eliminating duplicate results is: eliminate those with the same calculated coordinates of the cube center point and keep one.

5. The method according to claim 1, characterized in that: Before Step 3, a step of verifying dataset C' in Step 2 is also set; the specific method is: randomly select a pixel point in the three-dimensional RGB color dataset C and define it as point A. We look for whether there is a point A' in the set C'. Each coordinate of A' is less than or equal to 1 compared to each coordinate of A; if so, enter Step 3; If not, add the pixel point A to dataset C' and then enter Step 3.

6. The method according to claim 1, characterized in that: In Step 3, if R = 0, then take R = 1 for plane drawing.

7. The method according to claim 1, characterized in that: In Step 4, use a color picker to cover the regions that meet the set conditions in the three main regions with red; the specific method is: Step 41: First select the constants a, b, c, and d on the plane of the R to G / R ratio to determine the mask region; Step 42: Use the color picker to select the RGB values of some missing points on the plane of the R to G / R ratio, modify the mask region, and obtain new constants a, b, c, and d; Step 43, manually adjust the values ​​of a, b, c and d in small increments to modify the mask area; Step 44: Repeat step 43 multiple times to obtain the optimal mask area.

8. The method according to claim 7, characterized in that: The color picker is obtained by searching in the browser.

Images