Industrial robot vision recognition and positioning method and system

Abstract

The application discloses an industrial robot vision identification and positioning method and system, and belongs to the technical field of intelligent robots, and the technical solution points are as follows: a first image of a workpiece to be processed and a preset texture chain code are acquired; bad points in the first image are determined according to the first chain code corresponding to the first image and the preset texture chain code, and instructions are sent to the industrial robot according to the bad points, so that the industrial robot eliminates the bad points; a second image of the workpiece to be processed and a second chain code corresponding to the second image are acquired, the second image is an image of the workpiece to be processed after the bad points are eliminated; and control parameters are updated according to the second chain code and the preset texture chain code. The application describes the texture features of the workpiece to be processed through the chain code, identifies errors in the processing process based on chain code comparison, and then adjusts the control parameters of the industrial robot according to the errors, and is suitable for scenes in which the controller has limited computing power and cannot perform complex image processing algorithms in the industrial scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent robots, in particular to an industrial robot vision recognition and positioning method and system. BACKGROUND

[0002] In an industrial environment, PLC (Programmable Logic Controller) and MCU (Micro Control Unit) are commonly used control devices, which have stable logic operation and instruction execution capabilities, but are limited by hardware architecture and cost considerations, and their computing resources are relatively limited, resulting in poor vision recognition and positioning capabilities of current industrial robots, and thus there are many deficiencies. Moreover, the vision recognition and positioning capabilities of current industrial robots mostly rely on images, although image technology is relatively mature, but due to the limitation of the computing power of industrial robots, it is difficult to quickly and accurately position visually, and for visual positioning, it is currently only possible to rely on images, resulting in a significant reduction in the work efficiency of current industrial robots, which requires longer image recognition time, and thus the existing technology has deficiencies. SUMMARY

[0003] In view of the deficiencies of the prior art, the purpose of the present application is to provide an industrial robot vision recognition and positioning method and system, which converts texture features into chain codes, adjusts control parameters by comparing chain codes, and reduces computational complexity.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0005] The present application provides an industrial robot vision recognition and positioning method, comprising:

[0006] obtaining a first image of a workpiece to be processed and a preset texture chain code;

[0007] determining bad points in the first image according to a first chain code corresponding to the first image and the preset texture chain code, and sending instructions to an industrial robot according to the bad points to eliminate the bad points by the industrial robot;

[0008] obtaining a second image of the workpiece to be processed and a second chain code corresponding thereto, the second image being an image of the workpiece to be processed after the bad points are eliminated;

[0009] updating control parameters according to the second chain code and the preset texture chain code.

[0010] As a further improvement of the present application, obtaining the preset texture chain code comprises:

[0011] obtaining an original image corresponding to the workpiece to be processed;

[0012] obtaining a denoising image according to a preset structure element and the original image;

[0013] performing an iterative operation according to the denoised image to obtain a skeleton line corresponding to a texture feature in the denoised image;

[0014] obtaining the preset texture chain code according to pixel points included in the skeleton line.

[0015] As a further improvement of the application, the denoised image is obtained according to the preset structure element and the original image, including:

[0016] For each pixel point in the original image, a preset neighborhood corresponding to the pixel point is determined according to the structure element;

[0017] The pixel value of the pixel point corresponding to the preset neighborhood is updated according to the minimum pixel value and the maximum pixel value in the preset neighborhood to obtain the denoised image.

[0018] As a further improvement of the application, the denoised image is obtained according to the denoised image to obtain a skeleton line corresponding to a texture feature in the denoised image, including:

[0019] The denoised image is converted into a binary image;

[0020] An iterative operation is performed according to the binary image, and the iterative operation includes traversing all pixel points in the current binary image, deleting edge pixel points in the all pixel points, until a preset termination condition is reached, and outputting the current binary image;

[0021] The skeleton line is obtained according to non-zero pixel points in the current binary image.

[0022] As a further improvement of the application, the denoised image is obtained according to the denoised image to obtain a skeleton line corresponding to a texture feature in the denoised image, including:

[0023] A geometric center is determined according to the coordinates corresponding to the pixel points;

[0024] The relative direction of each pixel point is obtained by traversing the pixel points from the geometric center as the starting point, and the relative direction is the direction relative to the previous pixel point;

[0025] The preset texture chain code is obtained according to the relative direction and the length corresponding to the relative direction.

[0026] As a further improvement of the application, the denoised image is obtained according to the denoised image to obtain a skeleton line corresponding to a texture feature in the denoised image, including:

[0027] Boundary coordinates are determined according to the first chain code and the preset texture chain code;

[0028] The bad points are determined according to the boundary coordinates and a preset error.

[0029] As a further improvement of the present application, the determination of the boundary coordinates according to the first chain code and the preset texture chain code comprises:

[0030] Comparing the first chain code with the preset texture chain code;

[0031] Determining a mutation unit according to the comparison result;

[0032] Determining the boundary coordinates according to the coordinates corresponding to the mutation unit.

[0033] As a further improvement of the present application, the control parameters include first control parameters and second control parameters, and the updating of the control parameters according to the second chain code and the preset texture chain code comprises:

[0034] Determining a horizontal offset error and a rotation offset error according to the second chain code and the preset texture chain code;

[0035] Updating the first control parameters according to the horizontal offset error and the rotation offset error.

[0036] As a further improvement of the present application, after updating the first control parameters, the industrial robot visual recognition and positioning method further comprises:

[0037] Sending instructions to the industrial robot according to the updated first control parameters, so that the industrial robot performs corresponding operations;

[0038] Obtaining a third image and a third chain code corresponding thereto;

[0039] Obtaining a thermal expansion coefficient according to the second chain code and the third chain code;

[0040] Updating the second control parameters according to the thermal expansion coefficient.

[0041] The present application provides an industrial robot visual recognition and positioning system, which is applied to the above-mentioned industrial robot visual recognition and positioning method, and comprises a controller and an industrial robot.

[0042] The controller comprises:

[0043] A first receiving module is configured to obtain an original image, a first image and a second image of a workpiece to be processed, wherein the second image is an image of the workpiece to be processed after bad points are eliminated;

[0044] A calculation module is configured to obtain a preset texture chain code according to the original image, determine bad points in the first image according to a first chain code corresponding to the first image and the preset texture chain code, and update control parameters according to the second chain code and the preset texture chain code.

[0045] a control module configured to send an instruction to the industrial robot according to the bad points, so that the industrial robot eliminates the bad points;

[0046] The industrial robot comprises:

[0047] a camera configured to capture the original image, the first image and the second image;

[0048] a second receiving module configured to receive the instruction sent by the controller;

[0049] a mechanical arm configured to perform a corresponding operation according to the instruction.

[0050] Firstly, the first image of the workpiece to be processed is acquired, and the texture feature corresponding to the first image is extracted to obtain a first chain code. Then, the first chain code is compared with a preset texture chain code to eliminate the bad points in the first chain code, so as to obtain a second image. A second chain code is generated based on the texture feature of the second image. The horizontal offset error and the rotation offset error are identified through the difference between the second chain code and the preset texture chain code. Finally, the control parameters are adjusted according to the horizontal offset error and the rotation offset error. The length and direction of the texture feature are described by the chain code, and the image recognition is converted into the comparison between the chain codes. Compared with the traditional complex image recognition algorithm, the processing precision can be ensured while the calculation power is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The flowchart of the method steps of the present application is shown in the figure;

[0052] Figure 2 The schematic diagram of the skeleton line in the binary image is shown in the figure;

[0053] Figure 3 The schematic diagram of the skeleton line in the second image is shown in the figure;

[0054] Figure 4 The schematic diagram of the denoised image and the second image is shown in the figure. DETAILED DESCRIPTION

[0055] The technical scheme of the present application will be described in detail below by means of the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, but not limitations of the technical scheme of the present application.

[0056] Wherein, the same parts are denoted by the same reference numerals. In the following, the term "and / or" is only used to describe the association relationship of the associated objects, which means that there are three kinds of relationships, for example, A and / or B, which means that there are three kinds of relationships, such as A alone, A and B together, and B alone. In addition, the character " / " generally represents an "or" relationship between the front and rear associated objects.

[0057] As Figure 1 shown in the embodiments of the present application, a method for visual identification and positioning of an industrial robot is provided, comprising:

[0058] obtaining a first image of a workpiece to be processed and a preset texture chain code;

[0059] determining a bad point in the first image according to the first chain code corresponding to the first image and the preset texture chain code, and sending an instruction to the industrial robot according to the bad point, so that the industrial robot eliminates the bad point;

[0060] obtaining a second image of the workpiece to be processed and a second chain code corresponding thereto, the second image being an image of the workpiece to be processed after the bad point is eliminated;

[0061] updating a control parameter according to the second chain code and the preset texture chain code.

[0062] In the embodiments, the processing steps for the workpiece include milling, bad point removal and drilling. The method provided by the embodiments is for the two steps of bad point removal and drilling, i.e., the workpiece to be processed in the embodiments refers to a workpiece that has been subjected to milling but has not been subjected to bad point removal and drilling. The bad point refers to a local abnormal area on the surface of the workpiece due to processing defects, material flaws or environmental interference, such as linear scratches caused by tool wear during milling or oxidation points in the raw material. Bad point removal needs to be performed before the drilling step to avoid cracks during drilling. The control parameter includes the drilling position and the drilling speed of the industrial robot, etc.

[0063] Specifically, in the processing steps, the milling step is performed in one workbench, and the bad point removal and drilling steps are performed in another workbench. Therefore, after milling, the position of the workpiece to be processed needs to be moved first, and then the image of the workpiece to be processed is collected by the camera configured by the industrial robot, which is recorded as the first image. The chain code obtained according to the texture features in the first image is recorded as the first chain code. By comparing the first chain code with the preset texture chain code, the bad point in the first image can be identified, and an instruction is sent to the industrial robot according to the position corresponding to the bad point, so that the industrial robot eliminates the bad point. After eliminating the bad point, the image of the workpiece to be processed is collected by the camera, which is recorded as the second image, and the second chain code is obtained according to the texture features in the second image. The angle, distance and field of view range of the camera when collecting the image each time should be consistent to avoid the influence of environmental factors on the difference between the chain codes.

[0064] Moreover, since errors are likely to occur when moving and placing the workpiece to be processed to another workbench, the position of the placed workpiece to be processed may be shifted. If drilling is still performed according to the preset texture chain code, the processing accuracy will be affected. Therefore, the control parameter needs to be adjusted according to the comparison result of the preset texture chain code and the second chain code to ensure the processing accuracy.

[0065] The first image of the workpiece to be processed is acquired, and the texture features corresponding to the first image are extracted to obtain a first chain code. Then, the first chain code is compared with a preset texture chain code to remove bad points in the first chain code, obtain a second image, and generate a second chain code based on the texture features of the second image. The horizontal offset error and the rotation offset error are identified by the difference between the second chain code and the preset texture chain code. Finally, the control parameters are adjusted according to the horizontal offset error and the rotation offset error. The length and direction of the texture features are described by the chain code, and the image recognition is converted into the comparison between the chain codes. Compared with the traditional complex image recognition algorithm, the processing precision can be ensured while reducing the computing power.

[0066] At present, the visual recognition and positioning ability of industrial robots mostly rely on images. The application finds that although image technology is relatively mature, it is limited by the computing power of industrial robots and cannot quickly and accurately perform visual positioning. For visual positioning, it is currently only possible to rely on images, which greatly reduces the working efficiency of industrial robots and requires longer image recognition time. Therefore, the application first decouples image acquisition and image processing and converts image processing into low-dimensional data processing. However, the recognition accuracy is greatly affected by the current dimension reduction processing method. The visual recognition and positioning scheme for industrial robots provided by the application can ensure the recognition accuracy while speeding up the recognition speed.

[0067] Further, the embodiment provides a step of obtaining a preset texture chain code, comprising:

[0068] obtaining an original image corresponding to the workpiece to be processed;

[0069] obtaining a denoising image according to the preset structure element and the original image;

[0070] performing an iteration operation according to the denoising image to obtain a skeleton line corresponding to the texture features in the denoising image;

[0071] obtaining the preset texture chain code according to the pixel points included in the skeleton line.

[0072] The original image is the image of the workpiece after milling without bad points. Since there are various types of workpieces involved in industrial production, the original image is also multiple. When obtaining the preset texture chain code, the original image corresponding to the workpiece to be processed should be determined according to the type of the workpiece to be processed.

[0073] Specifically, after milling, even if the original image does not have a bad point, other defects may still occur, such as the presence of iron filings noise or milling texture fracture. The iron filings noise refers to the image noise formed by the strip or particle iron filings generated during the milling process, which is adsorbed on the milling texture formed by the workpiece surface existing magnetism caused by local magnetization due to high-speed cutting. The milling texture fracture refers to the texture fracture caused by tool wear and the like during the milling process.

[0074] In order to obtain more accurate preset texture chain code, it is necessary to first eliminate the noise problem in the original image and repair the fracture. Specifically, a structure element can be preset, and each pixel point in the original image is traversed through the structure element, and each pixel point is repaired according to the pixel points in the preset neighborhood of each pixel point, so as to remove the defects in the original image and obtain a denoising image. The preset neighborhood is determined according to the shape and size of the structure element, and the traversal direction is from left to right and from top to bottom. The denoising image includes a texture region and a background region. The texture region is a region where the milling texture exists. The milling texture usually includes a plurality of parallel lines left after milling. The background region is a plane that has not been milled. The skeleton line is usually a center axis corresponding to the milling texture, and has a width of 1 pixel and is used to locate the position of the texture. Since the milling texture includes a plurality of parallel lines, the skeleton line also includes a plurality of lines, and each line corresponds to a line. According to the coordinates of the pixel points in the skeleton line, the preset texture chain code corresponding to each line can be obtained. The preset texture chain code is used to describe the direction and length of the line. However, since the preset texture chain code is only related to the direction and length, the preset texture chain code corresponding to each line should be the same. Therefore, in actual application, only the preset texture chain code corresponding to one line needs to be obtained.

[0075] In this embodiment, the original image is denoised to obtain a denoising image, and the skeleton line and the preset texture chain code are extracted based on the denoising image. The texture features in the original image can be converted into digital chain codes, while the topology information of the texture is retained and the complexity of subsequent calculation is reduced.

[0076] Further, the embodiment provides a step of obtaining a denoising image according to a preset structure element and an original image, comprising:

[0077] For each pixel point in the original image, a corresponding preset neighborhood is determined according to the structure element;

[0078] According to the minimum pixel value and the maximum pixel value in the preset neighborhood, the pixel value of the pixel point corresponding to the preset neighborhood is updated to obtain a denoising image.

[0079] Specifically, first, the original image is converted into a gray image, then a structure element is slid on the gray image, and each pixel point in the gray image is traversed in turn, wherein the structure element can be a matrix or a vector, in the traversal process, the center element in the structure element should coincide with the pixel point, therefore the number of rows and columns of the structure element should be odd, and the number of rows and columns of the structure element is the same when the structure element is a matrix. When the center element in the structure element coincides with the pixel point, the structure element also covers a plurality of pixel points in addition to the pixel point, the region formed by the plurality of pixel points is called a preset neighborhood corresponding to the pixel point, then the minimum pixel value in the preset neighborhood is obtained, and the pixel value of the pixel point is modified to the minimum pixel value, until all the pixel points in the gray image are traversed, a first denoising image is obtained, and the first denoising image is an image in which the iron filings noise has been removed. Then, for the first denoising image, each pixel point in the first denoising image is also traversed using the above structure element, the maximum pixel value in the preset neighborhood corresponding to the pixel point is obtained, and the pixel value of the pixel point is modified to the maximum pixel value, until all the pixel points in the first denoising image are traversed, a denoising image is obtained, and the denoising image is an image in which the iron filings noise and the milling texture fracture have been removed.

[0080] The specific formula is that, assuming that the structure element is a matrix , , is odd, all the elements in the matrix are 1, first, each pixel point in the gray image is traversed, and the pixel value of the pixel point is modified to:

[0081]

[0082] wherein represents the coordinates of the pixel point, represents the modified pixel value, and are indexes for traversing each element in the preset neighborhood, represents the pixel value of the pixel point in the preset neighborhood in the gray image with the coordinates . Then, each element in the first denoising image is traversed, and the pixel value of the pixel point is modified to:

[0083]

[0084] wherein represents the pixel value of the pixel point in the denoising image with the coordinates , represents the pixel value of the pixel point in the preset neighborhood in the first denoising image with the coordinates .

[0085] The embodiment is based on the principle of morphology, and removes the iron filings noise and milling texture fracture in the image. Specifically, the application finds that the residual iron filings are smooth and have high reflectivity compared to the surrounding rough texture, so the gray scale of the iron filings noise is higher than that of the surrounding texture, and the iron filings noise should be an isolated point with a high gray scale in the gray scale image. The embodiment traverses the gray scale image by using a structure element. When the center element in the structure element coincides with the pixel point corresponding to the iron filings noise, the minimum pixel value in the preset neighborhood is obtained. Since the iron filings noise is located on the milling texture, the pixel points in the preset neighborhood are all normal texture corresponding pixel points. The pixel value of the pixel point corresponding to the iron filings noise is modified to the pixel value of the normal texture corresponding pixel point, so as to achieve the purpose of eliminating noise.

[0086] In addition, since the embodiment traverses all pixel points in the gray scale image when removing the iron filings noise, for the pixel points with noise, the noise can be effectively removed by the above steps. For the pixel points located in the background area, since the background area is not processed, when light is irradiated, diffuse reflection usually occurs. The grooves and ridges in the texture area are regularly arranged, and when light is irradiated, diffuse reflection occurs in the grooves and mirror reflection occurs in the ridges. Therefore, the reflection of the texture area is diffuse reflection and partial mirror reflection, so the gray scale value of the background area is smaller than that of the texture area. Based on this, for the pixel points in the background area, even if there are pixel points located in the texture area in the range covered by the structure element, the minimum pixel value in the corresponding preset neighborhood is still a pixel point located in the background area. Therefore, after the above steps are performed on the pixel points in the background area, the corresponding pixel value can be approximately unchanged.

[0087] For the pixel points without noise in the texture area, there will be pixel points in the background area in the corresponding preset neighborhood range. Therefore, after the above steps are performed on the pixel points without noise in the texture area, the pixel value will be reduced. In addition, there are normal textures and broken textures in the texture area. The broken textures have lower gray scale values than the normal textures because part of the background is exposed. After the above steps are performed, the broken part will be further reduced and the broken part will be further increased. Therefore, the embodiment performs a second traversal on the basis of the first denoising image, modifies the pixel value of each pixel point to the maximum pixel value in the corresponding preset neighborhood, repairs the broken part, compensates for the reduced pixel value of the normal texture, and alleviates the influence caused by the first traversal of the pixel points.

[0088] In the embodiment, the size and value of the structure element are not limited, and a person skilled in the art can select them according to the size of the noise and the broken texture. For example, if the iron filings noise is 1-2 pixel level and the width of the texture is greater than 3 pixels, the structure element can be a matrix of 3*3 to completely cover the noise and reduce the influence of the first traversal of the pixel points on the texture area. ​

[0089] In the embodiment, when the preset texture chain code is acquired, first, the pixel value of the pixel point with noise is reduced based on the structural element to obtain a first denoising image, then the pixel value in the texture region is adjusted again based on the structural element to compensate for the reduced pixel value of the normal texture while repairing the broken texture, and thus a denoising image with the iron filings noise and the broken milling texture is obtained, and then a more accurate preset texture chain code is extracted according to the denoising image, the accuracy of subsequent chain code comparison is improved, and thus the accuracy of the control parameter is improved.

[0090] Further, the embodiment provides a step of obtaining a skeleton line corresponding to a texture feature in the denoising image by performing an iterative operation on the denoising image, including:

[0091] Converting the denoising image into a binary image;

[0092] Performing an iterative operation on the binary image, the iterative operation including traversing all pixel points in the current binary image, deleting edge pixel points in all pixel points until a preset termination condition is reached, and outputting the current binary image;

[0093] Obtaining the skeleton line according to the non-zero pixel points in the current binary image.

[0094] Specifically, first, the denoising image is converted into a binary image. For example, a threshold value can be set, and the pixel points with a gray value greater than the threshold value are set to white (the gray value is 255), representing the texture region, and the remaining pixel points are set to black (the gray value is 0), representing the background region. Then, another structural element is set, each pixel point in the binary image is traversed, and the edge pixel points in the binary image are deleted. The edge pixel points are the pixel points adjacent to the background region in the texture region, so that the width of each texture in the finally obtained binary image is 1 pixel value, and the skeleton line is obtained according to the coordinates of the pixels in the texture.

[0095] For example, it is assumed that the binary image is as shown in the first image in Figure 2 , where there is only one texture, and the texture includes 12 pixel points. The skeleton line corresponding to the texture needs to be extracted. First, a cross-shaped structural element is set:

[0096]

[0097] Then, the texture is traversed using the structural element . Compared with the structural element with all 1s described above, only the four pixel points above, below, left and right are considered each time, which reduces the range of the neighborhood, so that the extracted skeleton line can delete the edge pixel points along the axis direction of the texture, avoiding the deviation or breakage of the skeleton line. Specifically, the structural element Each to arrive a pixel point, first need to identify the value of the pixel point is 1, namely to determine whether the pixel point belongs to the texture region, if yes, check its corresponding four neighborhood range, if the number of pixel points in its neighborhood including the pixel value is 1, and the number of jump point is 2, it is explained that the pixel point is an edge pixel point, the pixel point is placed in the delete list. Among them, the jump point is the number of times of pixel value from 0 to 1, specifically, the pixel value in the four neighborhood of the pixel point is arranged in the order of up, right, down, left, up, and the number of times of pixel value from 0 to 1 is observed, that is, the number of jump points, for example, assuming that the pixel value of the pixel point above and right is 1, and the pixel value of the pixel point below and left is 0, then it is arranged in order to get [1,1,0,0,1], and the number of times of pixel value from 0 to 1 is 1, that is, the number of jump points is 1.

[0098] After traversing all the pixel points, modify the pixel value of all the pixel points in the delete list to 0 (i.e. delete the edge pixel points from the texture region), get an updated binary image, as shown in the second image of Figure 2 After that, according to the above steps, traverse each pixel point in the updated binary image again until there is no edge pixel point, output the current binary image, at this time, there is only one pixel wide texture in the binary image, which is the skeleton line corresponding to the original texture, that is, the skeleton line corresponding to the texture feature in the binary image, as shown in the third image of Figure 2 , wherein Figure 2 The pixel value of the gray grid in

[0099] But the above example is only applicable to the case where the region where the texture is located has a center axis, that is, only applicable to the case where the number of rows or columns of the matrix corresponding to the region where the texture is located is odd, and the number of pixel points included in the center axis is even, wherein, at this time, the center axis is the longer symmetry axis of the matrix, for example Figure 2 The number of columns of the region where the texture is located is 3, and the number of pixel points included in the center axis is 4.

[0100] Therefore, the embodiment further needs to be improved on the basis of the above steps. Specifically, after the edge pixel point does not exist, the coordinates corresponding to all elements in the deletion list need to be checked. If the number of rows and the number of columns involved by all the coordinates are not equal, the current binary image is output. At this time, there is only one line with a width of 1 pixel in the binary image. If they are equal, it is judged whether the horizontal coordinates or the vertical coordinates corresponding to all elements in the deletion list are continuous. The standard for judging whether they are continuous is that the horizontal coordinates corresponding to each element are obtained, and it is judged whether the values corresponding to the horizontal coordinates can form a continuous sequence. If yes, it means that the horizontal coordinates are continuous. Then the vertical coordinates corresponding to each element are obtained, and it is judged whether the values corresponding to the vertical coordinates can form a continuous sequence. If yes, it means that the vertical coordinates are continuous.

[0101] If both the horizontal coordinates and the vertical coordinates are continuous, the pixel points added to the deletion list when the last time the pixel points are traversed are found from the deletion list, and two pixel points adjacent in the horizontal coordinates are selected at random from the deletion list, and all the pixel points with the same horizontal coordinates as the two pixel points are selected from the deletion list. The pixel points selected twice are deleted from the deletion list, that is, the pixel values of the pixel points selected twice do not need to be modified to 0. Then the pixel values of all the pixel points in the deletion list are modified to 0, and the current binary image is output. At this time, there is only one line with a width of 1 pixel in the binary image. The above horizontal coordinates can also be vertical coordinates.

[0102] If only one of the horizontal coordinates or the vertical coordinates is continuous, for example, if the horizontal coordinates are continuous, the median of the horizontal coordinates is determined. If the median is not an integer, a value closest to the median is selected at random, and the pixel values of the horizontal coordinates in the deletion list that are the median or the value are deleted. Then each pixel point is traversed again, and the pixel points with horizontal coordinates that are not the median or the value are added to the deletion list. Then the pixel values of all the pixel points in the deletion list are modified to 0, and the current binary image is output. Similarly, the steps when the vertical coordinates are continuous can be obtained, which will not be described herein.

[0103] Since the preset texture chain code corresponding to each line is the same, the embodiment only explains and describes that there is only one line in the binary image, but the embodiment is not limited thereto. That is, there can be multiple lines in the binary image.

[0104] This embodiment extracts the skeleton lines corresponding to texture features based on binary images, retaining only the core structure of the texture features, reducing the amount of data, and solving the problem that in the process of obtaining a denoised image based on the first denoised image, it is easy to include pixels in the background area into the texture area, resulting in a large number of redundant pixels in the texture area, which affects the accuracy of texture localization. At the same time, the skeleton lines, based on the characteristic of being 1 pixel wide, make the chain code direction encoding unique, which can accurately describe the geometric information of the texture, ensure the accuracy of the preset texture chain code, and thus improve the accuracy of subsequent chain code comparison and ensure processing precision.

[0105] Furthermore, embodiments of this application provide a step for obtaining a preset texture chain code based on the pixels included in the skeleton line, comprising:

[0106] Determine the geometric center based on the coordinates of the pixel.

[0107] Starting from the geometric center, traverse the pixels to obtain the relative direction of each pixel, which is the direction relative to the previous pixel.

[0108] Based on the relative direction and its corresponding length, the preset texture chain code is obtained.

[0109] Specifically, first, the coordinates of each pixel included in the skeleton line are obtained, and then the mean of the horizontal coordinates is calculated. and the mean of the ordinate , coordinate If the corresponding pixel is used as the geometric center, or If it is not an integer, then the coordinates in the skeleton line will be compared with... The nearest pixel is used as the geometric center. Then, starting from the geometric center, the pixels are traversed to obtain the relative direction of each pixel and the corresponding length for each relative direction. For example, suppose the coordinates of the pixels included in the skeleton line are... , , , , The geometric center is Next, starting from the geometric center, the pixels in the skeleton line are traversed sequentially from the four directions corresponding to the geometric center: up, down, left, and right. The four directions are numbered as follows: up = 1, down = 2, left = 3, and right = 4. In this example, the skeleton line only involves the left and right directions. First, we search to the left and find that there are 2 pixels to the left of the geometric center. After reaching the endpoint, we search to the right from the geometric center and find that there are 2 pixels to the right of the geometric center. Therefore, the preset texture chain code corresponding to the skeleton line is [(3,2),(4,2)], where 3 in (3,2) represents left and 2 represents that there are 2 pixels to the left of the geometric center.

[0110] The embodiment first takes the geometric center of the skeleton line as the starting point, eliminates the error caused by subjective selection of the starting point, and ensures the symmetry of the chain code to facilitate fast judgment of the shape of the texture when the chain code is compared in the subsequent process. Then, based on the geometric center, multiple geometric coordinates are converted into one chain code, which can compress the data size and intuitively describe the texture features. Then, based on the preset texture chain code, the chain code is compared, a small amount of data and calculation can be used to determine the bad points and adjust the control parameters, and a small amount of computing power can be used to ensure the machining precision.

[0111] Further, the embodiment of the present application provides a step of determining a bad point in a first image according to a first chain code corresponding to the first image and a preset texture chain code, comprising:

[0112] determining a boundary coordinate according to the first chain code and the preset texture chain code;

[0113] determining a bad point according to the boundary coordinate and a preset error.

[0114] Specifically, first, the first chain code is obtained according to the first image, and the way of obtaining the first chain code is the same as the way of obtaining the preset texture chain code described above, which is not repeated here. Since there are bad points in the first image, the plurality of first chain codes obtained based on each texture in the first image are not exactly the same as the preset texture chain code.

[0115] Then each first chain code is compared with the preset texture chain code, and the first chain code different from the preset texture chain code is obtained, which is different in length and same in direction, same in length and different in direction, and different in length and direction. For the case of different length and same direction, it can be further divided into two cases of length increase and length decrease. The reason for length decrease is that there is an oxidation point, which causes the texture to break. The reason for length increase is that there is a linear scratch along the texture direction. For the case of same length and different direction, the direction in the chain code is increased, which indicates that there is a linear scratch intersecting with the texture. For the case of different length and direction, it indicates that there are oxidation points and linear scratches. In this embodiment, only the case of at most one linear scratch in each texture is considered. Since the direction of the linear scratch intersecting with the texture is not fixed, and the direction of the texture is prone to error after moving the workpiece, the pixel point in the skeleton line may not be in the set four directions of up, down, left and right. At this time, the closest direction to the set four directions should be selected, and the direction error caused by moving the workpiece is assumed to be small in this embodiment, which does not affect the judgment of the relative direction of the pixel point. In order to avoid that the geometric center corresponding to the coordinate mean value is located outside the skeleton line, when extracting the first chain code, if the coordinates are located on the same horizontal line, the pixel point corresponding to the coordinate mean value is taken as the geometric center, if the coordinates are not located on the same horizontal line, the intersection point of the multiple line segments is taken as the geometric center, and if two vectors are obtained in the same direction, since the length of the linear scratch is small, the longer one is retained and the direction of the other one is modified. For example, assume that a skeleton line is as shown in the gray part of Figure 3 , where each grid corresponds to a pixel point. Since the coordinates in the skeleton line are not located on the same horizontal line, the intersection point B of the line segments AB and CD is taken as the geometric center. First, the left side of the geometric center is searched to obtain two pixel points, which are recorded as vectors . After reaching the end point, the right side of the geometric center is searched to obtain vectors corresponding to BA and corresponding to BD. Since BD is longer, the direction corresponding to needs to be modified. As can be seen from the figure, the directions closest to BA are 1 and 4, i.e. up and right. Since the direction before modification is right, the right needs to be modified to up, i.e. the direction of is modified to . Finally, the first chain code corresponding to Figure 3 is obtained as .

[0116] Then, whether there is a bad point is determined according to the preset error. Specifically, each preset texture chain code is compared with the first chain code, and the difference of the total length in each preset direction is calculated. If the difference in a certain horizontal direction exceeds the preset error, the texture corresponding to the first chain code is determined as the texture with a bad point. The preset directions include the horizontal direction (from left to right) and the vertical direction (from top to bottom). For example, assuming that the preset error is 2 pixels and the preset texture chain code is [(3, 2), (4, 2)], the chain code only includes the horizontal direction, and the total length is 4. One of the first chain codes is [(1, 1), (2, 2), (3, 2), (4, 2)], the total length of the chain code in the horizontal direction is 4, and the total length in the vertical direction is 3. The difference in the vertical direction exceeds the preset error, so the texture corresponding to the first chain code is determined as the texture with a bad point.

[0117] Then, the mutation unit is determined based on the part exceeding the preset error. The mutation unit is the unit with a bad point. For example, when the preset texture chain code is [(3, 2), (4, 2)], and the first chain code is [(1, 1), (2, 2), (3, 2), (4, 2)], the part exceeding the preset error is the part corresponding to [(1, 1), (2, 2)], so the pixel points corresponding to [(1, 1), (2, 2)] are taken as the mutation unit. If the difference in the horizontal direction exceeds the preset error, for example, the preset texture chain code is [(3, 5), (4, 5)], and the first chain code is [(3, 2), (4, 2)], the geometric centers corresponding to the two chain codes are first determined. If the geometric centers are the same, it means that the oxidation points appear at both ends of the horizontal direction. According to the difference in the horizontal direction, it can be obtained that the mutation unit is the three pixel points to the left of the left end point corresponding to the first chain code and the three pixel points to the right of the right end point corresponding to the first chain code.

[0118] If the difference in the horizontal direction and the difference in the vertical direction both exceed the preset error, a sudden change unit in one direction is first determined according to the direction of the preset texture chain code, and the first chain code is updated, and then a sudden change unit in the other direction is determined. For example, assuming that the preset texture chain code is [(3, 5), (4, 5)], and the first chain code is [(1, 1), (2, 2), (3, 1), (4, 3)], the direction corresponding to the preset texture chain code is first determined to be the horizontal direction, and then the sudden change unit in the vertical direction is determined, and the specific steps are the same as described above, and will not be repeated here. The part of the first chain code that has been determined as the sudden change unit is deleted to obtain [(3, 1), (4, 3)], and the geometric center is re-determined according to the coordinates corresponding to [(3, 1), (4, 3)], and the updated first chain code is [(3, 2), (4, 2)]. Then, the sudden change unit in the horizontal direction is determined according to the updated first chain code, and the specific steps are the same as described above, and will not be repeated here. After the sudden change unit is determined, each sudden change unit is extended by one pixel point along the direction corresponding to the chain code, and the coordinates of the extended pixel points and the coordinates corresponding to the sudden change unit are used as the boundary coordinates. The specific values of the chain code in this embodiment are only examples of edge understanding, and this embodiment is not limited thereto.

[0119] According to the above analysis, it can be summarized that the determination of the boundary coordinates according to the first chain code and the preset texture chain code comprises:

[0120] Comparing the first chain code with the preset texture chain code;

[0121] Determining a sudden change unit according to the comparison result;

[0122] Determining the boundary coordinates according to the coordinates corresponding to the sudden change unit.

[0123] By using the preset error and the preset texture chain code, the normal texture fluctuations can be excluded, and the textures with real bad points can be screened out. The pixel points are extended based on the sudden change unit to obtain the boundary coordinates of the bad points. Finally, the industrial robot is controlled according to the coordinates to eliminate the bad points. Compared with the complex image processing algorithm, the position of the bad points can be accurately located based on the comparison between the chain codes, and the method is suitable for industrial scenes with insufficient computing power.

[0124] Further, the control parameters include first control parameters and second control parameters, and the application provides a step of updating the control parameters according to the second chain code and the preset texture chain code, comprising:

[0125] Determining a horizontal offset error and a rotation offset error according to the second chain code and the preset texture chain code;

[0126] Updating the first control parameters according to the horizontal offset error and the rotation offset error.

[0127] The first control parameter is specifically a punching position. After the bad points are eliminated, a punching operation needs to be performed. First, the second image after the bad points are eliminated and a second chain code corresponding to the second image are obtained. The step of determining the second chain code is the same as the step of determining the preset texture chain code, which is not repeated here. Since there are no bad points in the second image, the second chain code corresponding to each texture in the second image is the same. In order to simplify the calculation, the second chain code can be determined only according to the outermost texture, for example, the second chain code is obtained according to the skeleton line corresponding to the leftmost texture in the second image.

[0128] Specifically, in an industrial scene, the position of the punching is determined based on the denoised image. However, the original image is collected after milling, and the second image is collected after removing the bad points. The workbench where the workpiece is placed is different when the workpiece is placed on the second workbench. Positioning errors are likely to occur when the workpiece is placed on the second workbench, causing the images collected at the same angle and distance to be offset. At this time, the coordinates of the punching position are also offset. For example, as shown in Figure 4 Figure 4 The black outer frame in the figure represents the workbench, and the grid structure in the back represents the image of the workpiece. Each grid corresponds to a pixel point. The gray part represents the texture region. Figure 4 The first image in the figure is a denoised image obtained on the first workbench. Figure 4 The second image in the figure is a second image obtained on the second workbench. The point A in the denoised image corresponds to the point E in the second image. Obviously, the error of placing the workpiece on the second workbench will cause the punching position to be offset.

[0129] Specifically, the embodiment is based on first obtaining a preset texture chain code and a second chain code. The preset texture chain code is also obtained based on the skeleton line corresponding to the leftmost texture in the denoised image. Then, two coordinates in the second chain code and the preset texture chain code are obtained, respectively. The two coordinates are connected to obtain two line segments. The included angle between the two line segments is calculated to obtain a rotation offset error. Then, the line segment corresponding to the second chain code is rotated according to the rotation offset error. The horizontal offset error is obtained according to the coordinates of the rotated line segment and the line segment corresponding to the preset texture chain code. Finally, the drilling position of the industrial robot on the workpiece is adjusted according to the horizontal offset error and the rotation offset error to ensure the machining precision. The directions of the two line segments should be approximately the same, for example, Figure 4 The line segment FG and the line segment MN in the figure are both vertical directions.

[0130] Further, after the first control parameter is updated, the above-mentioned industrial robot visual recognition and positioning method further comprises:

[0131] sending an instruction to the industrial robot according to the updated first control parameter, so that the industrial robot performs a corresponding operation; ​

[0132] obtaining a third image and a third chain code corresponding to the third image;

[0133] obtaining a thermal expansion coefficient according to the second chain code and the third chain code;

[0134] updating the second control parameter according to the thermal expansion coefficient.

[0135] The second control parameter is the position of the drilling and the rotation speed of the drill bit. Specifically, after adjusting the drilling position, an instruction is sent to the industrial robot according to the adjusted position, so that the industrial robot performs the drilling operation. During the drilling process, an image is collected every preset interval as a third image, and a third chain code corresponding to the third image is obtained. Similar to the step of obtaining the second chain code, the third chain code can be obtained according to the skeleton line corresponding to the leftmost line in the third image.

[0136] Then, a thermal expansion coefficient is obtained according to the total length of the second chain code and the third chain code. , wherein is the total length of the second chain code, is the total length of the third chain code. According to the thermal expansion coefficient, the position of the drilling and the rotation speed of the drill bit performed by the industrial robot can be further adjusted.

[0137] Specifically, during the drilling process, the workpiece generates heat by friction, which in turn causes the workpiece to expand due to heat, affecting the drilling position and the drilling depth. At this time, the control parameter needs to be further adjusted, wherein the drilling depth is determined by the rotation speed of the drill bit. When adjusting the control parameter, a relationship between the rotation speed of the drill bit and the thermal expansion coefficient can be established based on historical production data, and a relationship between the drilling position deviation and the thermal expansion coefficient can be established. Finally, according to the calculated thermal expansion coefficient the control parameter is updated, and a control instruction is sent to the industrial robot according to the updated control parameter, so that the industrial robot continues drilling according to the updated control parameter. The specific relationship is not limited in this embodiment, and the relationship is established based on historical production data, which is a technical means that can be realized by those skilled in the art, and will not be described here.

[0138] In this embodiment, the horizontal offset error and the rotational offset error are obtained by comparing the chain codes, and the thermal deformation rule of the workpiece is monitored during the machining process by comparing the lengths of the chain codes. Finally, the control parameter is adjusted according to the monitored error, the scrap rate of the workpiece is reduced, and the production precision is improved.

[0139] The embodiment of the present application provides an industrial robot visual recognition and positioning system, comprising a controller and an industrial robot.

[0140] The controller comprises:

[0141] The first receiving module is configured to acquire an original image, a first image and a second image of a workpiece to be processed, and the second image is an image of the workpiece to be processed after bad points are eliminated.

[0142] The computing module is configured to obtain a preset texture chain code according to the original image, determine the bad points in the first image according to a first chain code corresponding to the first image and the preset texture chain code, and update the control parameter according to the second chain code and the preset texture chain code.

[0143] The control module is configured to send an instruction to the industrial robot according to the bad points, so that the industrial robot eliminates the bad points.

[0144] The industrial robot comprises:

[0145] The camera is configured to acquire the original image, the first image and the second image.

[0146] The second receiving module is configured to receive the instruction sent by the controller.

[0147] The mechanical arm is configured to perform a corresponding operation according to the instruction.

[0148] The industrial robot vision recognition and positioning method and system provided by the embodiment of the application convert the texture features of the workpiece into chain codes, realize the recognition of errors such as bad points, horizontal offset errors and rotational offset errors through comparison between the chain codes, and then adjust the control parameter. Compared with the prior art, the recognition and control steps based on the chain codes and the morphological principle in the embodiment reduce the demand for computing power, can be applied to control equipment in an industrial scene, and ensure production efficiency and processing precision.

[0149] Those skilled in the art will understand that the embodiments of the application can be provided as a method, a system or a computer program product. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1apparatuses that implement the functions specified in the flowchart or flowcharts and / or blocks. Figure 1

[0151] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart or flowcharts and / or blocks. Figure 1 apparatuses that implement the functions specified in the flowchart or flowcharts and / or blocks. Figure 1

[0152] The above merely describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical scheme falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application shall also be considered as falling within the protection scope of the present application.​​

Claims

1. A visual recognition and positioning method for industrial robots, characterized in that, include: Obtain the first image and preset texture chain code of the workpiece to be processed; Based on the first chain code corresponding to the first image and the preset texture chain code, bad pixels in the first image are determined, and instructions are sent to the industrial robot based on the bad pixels so that the industrial robot can eliminate the bad pixels. Obtain a second image of the workpiece to be processed and its corresponding second chain code, wherein the second image is the image of the workpiece to be processed after the defective points have been eliminated; Update the control parameters based on the second chain code and the preset texture chain code.

2. The industrial robot visual recognition and positioning method according to claim 1, characterized in that, Obtaining the preset texture chain code includes: Obtain the original image corresponding to the workpiece to be processed; Based on the preset structural elements and the original image, a denoised image is obtained; Perform an iterative operation on the denoised image to obtain the skeleton lines corresponding to the texture features in the denoised image; The preset texture chain code is obtained based on the pixels included in the skeleton line.

3. The industrial robot visual recognition and positioning method according to claim 2, characterized in that, The step of obtaining a denoised image based on preset structural elements and the original image includes: For each pixel in the original image, its corresponding preset neighborhood is determined according to the structuring element; The pixel values ​​of the corresponding pixels in the preset neighborhood are updated based on the minimum and maximum pixel values ​​in the preset neighborhood to obtain the denoised image.

4. The industrial robot visual recognition and positioning method according to claim 2, characterized in that, The step of performing an iterative operation based on the denoised image to obtain the skeleton lines corresponding to the texture features in the denoised image includes: The denoised image is converted into a binary image; An iterative operation is performed on the binary image. The iterative operation includes traversing all pixels in the current binary image, deleting edge pixels from all pixels, until a preset termination condition is reached, and outputting the current binary image. The skeleton line is obtained based on the non-zero pixels in the current binary image.

5. The industrial robot visual recognition and positioning method according to claim 2, characterized in that, The step of obtaining the preset texture chain code based on the pixels included in the skeleton line includes: Determine the geometric center based on the coordinates corresponding to the pixel. Starting from the geometric center, traverse the pixels to obtain the relative direction of each pixel, where the relative direction is the direction relative to the previous pixel. The preset texture chain code is obtained based on the relative direction and its corresponding length.

6. The industrial robot visual recognition and positioning method according to claim 1, characterized in that, The step of determining bad pixels in the first image based on the first chain code corresponding to the first image and the preset texture chain code includes: Determine the boundary coordinates based on the first chain code and the preset texture chain code; The bad points are determined based on the boundary coordinates and the preset error.

7. The industrial robot visual recognition and positioning method according to claim 6, characterized in that, The step of determining the boundary coordinates based on the first chain code and the preset texture chain code includes: Compare the first chaincode with the preset texture chaincode; The mutation unit was determined based on the comparison results; The boundary coordinates are determined based on the coordinates corresponding to the mutation units.

8. The industrial robot visual recognition and positioning method according to claim 1, characterized in that, The control parameters include a first control parameter and a second control parameter. Updating the control parameters based on the second chaincode and the preset texture chaincode includes: Based on the second chain code and the preset texture chain code, determine the horizontal offset error and the rotational offset error; The first control parameter is updated based on the horizontal offset error and the rotational offset error.

9. The industrial robot visual recognition and positioning method according to claim 8, characterized in that, After updating the first control parameter, the industrial robot visual recognition and positioning method further includes: The industrial robot is given instructions based on the updated first control parameters so that it can perform the corresponding operation. Obtain the third image and its corresponding third chain code; The coefficient of thermal expansion is obtained based on the second chain code and the third chain code; The second control parameter is updated based on the coefficient of thermal expansion.

10. An industrial robot visual recognition and positioning system, used to implement the industrial robot visual recognition and positioning method as described in any one of claims 1-9, characterized in that, Including controllers and industrial robots; The controller includes: The first receiving module is used to acquire the original image, the first image, and the second image of the workpiece to be processed, wherein the second image is the image of the workpiece to be processed after the defects have been eliminated; The calculation module is used to obtain a preset texture chain code based on the original image; determine bad pixels in the first image based on the first chain code corresponding to the first image and the preset texture chain code; and update control parameters based on the second chain code and the preset texture chain code. The control module is used to send instructions to the industrial robot based on the defective pixels, so that the industrial robot can eliminate the defective pixels; The industrial robots include: A camera is used to capture the original image, the first image, and the second image; The second receiving module is used to receive instructions sent by the controller; A robotic arm is used to perform corresponding operations according to the instructions.

Images