Vision feature based uncalibrated motion control method

Abstract

The present application relates to a kind of visual feature-based calibration-free motion control method, belong to motion control technical field, solve the real-time poor calculation difficulty high problem in prior art motion control.It is collected by industrial camera the visual feature image of target object in current position;The visual feature image is preprocessed;Object feature point extraction is carried out to the visual feature image after preprocessing and the descriptor of object specific point is calculated, and the descriptor of object feature point is matched to current position target object feature point and desired position target object feature point based on object feature point;The relative deviation of the coordinates of current position target object feature point after matching and the coordinates of desired position target object feature point is calculated;The control quantity of motion mechanism is calculated based on relative deviation;Motion mechanism is displaced based on the control quantity, reaches desired position.It is realized that motion control is fast, accurate and efficient.

Description

Technical Field

[0001] This invention relates to the field of motion control technology, and in particular to a calibration-free motion control method based on visual features. Background Technology

[0002] In recent years, with the continuous development of science and technology and the expanding application demand in the field of motion control, people have placed higher demands on the level of intelligence of motion control and its adaptability to the external environment. Due to its flexibility and adaptability, vision-based control methods have gradually become an important technical means in the fields of industrial automation and intelligent manufacturing.

[0003] Currently, mainstream visual motion control systems are mainly divided into two categories: the first is position-based visual motion control systems, and the second is image-based visual motion control systems. Position-based systems have relatively simple controller design, but they require high measurement and calibration accuracy, necessitating a tedious and complex calibration process. Image-based visual motion control systems do not require three-dimensional spatial position information; they only need to collect the target's feature information in a two-dimensional plane as feedback information. However, this method also has some problems: existing feature extraction methods, such as SIFT and SURF, while able to extract feature points, require tedious and complex calibration processes, making it difficult to meet real-time requirements; and existing feature point matching algorithms consume significant computational resources when processing large-scale data, making efficient parallel processing and optimization difficult. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a calibration-free motion control method based on visual features, in order to solve the problems of existing motion control methods requiring complex calibration, poor real-time performance, and high computational difficulty.

[0005] On one hand, embodiments of the present invention provide a calibration-free motion control method based on visual features, comprising:

[0006] Visual feature images of the target object at the current location are acquired using an industrial camera;

[0007] The visual feature image is preprocessed;

[0008] The preprocessed visual feature image is processed to extract object feature points and calculate descriptors for specific points of the object. Based on the descriptors of the object feature points, the feature points of the target object at the current position are matched with the feature points of the target object at the desired position.

[0009] Calculate the relative deviation between the coordinates of the feature points of the target object at the current position after matching and the coordinates of the feature points of the target object at the desired position;

[0010] The control quantity of the motion mechanism is calculated based on the relative deviation.

[0011] The motion mechanism moves according to the control quantity to achieve the desired position.

[0012] As a further improvement to this application, the extraction of object feature points from the preprocessed visual feature image and the calculation of descriptors for specific points include:

[0013] Oriented FAST feature point detection is performed on the preprocessed visual feature image to detect all candidate feature points;

[0014] The detected candidate feature points are processed uniformly using the quadtree method to obtain object feature points;

[0015] The orientation of object feature points is determined by the gray-scale centroid method. Based on the determined orientation of object feature points, descriptors of object feature points are generated by the BRIEF algorithm.

[0016] As a further improvement to this application, the extraction of candidate feature points from the preprocessed visual feature image includes:

[0017] In the visual feature image, each pixel is used as a detection point, and candidate feature points are extracted through the following steps:

[0018] Record the grayscale value of the detection point;

[0019] A circular detection area is defined with the detection point as the center and a preset pixel range as the radius, and the gray value of each pixel within the detection area is recorded.

[0020] Calculate the difference between the gray value of each pixel within the detection area and the gray value of the detection point;

[0021] If the absolute value of the difference is greater than the preset threshold, then the corresponding pixel is confirmed as a significant point;

[0022] If there are n consecutive salient points at a given location, then the detection point is determined as a candidate feature point, where n is the preset number of consecutive salient points.

[0023] As a further improvement to this application, the object feature points are obtained by uniformly processing the detected candidate feature points using the quadtree method, including:

[0024] The visual feature image is divided into multiple initial quadtree regions;

[0025] Within each initial quadtree region, the Harris response value of all candidate feature points is calculated, and the candidate feature points with Harris response values ​​higher than a preset response value threshold are selected as the first feature points.

[0026] Calculate the distance between each first feature point within the same initial quadtree region, and evaluate the uniformity of each initial quadtree region based on the distance;

[0027] If the distribution of the first feature point within the initial quadtree region meets the preset uniformity standard, then the first feature point with the highest Harris response value is selected as the object feature point of the initial quadtree region.

[0028] If the distribution of the first feature point within the initial quadtree region does not meet the preset uniformity standard, the initial quadtree region is divided into four sub-regions, and the distance between the first feature points within each sub-region is calculated again. The uniformity of each sub-region is evaluated based on the distance.

[0029] If at least one sub-region satisfies the preset uniformity standard, the division is stopped, and the first feature point with the highest Harris response value among all sub-regions that satisfy the preset uniformity standard is taken as the object feature point of the initial quadtree region.

[0030] If none of the sub-regions meet the preset uniformity standard, then each sub-region is further divided into four sub-regions, and the distance between the first feature points in each sub-region is calculated. The uniformity of each sub-region is evaluated based on the distance, until a sub-region that meets the preset uniformity standard is found or the sub-regions are divided to reach the preset minimum sub-region size.

[0031] As a further improvement to this application, the method of determining the direction of object feature points using the gray-scale centroid method includes:

[0032] A circular neighborhood region of the object's feature points is defined with the feature points as the center and r as the radius.

[0033] Traverse all pixels within the neighborhood region and construct a pixel matrix that can cover all pixels within the neighborhood region;

[0034] Calculate the relative position of each pixel with respect to the object feature points, and convert the relative position into the index information of the pixel matrix, as shown in the calculation formula (1);

[0035] i = round(Δx / r × (m-1)) + (m-1)

[0036] j=round(Δy / r×(n-1))+(n-1) (1)

[0037] In the formula, m and n are the dimensions of the pixel matrix in the horizontal and vertical directions, respectively; Δx and Δy are the relative positions of the pixel and the object feature points on the x and y axes, respectively; i and j are the index information of the pixel on the x and y axes of the pixel matrix, and round() is the rounding down function.

[0038] Based on the index information, all pixels within the neighborhood area are filled into the pixel matrix;

[0039] Calculate the weighted sum of the gray values ​​of pixels along the x-axis and y-axis in the pixel matrix, where the weights are determined based on the relative positions of the pixels to the object feature points;

[0040] The coordinates of the gray centroid are determined based on the ratio of the weighted sum to the total weight, as shown in formula (2).

[0041]

[0042]

[0043] In the formula, G x and G y S represents the x-axis and y-axis coordinates of the grayscale centroid; x and S y w(x) is the weighted sum of the grayscale values ​​of pixels along the x and y axes. i y j ) is the coordinate (x i y j The weight of each pixel;

[0044] Based on the gray-scale centroid and the coordinates of the object's feature points, the orientation of the object's feature points is obtained.

[0045] As a further improvement to this application, based on the determined orientation of the object feature points, the descriptor of the object feature points generated by the BRIEF algorithm includes:

[0046] Divide the circular neighborhood region centered on the object's feature points into several identical sector regions;

[0047] In the fan-shaped region where the object feature point is located, select a pixel on the object feature point direction line as the reference starting point, and take the intersection of the object feature point direction line and the arc boundary of the fan-shaped region as the reference ending point.

[0048] In other sector regions, the direction of the relative direction line within each sector region is calculated based on the number of sector regions and the direction of the object feature points;

[0049] Select a pixel on each relative direction line as the relative starting point, and take the intersection of the relative direction line and the arc boundary of the corresponding sector area as the relative ending point.

[0050] Form point pairs between the starting and ending points of each sector region;

[0051] Descriptors of object feature points are obtained based on the gray values ​​of the starting and ending points of each sector region.

[0052] As a further improvement to this application, a descriptor for the object's feature points is obtained based on the grayscale values ​​of the start and end points of each sector region, including:

[0053] Compare the gray values ​​of the starting point and the ending point. If the gray value of the starting point is higher than that of the ending point, assign a binary value of 1 to the point pair; otherwise, assign a binary value of 0 to the point pair.

[0054] Arrange the binary results of all point pairs in the order of sector regions to form a binary code, which is a descriptor of the feature points of the object.

[0055] As a further improvement to this application, the matching of the feature points of the target object at the current position with the feature points of the target object at the desired position based on the descriptor of the object feature points includes:

[0056] In the visual feature image corresponding to the desired location, obtain the candidate matching feature points corresponding to each object feature point;

[0057] Calculate the Hamming distance between the descriptor of each object feature point and the corresponding candidate matching feature point descriptor;

[0058] Candidate matching feature points whose Hamming distance is less than or equal to a preset Hamming distance threshold are selected as matching points for the object's feature points.

[0059] As a further improvement to this application, the control quantity of the motion mechanism is calculated based on the relative deviation, including:

[0060] Obtain the coordinates of the object feature points and the coordinates of the matching points on the two-dimensional plane, and calculate the difference between the projected coordinates of the matched object feature points and the matching points on the two-dimensional plane.

[0061] The difference in projected coordinates on a two-dimensional plane is converted into a displacement in three-dimensional space using the image Jacobian matrix.

[0062] The displacement is converted into a control quantity for the motion mechanism.

[0063] As a further improvement to this application, the image Jacobian matrix is ​​constructed as follows:

[0064] The projection relationship between three-dimensional space and two-dimensional plane is determined based on the focal length of the industrial camera, as shown in the calculation formula (3);

[0065]

[0066]

[0067] In the formula, f is the focal length of the industrial camera; X, Y, and Z are the coordinates in three-dimensional space, and u and v are the coordinates in a two-dimensional plane.

[0068] The partial derivatives of the projection relationship are obtained by performing partial derivative operations on the two-dimensional plane coordinates and the three-dimensional spatial coordinates. Based on this, the Jacobian matrix is ​​constructed, and the Jacobian matrix is ​​shown in the calculation formula (4).

[0069]

[0070] in,

[0071] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0072] 1. The calibration-free motion control method of the present invention eliminates the cumbersome calibration steps in traditional motion control systems, thereby reducing the time and cost of system deployment and maintenance;

[0073] 2. By performing directional FAST feature point detection on the preprocessed visual feature image, and using the quadtree method to process the detected candidate feature points for uniformity, the orientation of the object feature points is determined by the gray-scale centroid method, and the descriptor of the object feature points is generated by the BRIEF algorithm, the feature matching speed is improved, which meets the requirements of real-time control.

[0074] 3. Since no complex calibration process is required, the calculation process is simplified, and it can be more widely applied to various industrial automation and intelligent manufacturing scenarios to meet the control requirements with high real-time and accuracy requirements.

[0075] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0076] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0077] Figure 1 A flowchart illustrating a calibration-free motion control method based on visual features, provided in an embodiment of the present invention;

[0078] Figure 2 This is a schematic diagram of a calibration-free motion control system based on visual features, provided as an embodiment of the present invention. Detailed Implementation

[0079] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0080] Example 1

[0081] A specific embodiment of the present invention discloses a calibration-free motion control method based on visual features, such as... Figure 1 As shown. The method includes:

[0082] S1. Acquire visual feature images of the target object at the current location using an industrial camera;

[0083] Specifically, the selection of industrial cameras needs to ensure that the frame rate and resolution are high enough. The frame rate needs to be sufficient to capture the fast movement of the target object in dynamic environments. The resolution of the industrial camera needs to ensure that the detailed features of the target object are captured.

[0084] Visual feature images can include shape information, texture information, and color information of an object. Shape information includes the outline and shape of the target object, texture information is the texture features of the target object, and color information can be used as an auxiliary feature to distinguish between objects with similar shape and texture information. It is understood that the target object in this invention can be any object in the field of motion control that requires precise control of position and orientation, such as industrial parts, robotic arm tools, etc., without being overly limited here.

[0085] S2. Preprocess the visual feature image;

[0086] In some embodiments, preprocessing of the visual feature image includes noise removal via Gaussian filtering and feature texture enhancement. Gaussian filtering is an image smoothing technique that effectively removes random noise from an image while preserving its main features. Feature texture enhancement enhances the feature texture in the image, making feature points more prominent and facilitating subsequent feature point extraction and matching.

[0087] In practice, removing noise from visual feature images using Gaussian filtering first requires selecting an appropriate Gaussian kernel size and then performing a convolution operation on the image. The choice of Gaussian kernel size depends on the intensity of the noise and the level of detail in the image.

[0088] Texture enhancement can employ various methods to enhance the texture features of an image, such as contrast enhancement and edge enhancement. The specific method chosen depends on the characteristics of the image and the requirements of subsequent processing.

[0089] In some embodiments, preprocessing may also include grayscale processing, image scaling, histogram equalization, image normalization, image rotation, and cropping, etc., which will not be elaborated on here.

[0090] The preprocessed visual feature image is processed to extract object feature points and calculate descriptors for specific points of the object. Based on the descriptors of the object feature points, the feature points of the target object at the current position are matched with the feature points of the target object at the desired position.

[0091] Specifically, extracting object feature points from the preprocessed visual feature image and calculating descriptors for specific points includes the following steps:

[0092] S31. Perform directional FAST feature point detection on the preprocessed visual feature image to detect all candidate feature points;

[0093] FAST (Features from Accelerated Segment Test) is an image feature corner detection algorithm that can detect feature points under conditions of changing viewpoint, lighting, or partial occlusion.

[0094] Furthermore, in the visual feature image, each pixel is used as a detection point, and candidate feature points are extracted through the following steps:

[0095] S311. Record the grayscale value of the detection point;

[0096] For each pixel in a visual feature image, its grayscale value is recorded. For grayscale images, each pixel corresponds to only one numerical value, i.e., the grayscale value. The grayscale values ​​can be recorded by accessing the pixels of a visual feature image using the OpenCV library. When accessing each pixel, its grayscale value is read and stored in a data structure, such as an array or list. The grayscale value can be an 8-bit, 16-bit, or 32-bit integer, depending on the depth of the image.

[0097] S312. Determine a circular detection area with the detection point as the center and the preset pixel range as the radius, and record the gray value of each pixel within the detection area.

[0098] Centered on the current detection point, a circular detection area is determined with a radius based on a preset range of pixels. Within this area, the central pixel is the detection point, and the radius is determined by the detection accuracy and the size of the visual feature image.

[0099] S313. Calculate the difference between the gray value of each pixel in the detection area and the gray value of the detection point;

[0100] Within the circular detection area, the grayscale value of each pixel is recorded and compared with the grayscale value of the detection point to calculate the difference.

[0101] S314. If the absolute value of the difference is greater than the preset threshold, then the corresponding pixel is confirmed as a significant point.

[0102] The preset threshold is set based on the properties of the visual feature image itself, including the image's contrast, brightness, and noise level. High-contrast images may require a lower threshold, while low-contrast or high-noise images may require a higher threshold. Specifically, the threshold can be determined by analyzing the image's grayscale histogram. The preset threshold can also be dynamically adjusted according to specific circumstances.

[0103] S315. If there are n consecutive salient points in a given location, then the detection point is determined as a candidate feature point, where n is the preset number of consecutive salient points.

[0104] Specifically, consecutive salient points refer to checking adjacent pixels starting from the current salient point along one or more directions (horizontal, vertical, diagonal). If n consecutive salient points are found, the condition is met. When n consecutive salient points are detected, the starting detection point is marked as a candidate feature point. The number of consecutive salient points, n, is a preset value that can be determined based on the properties of the visual feature image itself or through experimentation.

[0105] S32. Use the quadtree method to perform uniform processing on the detected candidate feature points to obtain object feature points;

[0106] The quadtree method is a data uniformity segmentation algorithm. In this invention, the quadtree method can be used to perform uniformity processing on the detected candidate feature points, that is, to select object feature points that are evenly distributed and highly representative from the candidate feature points.

[0107] Specifically, the object feature points obtained by uniformly processing the detected candidate feature points using the quadtree method include:

[0108] S321. Divide the visual feature image into multiple initial quadtree regions;

[0109] During implementation, the visual feature image is divided into several equal initial quadtree regions based on its width and height. The initial quadtree regions are the first level.

[0110] S322. Within each initial quadtree region, calculate the Harris response value of all candidate feature points, and select the candidate feature points whose Harris response values ​​are higher than the preset response value threshold as the first feature points.

[0111] The Harris response value is an indicator for evaluating the local variation of image brightness around feature points. The calculation of the Harris response value can refer to the calculation method of existing technology, and will not be elaborated further. After calculating the Harris response value of all candidate feature points, all candidate feature points with Harris response values ​​higher than a preset threshold within each initial quadtree region are selected as the first feature points.

[0112] S323. Calculate the distance between each first feature point within the same initial quadtree region, and evaluate the uniformity of each initial quadtree region based on the distance;

[0113] The uniformity of each initial quadtree region is evaluated by calculating the distances between the first feature points. The uniformity can be evaluated by calculating the distances between each pair of the first feature points and calculating the distance distribution parameters (such as standard deviation or distance ratio) of all distances.

[0114] S324. If the distribution of the first feature point within the initial quadtree region meets the preset uniformity standard, then the first feature point with the highest Harris response value is selected as the object feature point of the initial quadtree region.

[0115] Whether the uniformity standard is met can be assessed by setting a standard deviation threshold or an upper limit for the maximum / minimum distance ratio, and comparing the calculated standard deviation or distance ratio with the preset uniformity standard. If the parameter is within the threshold range, the distribution within the region is considered uniform.

[0116] S325. If the distribution of the first feature points within the initial quadtree region does not meet the preset uniformity standard, the initial quadtree region is divided into four sub-regions, and the distance between the first feature points within each sub-region is calculated again. The uniformity of each sub-region is evaluated based on the distance.

[0117] If the initial quadtree region does not meet the uniformity criterion, it is further divided into four sub-regions as the second level. The uniformity of the second level is then evaluated again using the uniformity evaluation method described in S325 above.

[0118] S326. If at least one sub-region satisfies the preset uniformity standard, then stop dividing and take the first feature point with the highest Harris response value among all sub-regions that satisfy the preset uniformity standard as the object feature point of the initial quadtree region.

[0119] If, in the second level, there exists a sub-region that meets the preset uniformity standard, then the first feature point with the highest Harris response value in that sub-region is taken as the object feature point of the initial quadtree region. If there are multiple sub-regions that meet the preset uniformity standard, then the Harris response values ​​of each first feature point in the multiple sub-regions are compared, and the first feature point with the highest Harris response value in the multiple sub-regions is taken as the object feature point of the initial quadtree region.

[0120] S327. If none of the sub-regions meet the preset uniformity standard, then each sub-region is further divided into four sub-regions, and the distance between the first feature points in each sub-region is calculated. The uniformity of each sub-region is evaluated based on the distance until a sub-region that meets the preset uniformity standard is found or the sub-regions are divided to reach the preset minimum sub-region size.

[0121] If none of the sub-regions in the second level meet the preset uniformity standard, then all the sub-regions in the second level will be further divided into four sub-regions in the third level, and so on, until at least one sub-region in a certain level meets the preset uniformity standard or reaches the minimum sub-region size, at which point the division will stop.

[0122] If at least one sub-region in a certain level meets the preset uniformity standard, then the first feature point with the highest Harris response value among all sub-regions that meet the preset uniformity standard is taken as the object feature point of the initial quadtree region. If the size of the divided sub-regions has reached the set minimum sub-region size, and no sub-region that meets the preset uniformity standard has been found, it means that the Harris response value threshold setting is unreasonable, resulting in the inability to find a sub-region that meets the preset uniformity standard under the current image resolution and preset parameters. The object feature point can be obtained by adjusting the preset Harris response value threshold and performing uniformity processing on the candidate feature points again.

[0123] S33. Determine the orientation of object feature points using the gray-scale centroid method. Based on the determined orientation of object feature points, generate descriptors for object feature points using the BRIEF algorithm.

[0124] Specifically, determining the orientation of object feature points using the gray-scale centroid method includes:

[0125] A circular neighborhood region of the object's feature points is defined with the feature points as the center and r as the radius.

[0126] Traverse all pixels within the neighborhood region and construct a pixel matrix that can cover all pixels within the neighborhood region;

[0127] Calculate the relative position of each pixel with respect to the object feature points, and convert the relative position into the index information of the pixel matrix, as shown in the calculation formula (1);

[0128] i = round(Δx / r × (m-1)) + (m-1)

[0129] j=round(Δy / r×(n-1))+(n-1) (1)

[0130] In the formula, m and n are the dimensions of the pixel matrix in the horizontal and vertical directions, respectively; Δx and Δy are the relative positions of the pixel and the object feature points on the x and y axes, respectively; i and j are the index information of the pixel on the x and y axes of the pixel matrix, and round() is the rounding down function.

[0131] It's important to note that the grayscale centroid method is a technique for determining the orientation of feature points in an image. It determines the orientation of feature points based on their grayscale values ​​and their positions relative to the feature points. The grayscale centroid refers to the weighted average position of all pixels within a region, with the weights typically proportional to the pixel's grayscale value. In practice, the weight of each pixel depends not only on its position but also on its grayscale value; brighter or darker pixels receive greater weight when calculating the centroid.

[0132] First, a circular neighborhood region is defined centered on the feature point. For each pixel within the neighborhood region, its offset (relative position) Δx and Δy relative to the object feature point are calculated. Then, the offsets are converted into indices in the pixel matrix.

[0133] Based on the index information, all pixels within the neighborhood area are filled into the pixel matrix;

[0134] Calculate the weighted sum of the gray values ​​of pixels along the x-axis and y-axis in the pixel matrix, where the weights are determined based on the relative positions of the pixels to the object feature points;

[0135] Specifically, the weights are positively correlated with the relative positions of pixels to object feature points. The closer a pixel is to an object feature point, the greater its influence on the centroid direction; the farther away it is, the smaller the influence. That is, the closer a pixel is to an object feature point, the greater its weight; conversely, the farther a pixel is from an object feature point, the smaller its weight. The sum of pixel weights is 1 in the x-axis and y-axis directions.

[0136] The coordinates of the gray centroid are determined based on the ratio of the weighted sum to the total weight, as shown in formula (2).

[0137]

[0138]

[0139] In the formula, G x and G yS represents the x-axis and y-axis coordinates of the grayscale centroid; x and S y w(x) is the weighted sum of the grayscale values ​​of pixels along the x and y axes. i y j ) is the coordinate (x i y j The weight of each pixel;

[0140] S x and S y The calculation is shown in formula (3):

[0141]

[0142]

[0143] In the formula, Z(x) i y j ) is the coordinate (x i y j The grayscale value of the pixel.

[0144] Based on the gray-scale centroid and the coordinates of the object's feature points, the orientation of the object's feature points is obtained.

[0145] Specifically, the coordinates of the grayscale centroid are (G x G y The coordinates of the object's feature points are (X... c Y c ), with object feature points (X) c Y c Starting from the gray centroid (G), x G y Construct a direction vector for the endpoint. As shown in calculation formula (4):

[0146]

[0147] The direction vector can be calculated. The direction of the object's feature points is obtained by the direction angle θ with the x-axis, as shown in the calculation formula (5);

[0148]

[0149] The direction angle θ represents the rotation direction of the direction vector relative to the x-axis.

[0150] Generating descriptors for object feature points using the BRIEF algorithm includes the following steps:

[0151] S331. Divide the circular neighborhood region centered on the object's feature points into several identical sector regions.

[0152] The BRIEF algorithm is a method for generating image feature point descriptors, particularly suitable for fast feature point matching. It divides a circular neighborhood centered on an object's feature point into several identical sector regions, each with an equal central angle. The number of sector regions is typically 4, 8, or 16.

[0153] S332. In the fan-shaped region where the object feature points are located, select a pixel on the object feature point direction line as the reference starting point, and take the intersection of the object feature point direction line and the arc boundary of the fan-shaped region as the reference ending point.

[0154] Within the sector region where the object's feature points are located, a pixel is selected as the reference starting point. The reference starting point is located on the direction line of the object's feature points. The intersection of the direction line of the object's feature points and the arc boundary of the sector region is taken as the reference ending point. The reference starting point and the reference ending point define the boundary of the feature descriptor.

[0155] S333. In other sector regions, based on the number of sector regions and the direction of object feature points, calculate the direction of the relative direction line within each sector region;

[0156] For other sector regions, firstly, the relative angle is determined based on the positional relationship between the sector region and the sector region where the object feature point is located. The relative angle is an integer multiple of the central angle of the sector. Then, the angle between the direction line of the object feature point and the sector region it is located is determined based on the direction line of the object feature point. The angle is added to the relative angle to the relative direction line in the current sector region.

[0157] For example, assuming the direction of the object's feature point is 60 degrees (the angle with the positive x-axis), the circular neighborhood is divided into four sector regions, with each sector having a central angle of 90 degrees. The relative angles of each sector are integer multiples of the central angle, i.e., 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0158] The feature point of the object is oriented in the first sector. Following a counter-clockwise direction, the other sectors are designated as the second, third, and fourth sectors. The second sector, relative to the first, is the next sector in the counter-clockwise direction; therefore, the relative angle between the second sector and the first sector is 90 degrees, and the direction of the relative direction line within the second sector is 150 degrees. Similarly, the directions of the relative direction lines within the third and fourth sectors are 240 degrees and 330 degrees, respectively.

[0159] Each relative direction line intersects the direction line of the object's feature point at the object's feature point.

[0160] S334. Select a pixel on each relative direction line as the relative starting point, and take the intersection of the relative direction line and the arc boundary of the corresponding sector area as the relative ending point.

[0161] S335. Form point pairs between the starting and ending points of each sector region;

[0162] S336. Obtain the descriptor of the object feature points based on the gray values ​​of the starting and ending points of each sector region.

[0163] Feature point descriptors are a method of encoding information to represent feature points in an image in computer vision. In the BRIEF algorithm, feature point descriptors are generated by comparing the gray values ​​of point pairs within neighboring regions.

[0164] Furthermore, based on the grayscale values ​​of the start and end points of each sector region, a descriptor for the object's feature points is obtained, including:

[0165] Compare the gray values ​​of the starting point and the ending point. If the gray value of the starting point is higher than that of the ending point, assign a binary value of 1 to the point pair; otherwise, assign a binary value of 0 to the point pair.

[0166] Arrange the binary results of all point pairs in the order of sector regions to form a binary code, which is a descriptor of the feature points of the object.

[0167] In implementation, the feature point descriptor can follow the starting order of the sector regions, beginning with the first point pair of the first sector and ending with the last point pair of the last sector. The binary code is a compact binary string that can uniquely represent the characteristics of the feature point.

[0168] Furthermore, matching the feature points of the target object at the current location with the feature points of the target object at the desired location based on the descriptor of the object feature points includes:

[0169] In the visual feature image corresponding to the desired location, obtain the candidate matching feature points corresponding to each object feature point;

[0170] Calculate the Hamming distance between the descriptor of each object feature point and the corresponding candidate matching feature point descriptor;

[0171] Candidate matching feature points whose Hamming distance is less than or equal to a preset Hamming distance threshold are selected as matching points for the object's feature points.

[0172] It should be noted that the Hamming distance is an index that measures the difference between two strings of equal length. It measures the similarity between two feature point descriptors by calculating the number of different characters in the same or corresponding positions of the two strings.

[0173] Specifically, a bitwise XOR operation is performed on the descriptor of each object feature point and the corresponding candidate matching feature point descriptor. The rule for the XOR operation is:

[0174] 0 XOR 0 = 0

[0175] 0 XOR 1 = 1

[0176] 1 XOR 0 = 1

[0177] 1 XOR 1 = 0

[0178] The XOR operation highlights the different bits between two descriptors at corresponding positions. The number of 1s in the XOR result string is the Hamming distance between the two descriptors.

[0179] S4. Calculate the relative deviation between the coordinates of the feature points of the target object at the current position after matching and the coordinates of the feature points of the target object at the desired position;

[0180] After obtaining the feature points that match the feature points of the target object at the current position, the relative deviation between the two can be calculated based on their coordinates.

[0181] S5. Calculate the control quantity of the motion mechanism based on the relative deviation;

[0182] Specifically, calculating the control quantity of the motion mechanism based on the relative deviation includes the following steps:

[0183] S51. Obtain the coordinates of the object feature points and the coordinates of the matching points on the two-dimensional plane, and calculate the difference between the projected coordinates of the matched object feature points and the matching points on the two-dimensional plane.

[0184] S52. Using the image Jacobian matrix, the difference in projected coordinates on the two-dimensional plane is converted into a displacement in three-dimensional space.

[0185] Specifically, the image Jacobian matrix is ​​constructed as follows:

[0186] The projection relationship between three-dimensional space and two-dimensional plane is determined based on the focal length of the industrial camera, as shown in the calculation formula (6).

[0187]

[0188]

[0189] In the formula, f is the focal length of the industrial camera; X, Y, and Z are the coordinates in three-dimensional space, and u and v are the coordinates in a two-dimensional plane.

[0190] The partial derivatives of the projection relationship are obtained by performing partial derivative operations on the two-dimensional plane coordinates and the three-dimensional spatial coordinates. Based on this, the Jacobian matrix is ​​constructed, and the Jacobian matrix is ​​shown in the calculation formula (7).

[0191]

[0192] in,

[0193] The difference in projected coordinates on the two-dimensional plane is converted into the displacement in three-dimensional space as shown in the calculation formula (8);

[0194]

[0195]

[0196]

[0197] In the formula, Δx and Δy are the differences in projected coordinates on the x-axis and y-axis image planes, respectively; ΔX, ΔY, and ΔZ are the displacements in three-dimensional space.

[0198] S53. Convert the displacement into a control quantity for the motion mechanism.

[0199] Specifically, after determining the displacement in three-dimensional space, the motion quantities of each degree of freedom of the motion mechanism are determined based on the displacement quantities, and the control quantities of the motion mechanism are defined based on the motion quantities of each degree of freedom, such as the PWM value of the motor, the angle of the servo motor, and the flow rate of the hydraulic system.

[0200] S6. The motion mechanism moves according to the control quantity to reach the desired position.

[0201] Compared with the prior art, the calibration-free motion control method based on visual features provided in this embodiment has the following advantages:

[0202] 1. It eliminates the cumbersome calibration steps required by traditional motion control systems, reducing system deployment time and costs; 2. By extracting object feature points from visual feature images and calculating descriptors for specific points of the object, and matching the feature points of the target object at the current position with those at the desired position based on the object feature point descriptors, the speed of feature extraction and matching is significantly improved, ensuring that the system can achieve rapid response, meet the stringent requirements of real-time control, and enhance the dynamic performance of control; 3. This invention eliminates the calibration step, has a fast response speed, and can be more flexibly applied to various industrial automation and intelligent manufacturing scenarios. It can be applied to complex environments with high requirements for real-time performance and control accuracy, improving the applicability and reliability of the system.

[0203] Example 2

[0204] A specific embodiment of the present invention discloses a calibration-free motion control system based on visual features, comprising: a host computer, an industrial camera, a motion mechanism, and an experimental platform.

[0205] like Figure 2As shown, an industrial camera is mounted on the experimental platform to acquire visual feature images of the target object at the current location.

[0206] The host computer is used to preprocess the visual feature image; extract object feature points from the preprocessed visual feature image and calculate descriptors for specific points of the object; match the target object feature points at the current position with the target object feature points at the desired position based on the descriptors of the object feature points; calculate the relative deviation between the coordinates of the target object feature points at the current position and the coordinates of the target object feature points at the desired position after matching; and calculate the control quantity of the motion mechanism based on the relative deviation.

[0207] The motion mechanism is used to perform precise displacement according to control commands issued by the host computer.

[0208] The experimental platform is used to place target objects and install industrial cameras and motion mechanisms to ensure the accuracy and stability of their relative positions.

[0209] Understandably, this vision-based, uncalibrated motion control system is related to the reference... Figure 1 The various processes in the visual feature-based uncalibrated motion method described correspond to each other and will not be repeated here.

[0210] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0211] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A calibration-free motion control method based on visual features, characterized in that, The method includes: Visual feature images of the target object at the current location are acquired using an industrial camera; The visual feature image is preprocessed; The preprocessed visual feature image is processed to extract object feature points and calculate object feature point descriptors. Based on the object feature point descriptors, the target object feature points at the current position are matched with the target object feature points at the desired position. Calculate the relative deviation between the coordinates of the feature points of the target object at the current position after matching and the coordinates of the feature points of the target object at the desired position; The control quantity of the motion mechanism is calculated based on the relative deviation. The motion mechanism moves according to the control quantity to achieve the desired position; The process of extracting object feature points from the preprocessed visual feature image and calculating the descriptors for these object feature points includes: Oriented FAST feature point detection is performed on the preprocessed visual feature image to detect all candidate feature points; The detected candidate feature points are processed uniformly using the quadtree method to obtain object feature points; The orientation of object feature points is determined by the gray-scale centroid method. Based on the determined orientation of object feature points, descriptors of object feature points are generated by the BRIEF algorithm. Candidate feature point extraction from the preprocessed visual feature image includes: In the visual feature image, each pixel is used as a detection point, and candidate feature points are extracted through the following steps: Record the grayscale value of the detection point; A circular detection area is defined with the detection point as the center and a preset pixel range as the radius, and the gray value of each pixel within the detection area is recorded. Calculate the difference between the gray value of each pixel within the detection area and the gray value of the detection point; If the absolute value of the difference is greater than the preset threshold, then the corresponding pixel is confirmed as a significant point; If there are n consecutive salient points at a given location, then the detection point is determined as a candidate feature point, where n is the preset number of consecutive salient points; The object feature points obtained by uniformly processing the detected candidate feature points using the quadtree method include: The visual feature image is divided into multiple initial quadtree regions; Within each initial quadtree region, the Harris response value of all candidate feature points is calculated, and the candidate feature points with Harris response values ​​higher than a preset response value threshold are selected as the first feature points. Calculate the distance between each first feature point within the same initial quadtree region, and evaluate the uniformity of each initial quadtree region based on the distance; If the distribution of the first feature point within the initial quadtree region meets the preset uniformity standard, then the first feature point with the highest Harris response value is selected as the object feature point of the initial quadtree region. If the distribution of the first feature point within the initial quadtree region does not meet the preset uniformity standard, the initial quadtree region is divided into four sub-regions, and the distance between the first feature points within each sub-region is calculated again. The uniformity of each sub-region is evaluated based on the distance. If at least one sub-region satisfies the preset uniformity standard, the division is stopped, and the first feature point with the highest Harris response value among all sub-regions that satisfy the preset uniformity standard is taken as the object feature point of the initial quadtree region. If none of the sub-regions meet the preset uniformity standard, then each sub-region is further divided into four sub-regions, and the distance between the first feature points in each sub-region is calculated. The uniformity of each sub-region is evaluated based on the distance, until a sub-region that meets the preset uniformity standard is found or the sub-regions are divided to reach the preset minimum sub-region size.

2. The calibration-free motion control method based on visual features according to claim 1, characterized in that, The method of determining the direction of object feature points using the gray-scale centroid method includes: A circular neighborhood region of the object's feature points is defined with the feature points as the center and r as the radius. Traverse all pixels within the neighborhood region and construct a pixel matrix that can cover all pixels within the neighborhood region; Calculate the relative position of each pixel with respect to the object feature points, and convert the relative position into the index information of the pixel matrix, as shown in the calculation formula (1); （1） In the formula, m and n are the dimensions of the pixel matrix in the horizontal and vertical directions, respectively; , represents the relative position of the pixel and the object's feature points on the x and y axes; i and j represent the pixel's index information on the x and y axes of the pixel matrix. To round down; Based on the index information, all pixels within the neighborhood area are filled into the pixel matrix; Calculate the weighted sum of the gray values ​​of pixels along the x-axis and y-axis in the pixel matrix, where the weights are determined based on the relative positions of the pixels to the object feature points; The coordinates of the gray centroid are determined based on the ratio of the weighted sum to the total weight, as shown in the calculation formula (2); （2） In the formula, and The x-axis and y-axis coordinates of the grayscale centroid; and This is a weighted sum of the grayscale values ​​of pixels along the x and y axes. coordinates The weight of each pixel; Based on the gray-scale centroid and the coordinates of the object's feature points, the orientation of the object's feature points is obtained.

3. The calibration-free motion control method based on visual features according to claim 2, characterized in that, Based on the determined orientation of object feature points, the descriptor of the object feature points generated by the BRIEF algorithm includes: Divide the circular neighborhood region centered on the object's feature points into several identical sector regions; In the fan-shaped region where the object feature point is located, select a pixel on the object feature point direction line as the reference starting point, and take the intersection of the object feature point direction line and the arc boundary of the fan-shaped region as the reference ending point. In other sector regions, the direction of the relative direction line within each sector region is calculated based on the number of sector regions and the direction of the object feature points; Select a pixel on each relative direction line as the relative starting point, and take the intersection of the relative direction line and the arc boundary of the corresponding sector area as the relative ending point. Form point pairs between the starting and ending points of each sector region; Descriptors of object feature points are obtained based on the gray values ​​of the starting and ending points of each sector region.

4. The calibration-free motion control method based on visual features according to claim 3, characterized in that, Based on the grayscale values ​​of the start and end points of each sector region, a descriptor for the object's feature points is obtained, including: Compare the gray values ​​of the starting point and the ending point. If the gray value of the starting point is higher than that of the ending point, assign a binary value of 1 to the point pair; otherwise, assign a binary value of 0 to the point pair. Arrange the binary results of all point pairs in the order of sector regions to form a binary code, which is a descriptor of the feature points of the object.

5. The calibration-free motion control method based on visual features according to claim 4, characterized in that, The object feature point-based descriptor matching of target object feature points at the current location with target object feature points at the desired location includes: In the visual feature image corresponding to the desired location, obtain the candidate matching feature points corresponding to each object feature point; Calculate the Hamming distance between the descriptor of each object feature point and the corresponding candidate matching feature point descriptor; Candidate matching feature points whose Hamming distance is less than or equal to a preset Hamming distance threshold are selected as matching points for the object's feature points.

6. The calibration-free motion control method based on visual features according to claim 5, characterized in that, The control quantities of the motion mechanism are calculated based on the relative deviation, including: Obtain the coordinates of the object feature points and the coordinates of the matching points on the two-dimensional plane, and calculate the difference between the projected coordinates of the matched object feature points and the matching points on the two-dimensional plane. The difference in projected coordinates on a two-dimensional plane is converted into a displacement in three-dimensional space using the image Jacobian matrix. The displacement is converted into a control quantity for the motion mechanism.

7. The calibration-free motion control method based on visual features according to claim 6, characterized in that, The image Jacobian matrix is ​​constructed as follows: The projection relationship between three-dimensional space and two-dimensional plane is determined based on the focal length of the industrial camera, as shown in the calculation formula (3); （3） In the formula, The focal length of the industrial camera is denoted by X, Y, and Z, which are coordinates in three-dimensional space, and u and v are coordinates in a two-dimensional plane. The partial derivatives of the projection relationship are obtained by performing partial derivative operations on the two-dimensional plane coordinates and the three-dimensional spatial coordinates. Based on this, the Jacobian matrix is ​​constructed, and the Jacobian matrix is ​​shown in the calculation formula (4). （5） in, , , , , , .

Images