Scribing robot automatic calibration method based on visual guidance

Abstract

The invention relates to the technical field of image analysis, in particular to an automatic marking robot calibration method based on visual guidance, which comprises the following steps of: establishing an image set comprising different resolution levels based on operation site image data acquired by a marking robot, and performing corner detection and straight line segment detection on each level image in parallel. According to the method, through a parallel detection mechanism of a multi-resolution hierarchical image set, angular point and straight line segment features under different scales are synchronously extracted, and a multi-scale feature point set with high robustness is constructed in combination with cross-hierarchical coordinate stability measurement and response intensity quantification. And performing dynamic screening and grouping association on the feature points based on a preset geometric constraint condition, eliminating noise interference and false detection features, and generating a candidate calibration structure set with spatial consistency. Weighted contribution value fitting is adopted, cross-scale stability and detection confidence of feature points are integrated, and geometric accuracy and anti-interference capability of calibration reference point coordinates are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to an automatic calibration method for a marking robot based on vision guidance. Background Art

[0002] The field of image analysis technology focuses on extracting, parsing and quantifying information in digital images through algorithms and computational models, covering core areas such as image enhancement, feature detection, target recognition, and spatial measurement.

[0003] Existing image analysis technologies struggle to effectively address issues such as missed or false feature detection caused by scale variations in complex scenes. For example, low-resolution images may miss small corner points, while high-resolution images are susceptible to noise. Traditional geometric constraints often rely on static threshold settings, lacking the ability to dynamically adjust for line drawing tasks. This can lead to the inadvertent deletion of valid features or the retention of redundant points during feature screening. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an automatic calibration method for a marking robot based on vision guidance.

[0005] In order to achieve the above object, the present invention adopts the following technical solution, a method for automatic calibration of a marking robot based on vision guidance, comprising the following steps:

[0006] Based on the work site image data acquired by the marking robot, a collection of images with different resolution levels is established. Corner point detection and line segment detection are performed in parallel on the images at each level to generate a multi-scale basic geometric feature point set.

[0007] Based on the multi-scale basic geometric feature point set, calling the preset geometric constraint conditions for the line drawing task, performing constraint compliance judgment on each feature point in the multi-scale basic geometric feature point set, obtaining a subset of feature points that meet the constraints, and grouping and associating the feature points based on the subset of feature points that meet the constraints to establish a candidate calibration mark structure set;

[0008] Based on the candidate calibration marker structure set, traversing the feature points in each structure, quantifying the detection response strength and the coordinate stability measurement value across scale levels in the multi-scale basic geometric feature point set, determining the contribution weight parameter of each feature point to obtain a weighted feature point contribution value, and fitting the feature point coordinates associated with each candidate structure based on the weighted feature point contribution value to obtain the calibration reference point coordinates;

[0009] Based on the coordinates of the calibrated reference point as the target position, the marking robot is instructed to perform a trial marking, and the splash dispersion morphology image at the moment when the nozzle end of the marking robot contacts the working ground is synchronously collected to obtain the microscopic mark morphological geometric features. Based on the microscopic mark morphological geometric features, the current position parameters of the tool end point of the marking robot are adjusted.

[0010] Preferably, the steps of obtaining the multi-scale basic geometric feature point set are:

[0011] Perform multi-level resolution reconstruction on the work site image data, use the Gaussian pyramid generation method to adjust the resolution layer by layer and superimpose them into a set to form a multi-resolution image set;

[0012] Based on the multi-resolution image set, corner point detection and line segment detection are performed in parallel on each level of the image, wherein the corner point detection calculates the corner point response value of each pixel based on the gradient intensity matrix, and the line segment detection extracts the line segment endpoints through edge direction histogram statistics and line segment connectivity criteria, and the corner points and line segment endpoints whose response values exceed a dynamic threshold are screened as candidate geometric response points to generate a multi-level geometric response point set;

[0013] Coordinate interpolation and scale attribute calculation are performed on each candidate geometric response point in the multi-level geometric response point set, wherein the coordinate interpolation adopts a quadratic polynomial fitting method to optimize the coordinate accuracy, and the scale attribute is determined based on the ratio of the resolution of the level where the geometric response point is located to the original resolution, so as to obtain a multi-scale basic geometric feature point set.

[0014] Preferably, the steps of obtaining the feature point subset that meets the constraints are:

[0015] Based on the preset geometric constraint parameter configuration table of the marking task, the minimum and maximum angles of the target angle range, the lower limit threshold of the number of collinear points, and the minimum and maximum distances between point pairs are called;

[0016] Traverse each feature point in the multi-scale basic geometric feature point set, calculate the angle formed by the feature point and the line connecting the two adjacent points, count the number of collinear points continuously distributed on the straight line segment where the feature point is located, measure the Euclidean distance between the feature point and the nearest neighbor point, and judge whether the angle formed by the feature point and the line connecting the two adjacent points is within the minimum angle and maximum angle range, whether the number of collinear points is greater than the lower limit threshold of the number of collinear points, and whether the Euclidean distance is within the minimum distance and maximum distance range of the point pair spacing. If any of the judgment results is true, mark the feature point as a valid candidate point, and generate a constraint compliance judgment result set;

[0017] Based on the constraint compliance determination result set, all feature points marked as valid candidate points are screened, and unmarked feature points are removed to form a feature point subset that meets the constraints.

[0018] Preferably, the steps of obtaining the candidate calibration marker structure set are:

[0019] Based on the preset corner point combination rule parameter table, the minimum allowable distance value between corner points, the maximum allowable distance value between corner points, the minimum allowable angle value between corner points, the maximum allowable angle value between corner points, the minimum allowable spacing value between collinear points, the minimum requirement value for the number of collinear points, and the fixed distance pairing allowable error value are called to construct a geometric grouping rule parameter set including the corner point combination distance interval, the corner point combination angle range, the collinear arrangement spacing threshold and number threshold, and the fixed distance pairing error threshold;

[0020] Traverse all feature points in the feature point subset that meet the constraints, and perform the following operations on each feature point: for the corner point combination rule, calculate the straight-line distance value between the feature point and other feature points, determine whether the distance value is within the corner point combination distance interval, and at the same time calculate the angle value between the line connecting the two points and the reference coordinate axis, determine whether the angle value is within the preset angle range, and select feature point pairs that meet both the distance interval and angle range conditions as candidate corner point combinations; for the collinear arrangement rule, search for the spacing values of adjacent feature points on the straight line segment where the feature point is located, count the number of feature points whose continuous spacing does not exceed the collinear arrangement spacing threshold, and select feature point sequences whose number reaches the minimum required value of the number of collinear points as candidate collinear groups; for the fixed distance pairing rule, measure the actual distance value between the feature point and all feature points, and select feature point pairs whose absolute value of the difference between the actual distance and the preset fixed distance does not exceed the allowable error value as candidate fixed distance pairing groups, and generate a candidate grouping set including candidate corner point combinations, candidate collinear groups and candidate fixed distance pairing groups;

[0021] Based on the candidate grouping set, when the same feature point is assigned to multiple candidate groups, the group that meets the most grouping rules is retained first, and the groups that have coordinate overlap or logical conflicts with other groups are eliminated. The remaining independent and non-conflicting candidate corner point combinations, candidate collinear groups, and candidate fixed-distance pairing groups are merged into complete feature point combinations to establish a candidate calibration mark structure set.

[0022] Preferably, the steps for obtaining the weighted feature point contribution value are:

[0023] Traversing each feature point in the candidate calibration marker structure set, extracting the response strength of the feature point in each level of image corner detection, as well as the coordinate data sets at different resolution levels in the multi-scale basic geometric feature point set, to generate a feature point response strength-multi-scale coordinate data set;

[0024] Based on the feature point response strength-multi-scale coordinate data set, calculating a coordinate stability metric value of each feature point across scale levels to obtain a feature point coordinate stability set;

[0025] A weighted feature point contribution value is calculated based on the feature point response strength-multi-scale coordinate data set and the feature point coordinate stability set.

[0026] Preferably, the steps for obtaining the coordinates of the calibration reference points are:

[0027] Traversing each candidate structure in the candidate calibration mark structure set, extracting a weighted feature point contribution value set and a corresponding feature point coordinate set of all feature points in the structure, and generating a candidate structure feature point contribution-coordinate data set;

[0028] Based on the candidate structure feature point contribution-coordinate data set, calibration reference point fitting coordinates are calculated to generate calibration reference point coordinates of all candidate structures.

[0029] Preferably, the steps for obtaining the morphological and geometric features of the microscopic markers are:

[0030] The coordinates of the calibration reference points are transmitted to the marking robot console, the robot is triggered to perform a trial marking operation, and a splash dispersion morphology image is simultaneously collected at the moment when the nozzle end contacts the ground to generate a splash image sequence;

[0031] Based on the splash image sequence, the following processing is performed on each frame of the image: the grayscale threshold is determined by the maximum inter-class variance method to complete binary segmentation, the binary image is subjected to morphological closing operation to eliminate holes, and the splash area edge is extracted by applying the Canny operator to generate a splash contour dataset;

[0032] Traverse the splash contour data set and perform geometric moment calculation on each closed contour point set: calculate the first-order moment M of the closed contour point set in the x-axis direction 10 and the first-order moment M in the y-axis direction 01 , with the contour area zero-order moment M 00 As a benchmark, through x c =M 10 / M 00 with y c =M 01 / M 00 Get the center of mass physical coordinate x c and y c , constructing the morphological and geometric features of microscopic markers.

[0033] Preferably, based on the geometric features of the microscopic mark morphology, the spatial deviation vector between the center of mass position coordinates and the calibration reference point coordinates is calculated, and the steps of adjusting the current position parameters and posture parameter settings of the tool end point of the marking robot according to the spatial deviation vector are as follows:

[0034] Based on the centroid physical coordinates of the microscopic mark morphological geometric features, the difference between the centroid physical coordinates and the preset coordinate components is calculated axis by axis to generate a deviation vector data set including an x-axis deviation and a y-axis deviation;

[0035] According to the deviation vector data set, the deviation is decomposed into the lateral translation compensation step and the longitudinal translation compensation step in the robot base coordinate system to generate a set of posture correction parameters;

[0036] Based on the posture correction parameter set, the current position parameters of the tool end point of the marking robot are adjusted.

[0037] Compared with the prior art, the advantages and positive effects of the present invention are:

[0038] The present invention uses a parallel detection mechanism for multi-resolution hierarchical image sets to synchronously extract corner point and straight line segment features at different scales, and combines cross-level coordinate stability measurement with response intensity quantification to construct a highly robust multi-scale feature point set. Based on preset geometric constraints, the feature points are dynamically screened and grouped to eliminate noise interference and false detection features, generating a candidate calibration structure set with spatial consistency. Weighted contribution value fitting is adopted to comprehensively consider the cross-scale stability of feature points and the detection confidence, thereby improving the geometric accuracy and anti-interference ability of the calibration reference point coordinates. Through the feedback of the micro-geometric features of the trial line splash morphology, the robot end position and posture parameters are corrected in real time to form a closed-loop calibration process. The calibration process is made adaptable to dynamic environments, the dependence on manual intervention is reduced, and the calibration accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] See also Figure 1 The present invention provides a technical solution, a method for automatic calibration of a marking robot based on vision guidance, comprising the following steps:

[0042] Based on the work site image data acquired by the marking robot, a collection of images with different resolution levels is established. Corner point detection and line segment detection are performed in parallel on the images at each level to generate a multi-scale basic geometric feature point set.

[0043] Based on the multi-scale basic geometric feature point set, the preset geometric constraint conditions for the line drawing task are called, and the constraint compliance of each feature point in the multi-scale basic geometric feature point set is judged to obtain a subset of feature points that meet the constraints. Based on the subset of feature points that meet the constraints, grouping and association are performed to establish a candidate calibration mark structure set;

[0044] Based on the candidate calibration marker structure set, the feature points in each structure are traversed, the detection response strength and the coordinate stability measurement value across scale levels in the multi-scale basic geometric feature point set are quantified, the contribution weight parameter of each feature point is determined, and the weighted feature point contribution value is obtained. Based on the weighted feature point contribution value, the coordinates of the feature points associated with each candidate structure are fitted to obtain the calibration reference point coordinates;

[0045] Based on the coordinates of the calibrated reference point as the target position, the marking robot is instructed to perform a trial marking. The splash dispersion morphology image at the moment when the nozzle end of the marking robot contacts the working ground is synchronously collected to obtain the microscopic mark morphological geometric features. Based on the microscopic mark morphological geometric features, the current position parameters of the tool end point of the marking robot are adjusted.

[0046] The steps for obtaining the multi-scale basic geometric feature point set are:

[0047] Perform multi-level resolution reconstruction on the work site image data, use the Gaussian pyramid generation method to adjust the resolution layer by layer and superimpose them into a set to form a multi-resolution image set;

[0048] Based on a multi-resolution image collection, corner detection and line segment detection are performed in parallel on each level of the image. Corner detection calculates the corner response value of each pixel based on the gradient intensity matrix, and line segment detection extracts line segment endpoints through edge direction histogram statistics and line segment connectivity criteria. Corner points and line segment endpoints with response values exceeding a dynamic threshold are selected as candidate geometric response points to generate a multi-level geometric response point set.

[0049] Coordinate interpolation and scale attribute calculation are performed on each candidate geometric response point in the multi-level geometric response point set. The coordinate interpolation uses the quadratic polynomial fitting method to optimize the coordinate accuracy, and the scale attribute is determined based on the ratio of the resolution of the level where the geometric response point is located to the original resolution, thereby obtaining a multi-scale basic geometric feature point set.

[0050] Specifically, the work site image data named "workshop floor area A" with a resolution of 1920×1080 pixels is reconstructed at multiple levels of resolution. The Gaussian pyramid generation process is started. First, the original image (layer 0) is Gaussian blurred using a 5×5 Gaussian kernel with a σ value of 1.0. Then, the blurred image is sampled alternately row and column to obtain the first layer image with a resolution of 960×540 pixels. This blurring and sampling process is repeated to generate the second layer image with a resolution of 480×270 pixels based on the first layer image. Three layers of images, namely the original image (1920×1080), the first layer image (960×540), and the second layer image (480×270), are superimposed and stored to form a multi-resolution image set containing [image layer 0, image layer 1, image layer 2]. Then, based on this multi-resolution image set, the corner detection and line segment detection tasks are synchronously started on the 0th, 1st, and 2nd layer images. For corner detection, taking a pixel point P(100, 150) on the first layer image as an example, the x-direction gradient I of all pixels in its 3×3 neighborhood is calculated. x and the y-direction gradient I y , for example, to obtain the gradient value matrix and then construct the gradient intensity matrix

[0051] For example, we can calculate Set the parameter k of Harris corner response to 0.04 and calculate the corner response value of the pixel R = det(M) - k·trace(M) 2 =(850×600-120 2 )-0.04×(850+600) 2 =(510000-14400)-0.04×(1450) 2 =

[0052] 495600-0.04×2102500=495600-84100=411500. Repeat this calculation for all pixels in this layer. For straight line segment detection, first apply an edge detection algorithm, such as the Canny operator, to extract edge points and their gradient directions in the image. Then analyze the edge image, count the gradient direction histogram of the edge points in the local area, find the straight line direction corresponding to the histogram peak, connect them based on the distance and direction consistency between the edge points, form candidate line segments, and extract the coordinates of the two endpoints of these line segments. For example, a line segment is detected in the first layer image, and its endpoints are L1_start(50,80) and L1_end(250,85). Next, screen all calculated corner point response values and extracted line segment endpoints, and set a dynamic threshold. The threshold is determined according to the statistical characteristics of the image response values of each layer. For example, for the corner point response value of the first layer image, calculate its average value μ R and standard deviation σ R , for example μ R =50000,σ R =100000, set the threshold T R =μ R +1.5×σ R =50000+1.5×100000=200000, the corner response value R=411500 and the threshold T R =200000, 411500>200000, so the pixel point P(100, 150) is retained, and all corner points and line segment endpoints that pass the respective threshold screening constitute candidate geometric response points. The points screened at all levels are collected to generate a multi-level geometric response point set containing [layer 0 response point set, layer 1 response point set, layer 2 response point set]. Subsequently, for each candidate geometric response point in this multi-level geometric response point set, for example, the corner point P(100, 150) screened out in the first layer, whose response value is 411500, a quadratic polynomial curve is performed on the response values within its 3×3 neighborhood. Surface fitting: The sub-pixel coordinates of the surface peak are found by solving partial derivatives. For example, the optimized coordinates are P'(100.21, 150.35), and then its scale attribute is calculated. The point is located in the first layer, its resolution is 960×540, and its original resolution is 1920×1080. Its scale attribute is s=960 / 1920=0.5. The optimized coordinates (100.21, 150.35) and the scale attribute 0.5 are assigned to the point. Coordinate interpolation and scale attribute calculation are performed on all candidate geometric response points to obtain a multi-scale basic geometric feature point set containing precise coordinate and scale information.

[0053] The steps to obtain the feature point subset that meets the constraints are:

[0054] Based on the preset geometric constraint parameter configuration table of the marking task, the minimum and maximum angles of the target angle range, the lower limit threshold of the number of collinear points, and the minimum and maximum distances between point pairs are called;

[0055] Traverse each feature point in the multi-scale basic geometric feature point set, calculate the angle formed by the feature point and the line connecting the two adjacent points, count the number of collinear points continuously distributed on the straight line segment where the feature point is located, measure the Euclidean distance between the feature point and the nearest neighbor point, and judge whether the angle formed by the feature point and the line connecting the two adjacent points is within the minimum angle and maximum angle range, whether the number of collinear points is greater than the lower limit threshold of the number of collinear points, and whether the Euclidean distance is within the minimum distance and maximum distance range of the point pair spacing. If any of the judgment results are true, mark the feature point as a valid candidate point and generate a constraint compliance judgment result set;

[0056] Based on the constraint compliance judgment result set, all feature points marked as valid candidate points are screened, and unmarked feature points are removed to form a subset of feature points that meet the constraints.

[0057] Specifically, based on the preset line marking task geometric constraint parameter configuration table, the minimum angle value of the target angle range is called and set to 178 degrees, and the maximum angle value is set to 182 degrees. This range is used to identify the point arrangement of the approximate straight line. The lower limit threshold of the number of collinear points is called and set to 5 points. It is required that a valid straight line segment contains at least 5 feature points. The minimum distance of the point pair spacing is called and set to 8 pixels. The maximum distance is set to 120 pixels to control the spatial distribution between the valid feature point and its nearest neighbor point. Then, the system traverses each feature point recorded in the multi-scale basic geometric feature point set generated in the previous step, and the currently traversed feature point is set as P i , whose coordinates are (X i ,Y i ), scale is S i , for the angle constraint, find two adjacent feature points P in its neighborhood (for example, at the same scale level or on the same detected straight line segment) i-1 and P i+1 , calculate the vector and Calculate the angle between two vectors by using the dot product For example, if P i-1 (90.1,148.2), P i (100.2,150.3),

[0058] P i+1 (110.5,152.4), then Calculate the angle θ i≈179.5 degrees, determine whether the angle is in the range of [178, 182] degrees. If it is (178≤179.5≤182), then the angle constraint judgment result is true. For the constraint on the number of collinear points, identify the feature point P i The total number of continuously distributed feature points N on the straight line segment to which it belongs is counted i For example, it is found that P i There are 7 feature points on the straight line segment L2. Check whether the number is greater than the lower limit threshold of 5 for the number of collinear points. If it is (7>5), the result of the collinear point number constraint is true. For the point pair spacing constraint, find the distance feature point P. i The nearest neighboring feature point P neighbor (regardless of whether they are on the same line), calculate the Euclidean distance between two points, for example, find the nearest neighbor point P neighbor (105.0, 158.0), calculate the distance, and determine whether the distance is within the interval of [8, 120] pixels. Then the point pair spacing constraint judgment result is true. i The three judgment results of the angle, number of collinear points and point pair spacing are all true, so the feature point P is marked i To be a valid candidate point, the above calculation and judgment process is repeated for all feature points in the multi-scale basic geometric feature point set, and a constraint compliance judgment result set is generated to record whether each feature point is a valid candidate point, forming a feature point subset that meets the preset geometric constraint conditions.

[0059] The steps for obtaining the candidate calibration marker structure set are:

[0060] Based on the preset corner point combination rule parameter table, the minimum allowable distance value between corner points, the maximum allowable distance value between corner points, the minimum allowable angle value between corner points, the maximum allowable angle value between corner points, the minimum allowable spacing value between collinear points, the minimum requirement value for the number of collinear points, and the fixed distance pairing allowable error value are called to construct a geometric grouping rule parameter set including the corner point combination distance interval, the corner point combination angle range, the collinear arrangement spacing threshold and number threshold, and the fixed distance pairing error threshold;

[0061] Traverse all feature points in the feature point subset that meet the constraints, and perform the following operations on each feature point: for the corner point combination rule, calculate the straight-line distance value between the feature point and other feature points, determine whether the distance value is within the corner point combination distance interval, and at the same time calculate the angle value between the line connecting the two points and the reference coordinate axis, determine whether the angle value is within the preset angle range, and select feature point pairs that meet both the distance interval and angle range conditions as candidate corner point combinations; for the collinear arrangement rule, search for the spacing values of adjacent feature points on the straight line segment where the feature point is located, count the number of feature points whose continuous spacing does not exceed the collinear arrangement spacing threshold, and select feature point sequences whose number reaches the minimum required value of the number of collinear points as candidate collinear groups; for the fixed distance pairing rule, measure the actual distance value between the feature point and all feature points, and select feature point pairs whose absolute value of the difference between the actual distance and the preset fixed distance does not exceed the allowable error value as candidate fixed distance pairing groups, and generate a candidate grouping set including candidate corner point combinations, candidate collinear groups and candidate fixed distance pairing groups;

[0062] Based on the candidate grouping set, when the same feature point is assigned to multiple candidate groups, the group that meets the most grouping rules is retained first, and the groups that have coordinate overlap or logical conflicts with other groups are eliminated. The remaining independent and non-conflicting candidate corner point combinations, candidate collinear groups, and candidate fixed-distance pairing groups are merged into complete feature point combinations to establish a candidate calibration mark structure set.

[0063] Specifically, based on the preset corner point combination rule parameter table, the system calls a series of parameter values for geometric grouping, including: the minimum allowable distance value between corner points, set to 40 pixels, the maximum allowable distance value between corner points, set to 200 pixels, the minimum allowable angle value between corner points, set to 85 degrees, the maximum allowable angle value between corner points, set to 95 degrees, the maximum allowable spacing value between collinear points, set to 15 pixels, the minimum requirement value for the number of collinear points, set to 4 points, and the fixed distance pairing allowable error value, set to 3 pixels. These parameters together construct a corner point combination distance interval of [40, 200] pixels, a corner point combination angle range of [85, 95] degrees, and a collinear arrangement spacing threshold. A set of geometric grouping rule parameters with a threshold of 15 pixels and a quantity threshold of 4 points and a fixed distance of 150 pixels and a pairing error threshold of 3 pixels is used. Subsequently, all feature points in the feature point subset that meets the constraints generated in the previous step are traversed, and a geometric grouping rule matching operation is performed on each feature point P. According to the corner point combination rule, the straight-line distance d(P, Q) between P and all other feature points Q is calculated, and it is determined whether d(P, Q) is in the [40, 200] pixel interval. If it is satisfied, the third feature point R is further searched, and the angle value of the angle ∠QPR (or ∠PQR, etc., depending on the rule definition, here we take ∠QPR as an example) formed by P, Q, R is calculated, and the angle value of the angle value is determined whether it is in [85, 95] degrees. For example, for P(100.2,150.3), Q(102.5,248.1), R(195.8,155.0), we get d(P,Q)≈98 pixels (in [40,200]), d(P,R)≈96 pixels (in [40,200]), and the angle ∠QPR≈91.5 degrees (in [85,95]). Then, P,Q,R form a candidate corner point combination {P,Q,R}. According to the collinear arrangement rule, we search for adjacent feature points on the straight line segment where the feature point P is located, and calculate the spacing between them. For example, the ordered points on the line segment L2 where P is located are P1,P2,P,P4,P5, and the spacing d(P1,P2) is calculated as 12, d(P2, P) = 10, d(P, P4) = 13, d(P4, P5) = 11, all spacings do not exceed the collinear arrangement spacing threshold of 15 pixels, the number of feature points in the continuous sequence is counted as 5, and it is judged whether the number reaches the minimum requirement of the number of collinear points 4. If it is (5 ≥ 4), the feature point sequence {P1, P2, P, P4, P5} is selected as a candidate collinear group. According to the fixed distance pairing rule, the actual distance d(P, Q) between the feature point P and all other feature points Q is calculated, and it is judged whether the absolute value of the difference between the distance and the preset fixed distance of 150 pixels does not exceed the allowable error value of 3 pixels, that is, |d(P, Q)-150|≤3. For example, the difference between P and Q is calculated. ′ The distance of (248.5,160.9) Pixels, calculate the absolute value of the difference |148.68-150|=1.32, judge 1.32≤3, if yes, then filter out the feature point pair {P, Q'} as a candidate fixed distance pairing group, after completing the above rule matching for all feature points, generate a candidate grouping set containing all found candidate corner point combinations, candidate collinear groups and candidate fixed distance pairing groups, finally, handle potential allocation conflicts, check whether a feature point is assigned to multiple candidate groups, for example, point P belongs to both the candidate corner point combination {P, Q, R} and the candidate collinear group {P1, P2, P, P4, P5}, retain the group with the most members, and form a complete feature point combination set, which is the candidate calibration mark structure set.

[0064] The steps to obtain the weighted feature point contribution value are:

[0065] Traverse each feature point in the candidate calibration marker structure set, extract the response strength of the feature point in each level of image corner detection, and the coordinate data set at different resolution levels in the multi-scale basic geometric feature point set, and generate a feature point response strength-multi-scale coordinate data set;

[0066] Based on the feature point response intensity-multi-scale coordinate dataset, the coordinate stability metric of each feature point across scale levels is calculated to obtain the feature point coordinate stability set. The formula is:

[0067]

[0068] Among them, S j is the coordinate stability measure of feature point j, k is the number of cross-scale levels of feature point j in the multi-scale basic geometric feature point set, x jm is the x-axis coordinate value of feature point j in the m-th level image, y jm is the y-axis coordinate value of feature point j in the m-th level image, is the mean x-axis coordinate of feature point j, is the mean y-axis coordinate of feature point j;

[0069] Based on the feature point response intensity-multi-scale coordinate data set and the feature point coordinate stability set, the weighted feature point contribution value is calculated using the formula:

[0070]

[0071] Among them, C p is the weighted feature point contribution value of feature point p, R p is the response intensity of feature point p, S p is the coordinate stability measure of the feature point p, max(R q) is the maximum response intensity among all feature points, max(S q ) is the maximum coordinate stability measure among all feature points.

[0072] Specifically, traverse each feature point included in the candidate calibration marker structure set established in the previous step, and perform the following operations for each feature point p to calculate its weighted contribution value. First, extract the response intensity value corresponding to the feature point p at each level of the image where it is detected when the corner point is initially detected from the multi-scale basic geometric feature point set. For example, the response value of the feature point p (i.e., point 3 before) at the 0th level is R p,0 =650, the first layer response value is R p,1 =680, the second layer response value is R p,2 =590, usually the response value at the most stable or highest resolution level is taken, or the average value / maximum value is taken as its representative response intensity R p , here we take the response value R of the highest layer, that is, layer 0 p =R p,0 =650, and at the same time, extract the coordinate data set of the feature point p recorded in the multi-scale basic geometric feature point set across different resolution levels, that is, its (interpolation optimized) coordinates (x pm ,y pm ), where m represents the level index, such as in the previous example (the coordinates have been normalized to the original resolution): the coordinate set of p=3 is {level 1 (m=1): (100.35, 200.72), level 2 (m=2): (100.24, 200.60), level 3 (m=3): (100.00, 200.40)} (here the level index starts from 1, with a total of k=3 levels). These extracted response intensities and multi-scale coordinate data are integrated to generate a response intensity-multi-scale coordinate dataset of the feature point p. Then, based on the multi-scale coordinate data in the dataset, the cross-scale level coordinate stability measure S of the feature point p is calculated. p , and its calculation formula is:

[0073]

[0074] Formula explanation: S j Represents the coordinate stability measure of feature point j. The calculated result is between 0 and 1. The closer the value is to 1, the higher the stability. k is the number of scale levels where feature point j appears and its coordinates are recorded. Indicates the sum of these k levels, x jm and y jm are the (normalized) x-axis and y-axis coordinate values of feature point j in the m-th scale level image, and They are the mean x-axis coordinate and the mean y-axis coordinate of feature point j at these k levels, and are calculated as follows: as well as The core part of the formula The coordinates of the points (x jm ,y jm ) relative to its mean position The root mean square deviation (RMSD) reflects the discrete degree or instability of the coordinate points at different scales. Finally, the instability measure (RMSD) is converted into the stability measure S by 1 / (1+RMSD) j , the smaller the RMSD (the more stable), S j The closer it is to 1, the larger the RMSD (the more unstable), S j The closer to 0, the more 1 is added to avoid the denominator being zero and to ensure the range.

[0075] Parameter acquisition and calculation example: For feature points p = 3, k = 3, the coordinate set (normalized) is {(100.35, 200.72), (100.24, 200.60), (100.00, 200.40)},

[0076] Compute the mean coordinates:

[0077]

[0078] Calculate the square of the RMSD:

[0079]

[0080] Calculate RMSD:

[0081] Computational stability

[0082] The calculation results S of all feature points j Stored in the feature point coordinate stability set, then based on the response strength R of the feature point p p =650 and coordinate stability S p =0.8355, and the maximum response intensity max(R q ) and maximum coordinate stability max(S q )(For example, by traversing all feature points to get max(R q )=820 and max(S q )=0.9600), calculate the weighted feature point contribution value C p , and its calculation formula is:

[0083]

[0084] Formula explanation: C p Represents the weighted feature point contribution value of feature point p, which is a normalized comprehensive evaluation score. p is the response strength of feature point p, S p is its coordinate stability measure, R p ·S p represents the raw contribution combining strength and stability, max(R q ) is the maximum response intensity value observed among all candidate feature points, q traverses all feature points, max(S q ) is the maximum coordinate stability metric calculated among all candidate feature points, and the product of these two maximum values is used as the denominator for normalization. p The value is scaled to the approximate [0, 1] interval to make the contribution value comparable. The core logic of this formula is to multiply R p ·S p Feature points with both high response strength (easy to detect) and high cross-scale coordinate stability (reliable localization) are rewarded and normalized by the global maximum.

[0085] For feature points Repeat this calculation for each feature point in the candidate calibration marker structure set to obtain the weighted contribution value of all feature points.

[0086] The benefit of the formula is that S j By quantifying the coordinate consistency of feature points in images with different resolutions, the recognition capability of feature points that are insensitive to scale changes and have high positioning accuracy is improved. p The detection significance of feature points (R p ) and positioning stability (S p ), and provides a relatively objective quantitative indicator to measure the importance of each feature point to the subsequent calibration task through normalization, which is more accurate than using R alone. p Or geometric attributes, it can better filter out high-quality feature points.

[0087] The result C3≈0.6898 indicates that feature point 3 has a medium to high contribution value, and its response strength and stability are relatively reliable. This contribution value will be used as a weight in the next step to calculate the coordinates of the calibration reference point.

[0088] The steps for obtaining the coordinates of the calibration reference point are:

[0089] Traverse each candidate structure in the candidate calibration mark structure set, extract the weighted feature point contribution value set and the corresponding feature point coordinate set of all feature points in the structure, and generate a candidate structure feature point contribution-coordinate data set;

[0090] Calculate the fitting coordinates of the calibration reference points based on the candidate structural feature point contribution-coordinate dataset Generate the calibration reference point coordinates of all candidate structures, and the calculation formula is:

[0091] and

[0092] Among them, C i is the weighted feature point contribution value of the i-th feature point in the candidate structure, X i is the x-axis coordinate value of the i-th feature point, Y i is the y-axis coordinate value of the i-th feature point, and n is the total number of feature points in the candidate structure.

[0093] Specifically, traverse each candidate structure g in the candidate calibration mark structure set generated in the previous step, the system first extracts all the feature points contained in the structure g, and for each feature point i, obtains its corresponding weighted feature point contribution value C from the calculation result of the previous step i And its exact coordinates (X i ,Y i ), collect these data and generate a feature point contribution-coordinate data set of the candidate structure g. For example, the data set of structure g is: {point 1: C1 = 0.650, (X1, Y1) = (90.1, 100.5); point 3: C3 = 0.6898, (X3, Y3) = (100.2, 150.3); point 5: C5 = 0.580, (X5, Y5) = (110.8, 99.8)}, which contains a total of n = 3 feature points. Then, based on this data set, the weighted average formula is applied to calculate the calibration reference point fitting coordinates of the candidate structure g. The calculation formulas are:

[0094]

[0095] Formula explanation: and They represent the x-axis and y-axis coordinates of the fitting calibration reference point of the candidate structure g, n is the total number of feature points in the structure, Indicates the sum of all n feature points in the structure, C i is the weighted feature point contribution value of the i-th feature point (as weight), X i and Y i is the x-axis and y-axis coordinate value of the i-th feature point, the molecular part The weighted sum of the x coordinates of all feature points is calculated, and the molecular part The weighted sum of the y coordinates of all feature points is calculated, and the denominator is The sum of the weights of all feature points is calculated. The entire formula calculates the weighted average of the feature point coordinates within the structure. The feature points with higher contribution values have a greater impact on the final reference point position. This calculation method aims to obtain a more stable and accurate reference point coordinate that represents the center or key position of the geometric structure by giving higher weights to high-quality feature points.

[0096] Parameter acquisition and calculation example: Use the data set of structure g for calculation,

[0097] Calculate the sum of weights:

[0098] Calculate the weighted sum of the x-coordinates:

[0099] Calculate the weighted sum of the y coordinates:

[0100] Calculate the fitted coordinates:

[0101]

[0102] Therefore, the calibration reference point fitting coordinates of the candidate structure g are (99.98, 118.18). This calculation is performed for each structure in the set of all candidate calibration labeled structures to generate a list of calibration reference point coordinates for all candidate structures.

[0103] The benefit of the formula is that by using the weighted feature point contribution value C i The coordinate averaging is performed as weight instead of simple arithmetic mean (geometric center), so that the calculated calibration reference point position can better resist the positioning error caused by individual low-quality (low response strength or low stability) feature points in the structure, thereby improving the robustness and accuracy of the reference point coordinates.

[0104] The result It represents the best representative point position of the geometric structure determined by comprehensively considering the quality of each feature point in the structure. This coordinate will be used as the target point for the robot to test the line.

[0105] The steps for obtaining the geometric features of microscopic markers are as follows:

[0106] The coordinates of the calibration reference points are transmitted to the marking robot console, triggering the robot to perform a trial marking operation. The splash dispersion morphology image at the moment when the nozzle end contacts the ground is simultaneously collected to generate a splash image sequence.

[0107] Based on the splash image sequence, the following processing is performed on each frame: the maximum inter-class variance method is used to determine the grayscale threshold to complete binary segmentation, the morphological closing operation is performed on the binary image to eliminate holes, and the Canny operator is used to extract the edge of the splash area to generate a splash contour dataset;

[0108] Traverse the splash contour data set and perform geometric moment calculation on each closed contour point set: calculate the first-order moment M of the closed contour point set in the x-axis direction 10 and the first-order moment M in the y-axis direction 01 , with the contour area zero-order moment M 00 As a benchmark, through x c =M 10 / M 00 with y c =M 01 / M 00 Get the center of mass physical coordinate x c and y c , constructing the morphological and geometric features of microscopic markers.

[0109] Specifically, the coordinates of one or more calibration reference points calculated in the previous step, such as the coordinates of structure g The control system transmits the target coordinates to the console system of the marking robot. After receiving the target coordinates, the control system drives the robot to move so that the end of its nozzle moves to the corresponding physical position, and then triggers a short spraying action to perform a test marking operation. At the moment when the end of the robot nozzle touches the ground (or very close to the ground) and sprays paint, the camera associated with the nozzle (for example, a camera installed on the end effector of the robot with a field of view covering the area below the nozzle) is synchronously started to capture images to obtain images of the splashing dispersion morphology. A single frame image or a short image sequence may be captured to form a splashing image sequence. Then, an image processing process is performed on each valid image frame in this splashing image sequence (an image that records the splashing morphology, such as the frame after the splashing is stabilized). First, the maximum inter-class variance method is applied to the grayscale image of the frame to automatically calculate an optimal grayscale threshold T. otsu , used to distinguish the splash area (foreground) from the background. For example, for a 200×200 pixel splash image, T is calculated. otsu=128, and then the image is binarized based on this threshold. Pixel grayscale values greater than 128 are set to 255 (white, representing the splash area), and those less than or equal to 128 are set to 0 (black, representing the background) to obtain a binary image. Subsequently, a morphological closing operation is performed on the binary image. A circular or square structural element, such as a 3×3 pixel structure element, is selected. The white area is first dilated and then eroded. This process can effectively fill the small holes or fractures that may exist inside the splash area, while having little effect on the edge, keeping the shape of the splash body intact. After processing, the Canny edge detection operator is applied to extract the edge contour of the splash area in the processed binary image, perform edge connection, and generate a set of pixel coordinates that constitute the closed contour of the splash area. For example, a pixel containing N contour The contour point set of all points is collected, and the contour data obtained from all processed frames are collected to form a splash contour data set. Finally, each closed contour point set in this splash contour data set is traversed, and geometric moment calculation is performed on it to calculate the zero-order moment M of the area surrounded by the contour. 00 (i.e., area, number of pixels), the first-order moment M in the x-axis direction 10 and the first-order moment M in the y-axis direction 01 , for example, calculate the area M of the splash region 00 =950 pixels 2 , first-order moment M in the x-direction 10 =94850 pixels 3 , the first-order moment M in the y direction 01 =112200 pixels 3 , with the zero-order moment M 00 As a benchmark, calculate the center of mass physical coordinates = M 10 / M 00 =94850 / 950=99.84 pixels, y c =M 01 / M 00 = 112200 / 950 = 118.11 pixels, and the calculated centroid coordinates (x c ,y c )=(99.84,118.11) is used as the core geometric feature to describe the microscopic mark morphology of this trial marking, and the microscopic mark morphological geometric feature data including the centroid coordinates is constructed.

[0110] Based on the geometric features of the microscopic markers, the spatial deviation vector between the center of mass position coordinates and the calibration reference point coordinates is calculated. The steps for adjusting the current position parameters and posture parameters of the tool end point of the marking robot according to the spatial deviation vector are as follows:

[0111] Based on the centroid physical coordinates of the microscopic marker morphological geometric features, the difference between the centroid physical coordinates and the preset coordinate components is calculated axis by axis to generate a deviation vector data set including the x-axis deviation and the y-axis deviation;

[0112] According to the deviation vector data set, the deviation is decomposed into the lateral translation compensation step and the longitudinal translation compensation step in the robot base coordinate system to generate a set of posture correction parameters;

[0113] Based on the posture correction parameter set, the current position parameters of the tool end point of the marking robot are adjusted.

[0114] Specifically, based on the geometric features of the microscopic mark obtained in the previous step, the physical coordinates of the center of mass of the actual splash mark (x c ,y c )=(99.84,118.11), the system starts to calculate the actual marker position and the expected calibration reference point coordinates The spatial deviation vector between them is first calculated by calculating the difference of the coordinate components axis by axis to calculate the x-axis deviation. Unit (e.g. mm), calculate the y-axis deviation Unit (e.g. mm), combine these two deviations into a deviation vector And store this vector in the deviation vector data set, and calculate the deviation vector Decomposed into the compensation movement in the robot base coordinate system.

[0115] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for automatic calibration of a marking robot based on vision guidance, characterized in that: The following steps are involved: Based on the work site image data acquired by the marking robot, a collection of images with different resolution levels is established. Corner point detection and line segment detection are performed in parallel on the images at each level to generate a multi-scale basic geometric feature point set. Based on the multi-scale basic geometric feature point set, calling the preset geometric constraint conditions for the line drawing task, performing constraint compliance judgment on each feature point in the multi-scale basic geometric feature point set, obtaining a subset of feature points that meet the constraints, and grouping and associating the feature points based on the subset of feature points that meet the constraints to establish a candidate calibration mark structure set; Based on the candidate calibration marker structure set, traversing the feature points in each structure, quantifying the detection response strength and the coordinate stability measurement value across scale levels in the multi-scale basic geometric feature point set, determining the contribution weight parameter of each feature point to obtain a weighted feature point contribution value, and fitting the feature point coordinates associated with each candidate structure based on the weighted feature point contribution value to obtain the calibration reference point coordinates; Based on the coordinates of the calibrated reference point as the target position, the marking robot is instructed to perform a trial marking, and the splash dispersion morphology image at the moment when the nozzle end of the marking robot contacts the working ground is synchronously collected to obtain the microscopic mark morphological geometric features. Based on the microscopic mark morphological geometric features, the current position parameters of the tool end point of the marking robot are adjusted.

2. The automatic calibration method of a marking robot based on vision guidance according to claim 1, characterized in that: The steps for obtaining the multi-scale basic geometric feature point set are: Perform multi-level resolution reconstruction on the work site image data, use the Gaussian pyramid generation method to adjust the resolution layer by layer and superimpose them into a set to form a multi-resolution image set; Based on the multi-resolution image set, corner point detection and line segment detection are performed in parallel on each level of the image, wherein the corner point detection calculates the corner point response value of each pixel based on the gradient intensity matrix, and the line segment detection extracts the line segment endpoints through edge direction histogram statistics and line segment connectivity criteria, and the corner points and line segment endpoints whose response values exceed a dynamic threshold are screened as candidate geometric response points to generate a multi-level geometric response point set; Coordinate interpolation and scale attribute calculation are performed on each candidate geometric response point in the multi-level geometric response point set, wherein the coordinate interpolation adopts a quadratic polynomial fitting method to optimize the coordinate accuracy, and the scale attribute is determined based on the ratio of the resolution of the level where the geometric response point is located to the original resolution, so as to obtain a multi-scale basic geometric feature point set.

3. The automatic calibration method of a marking robot based on vision guidance according to claim 1, characterized in that: The steps for obtaining the feature point subset that meets the constraints are: Based on the preset geometric constraint parameter configuration table of the marking task, the minimum and maximum angles of the target angle range, the lower limit threshold of the number of collinear points, and the minimum and maximum distances between point pairs are called; Traverse each feature point in the multi-scale basic geometric feature point set, calculate the angle formed by the feature point and the line connecting the two adjacent points, count the number of collinear points continuously distributed on the straight line segment where the feature point is located, measure the Euclidean distance between the feature point and the nearest neighbor point, and judge whether the angle formed by the feature point and the line connecting the two adjacent points is within the minimum angle and maximum angle range, whether the number of collinear points is greater than the lower limit threshold of the number of collinear points, and whether the Euclidean distance is within the minimum distance and maximum distance range of the point pair spacing. If any of the judgment results is true, mark the feature point as a valid candidate point, and generate a constraint compliance judgment result set; Based on the constraint compliance determination result set, all feature points marked as valid candidate points are screened, and unmarked feature points are removed to form a feature point subset that meets the constraints.

4. The automatic calibration method of a marking robot based on vision guidance according to claim 1, characterized in that: The steps for obtaining the candidate calibration marker structure set are: Based on the preset corner point combination rule parameter table, the minimum allowable distance value between corner points, the maximum allowable distance value between corner points, the minimum allowable angle value between corner points, the maximum allowable angle value between corner points, the minimum allowable spacing value between collinear points, the minimum requirement value for the number of collinear points, and the fixed distance pairing allowable error value are called to construct a geometric grouping rule parameter set including the corner point combination distance interval, the corner point combination angle range, the collinear arrangement spacing threshold and number threshold, and the fixed distance pairing error threshold; Traverse all feature points in the feature point subset that meet the constraints, and perform the following operations on each feature point: for the corner point combination rule, calculate the straight-line distance value between the feature point and other feature points, determine whether the distance value is within the corner point combination distance interval, and at the same time calculate the angle value between the line connecting the two points and the reference coordinate axis, determine whether the angle value is within the preset angle range, and select feature point pairs that meet both the distance interval and angle range conditions as candidate corner point combinations; for the collinear arrangement rule, search for the spacing values of adjacent feature points on the straight line segment where the feature point is located, count the number of feature points whose continuous spacing does not exceed the collinear arrangement spacing threshold, and select feature point sequences whose number reaches the minimum required value of the number of collinear points as candidate collinear groups; for the fixed distance pairing rule, measure the actual distance value between the feature point and all feature points, and select feature point pairs whose absolute value of the difference between the actual distance and the preset fixed distance does not exceed the allowable error value as candidate fixed distance pairing groups, and generate a candidate grouping set including candidate corner point combinations, candidate collinear groups and candidate fixed distance pairing groups; Based on the candidate grouping set, when the same feature point is assigned to multiple candidate groups, the group that meets the most grouping rules is retained first, and the groups that have coordinate overlap or logical conflicts with other groups are eliminated. The remaining independent and non-conflicting candidate corner point combinations, candidate collinear groups, and candidate fixed-distance pairing groups are merged into complete feature point combinations to establish a candidate calibration mark structure set.

5. The automatic calibration method of a marking robot based on vision guidance according to claim 1, characterized in that: The steps for obtaining the weighted feature point contribution value are as follows: Traversing each feature point in the candidate calibration marker structure set, extracting the response strength of the feature point in each level of image corner detection, as well as the coordinate data sets at different resolution levels in the multi-scale basic geometric feature point set, to generate a feature point response strength-multi-scale coordinate data set; Based on the feature point response strength-multi-scale coordinate data set, calculating a coordinate stability metric value of each feature point across scale levels to obtain a feature point coordinate stability set; A weighted feature point contribution value is calculated based on the feature point response strength-multi-scale coordinate data set and the feature point coordinate stability set.

6. The automatic calibration method of a marking robot based on vision guidance according to claim 1, characterized in that: The steps for obtaining the coordinates of the calibration reference point are: Traversing each candidate structure in the candidate calibration mark structure set, extracting a weighted feature point contribution value set and a corresponding feature point coordinate set of all feature points in the structure, and generating a candidate structure feature point contribution-coordinate data set; Based on the candidate structure feature point contribution-coordinate data set, calibration reference point fitting coordinates are calculated to generate calibration reference point coordinates of all candidate structures.

7. The automatic calibration method of a marking robot based on vision guidance according to claim 1, characterized in that: The steps for obtaining the geometric features of the microscopic marker morphology are as follows: The coordinates of the calibration reference points are transmitted to the marking robot console, the robot is triggered to perform a trial marking operation, and a splash dispersion morphology image is simultaneously collected at the moment when the nozzle end contacts the ground to generate a splash image sequence; Based on the splash image sequence, the following processing is performed on each frame of the image: the grayscale threshold is determined by the maximum inter-class variance method to complete binary segmentation, the binary image is subjected to morphological closing operation to eliminate holes, and the splash area edge is extracted by applying the Canny operator to generate a splash contour dataset; Traverse the splash contour data set and perform geometric moment calculation on each closed contour point set: calculate the first-order moment M of the closed contour point set in the x-axis direction 10 and the first-order moment M in the y-axis direction 01 , with the contour area zero-order moment M 00 As a benchmark, through x c =M 10 / M 00 with y c =M 01 / M 00 Get the center of mass physical coordinate x c and y c , constructing the morphological and geometric features of microscopic markers.

8. The automatic calibration method of a marking robot based on vision guidance according to claim 1, characterized in that: Based on the geometric features of the microscopic mark, the spatial deviation vector between the center of mass position coordinates and the calibration reference point coordinates is calculated, and the steps of adjusting the current position parameters and posture parameter settings of the tool end point of the marking robot according to the spatial deviation vector are as follows: Based on the centroid physical coordinates of the microscopic mark morphological geometric features, the difference between the centroid physical coordinates and the preset coordinate components is calculated axis by axis to generate a deviation vector data set including an x-axis deviation and a y-axis deviation; According to the deviation vector data set, the deviation is decomposed into the lateral translation compensation step and the longitudinal translation compensation step in the robot base coordinate system to generate a set of posture correction parameters; Based on the posture correction parameter set, the current position parameters of the tool end point of the marking robot are adjusted.

Images