A method for detecting the center of a retro-reflective marker point area

Abstract

The application provides a retroreflective marker point area center detection method, comprising the following steps: S1, a single-lens reflex camera is used to shoot a retroreflective marker point, and original picture data is obtained; S2, a picture with more than 10 gray scales is obtained according to the original picture data; S3, a gradient graph and a binary graph of the gray scale picture are calculated; S4, small and large pixel area light spot regions in the binary graph are removed, and the remaining light spot region point set is searched in a maximum gradient region range, and an effective light spot region is obtained. The retroreflective marker point area center detection method can screen the retroreflective marker point through a gradient threshold value, a gray scale threshold value and a residual threshold value, and can determine the marker point region center after the retroreflective marker point is determined, so that the accuracy of the marker point positioning can be greatly improved, the influence of a complex light environment can be excluded, and an operator can efficiently and quickly measure and calculate under the complex light condition.

Description

Technical Field

[0001] This invention relates to the field of visual measurement technology, and in particular to a method for detecting the center of a reflective marker area. Background Technology

[0002] When measuring the parameters of large objects, the method of attaching visual markers to the object's surface is usually adopted. This method is convenient to operate and requires less calculation. At the same time, since the image of a circular marker is an ellipse or a perfect circle in images from different perspectives, it is easy to fit. Therefore, most visual markers are circular. For example, in factories, the basic dimensions of objects are calculated by taking pictures and detecting markers; the flatness of circular pillars on marine platforms is measured; and some key peripheral dimensions of vehicle frames are measured in indoor environments.

[0003] Many existing shooting devices used in photogrammetry are developed based on industrial cameras with auxiliary red or blue light sources. Direct shooting requires repeated adjustments to the light source brightness or exposure parameters under complex lighting conditions to obtain the best shooting effect. There are also methods, such as the "Metering Method for Marker Points in Industrial Photogrammetry" with the authorized publication number CN104729692B, which adjusts the light source brightness by repeatedly acquiring multiple images to obtain the best image. However, this also requires taking multiple images to output a single image with good quality, resulting in low actual processing and measurement efficiency. For projects or equipment with a large amount of shooting and measurement, this method is time-consuming and labor-intensive. Summary of the Invention

[0004] In view of this, the present invention proposes a method for detecting the center of a reflective marker area. The method uses the original image data, processes it to remove unreasonable light spots in multiple steps, and then locates the center of the determined reflective marker area to improve the detection accuracy of the marker and improve the shooting and measurement efficiency.

[0005] The technical solution of this invention is implemented as follows: This invention provides a method for detecting the center of a reflective marker area, comprising the following steps:

[0006] S1. Use a DSLR camera to photograph the reflective markers to obtain the original image data;

[0007] S2. Based on the original image data, obtain an image with a grayscale level of 10 bits or more;

[0008] S3. Calculate the gradient map and binarization map of the grayscale image;

[0009] S4. Remove the spot areas with excessively small or large pixel areas in the binarized image, and perform a maximum gradient region search on the remaining spot area point set to obtain the effective spot area.

[0010] S5. Calculate the centroid of the effective light spot area, obtain the minimum bounding rectangle of the effective light spot area, find the maximum gradient point on the line connecting the mass point and the points on the side length of the minimum bounding rectangle, collect the points between the maximum gradient point and the mass point, and obtain the light spot area point set;

[0011] S6. Extract the outermost contour points from the light spot area point set to obtain the boundary contour point set, and剔除 the effective light spot area corresponding to the non-closed boundary contour point set;

[0012] S7. Calculate the average gradient of the boundary contour point set and screen the remaining effective light spot areas;

[0013] S8. Extract the sub-pixel points of the boundary points of the boundary contour point set of the remaining effective light spot areas, perform ellipse fitting to obtain an ideal ellipse, then calculate the average distance residual between the corresponding boundary contour point set and the ideal ellipse, screen out the effective reflective marker points, and record the center of the ideal ellipse as the area center of the reflective marker points.

[0014] Based on the above technical solutions, preferably, the DSLR camera is a full-frame DSLR camera with more than 12 million pixels and is equipped with an auxiliary flash with high-speed synchronization function.

[0015] Based on the above technical solutions, preferably, in step S2, it further includes parsing the raw image data of the DSLR camera through the Dcraw open-source library, performing demosaicing processing, converting the Bayerpattern image into RGB data, only extracting the data of the green channel as grayscale image data, and performing basic sharpening on the image.

[0016] Based on the above technical solutions, preferably, the gradient map in step S3 is obtained by calculating with a Gaussian gradient operator, and the binary map is obtained based on a gradient threshold and a grayscale threshold.

[0017] Further preferably, let i represent the i-th row of the image, j represent the j-th column of the image, Value(i, j) represent the grayscale value of the pixel at the i-th row and j-th column of the image; Grad(i, j) represent the gradient value of the pixel at the i-th row and j-th column of the image; minValue is the minimum grayscale threshold, maxValue is the maximum grayscale threshold, and minGrad is the minimum gradient threshold; the pixels where Grad(i, j) < minGrad and Value(i, j) > minValue or Value(i, j) > maxValue are tentatively defined as the pixels of the marker point elliptical light spot, and the remaining pixels are tentatively defined as background pixels.

[0018] Based on the above technical solutions, preferably, in step S4, removing spot areas with excessively small or large pixel areas in the binarized image includes removing spots with pixel areas larger than the largest pixel area and spots with pixel areas smaller than the smallest pixel area.

[0019] Based on the above technical solution, preferably, in step S4, the maximum gradient region range search is performed on the remaining spot area point set, including marking all pixels in all spot area point sets as interior points, each interior point queries other pixels in eight directions, if the gradient value corresponding to the queried pixel other than the interior point is greater than the gradient value of the interior point, then the point is marked as an interior point, and the recursive search is performed repeatedly until no new pixel can be obtained as an interior point.

[0020] More preferably, in step S5, the centroid of the effective spot region is calculated, where the centroid is an interior point of the effective spot region point set; otherwise, it is marked as a non-reflective marker spot region.

[0021] More preferably, in step S7, the remaining effective light spot areas are screened, and the average gradient value of the outermost contour points is compared with the minimum gradient threshold. If the average gradient value of the outermost contour points is less than the minimum gradient threshold, it is recorded as a non-reflective marker point light spot area.

[0022] Based on the above technical solution, preferably, step S8 further includes setting a maximum residual threshold. The set of boundary contour points whose average residual value is less than the maximum residual threshold is the boundary of the reflective marker spot area, and the center point of its ideal ellipse is the center of the reflective marker area.

[0023] The method for detecting the center of reflective marker area of ​​the present invention has the following advantages over the prior art:

[0024] (1) By acquiring the original image data taken by the SLR camera, which has richer contrast values ​​compared with ordinary grayscale image data, the light spot area of ​​the reflective marker can have a larger grayscale value difference with the ordinary background. The reflective marker is screened by gradient threshold, grayscale threshold and residual threshold. After the reflective marker is determined, the center of the marker area is located, which can greatly improve the accuracy of the marker location. At the same time, the influence of complex external lighting environment is eliminated, so that the operator can perform calculations efficiently and quickly under complex lighting conditions.

[0025] (2) After binarization, the obtained spot area may be smaller than the actual spot area of ​​the marker point. The gradient value of the boundary of the actual spot area of ​​the marker point is theoretically larger than the gradient value of the inner spot area. After searching around with the goal of finding the maximum gradient point, more pixels can be found. Then, by using the minimum bounding rectangle method, the maximum gradient point on the line segment is found by connecting the center point and the boundary point of the rectangle. The points between the maximum gradient point and the center point are all pixels of the spot area. The new spot area obtained in this way will be closer to the actual spot, which can improve the detection accuracy. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a perspective view of the reflective marker point area center detection method of the present invention;

[0028] Figure 2 This is a processed PNG image taken by a DSLR camera under the required settings in the reflective marker area center detection method of the present invention.

[0029] Figure 3 This is the original image after processing in step S2 of the original data image in the reflective marker area center detection method of the present invention. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0031] Many existing shooting devices used in photogrammetry are developed based on industrial cameras with auxiliary red or blue light sources. The shooting effect is generally poor in complex lighting conditions such as outdoors. Photogrammetry devices based on SLR cameras cannot achieve a good distinction between the background and the target by directly processing the image with the camera. Furthermore, the image processing of the camera itself leads to the distortion of data accuracy.

[0032] like Figure 1-3As shown, the reflective marker area center detection method of the present invention can process reflective marker images taken under complex lighting conditions and solve the above-mentioned problems, specifically including steps S1-S7.

[0033] To ensure that photogrammetry can capture markers a few millimeters in diameter at distances of 3 meters or more with sufficient pixel extraction, a high-resolution camera is required. This means an effective pixel count of 10 million or even 40 million. In this proposal, the DSLR camera referred to is a full-frame DSLR camera with at least 12 million pixels. A high-end full-frame DSLR camera can ensure sufficient brightness in photos taken with smaller exposures, resulting in better image quality. At the same time, it also has a larger shooting range and larger size. With the same lens and aperture, the field of view is larger, and the larger the field of view, the more markers can be captured at the same distance, resulting in a larger measurement range.

[0034] Step S1: Use an SLR camera to photograph the reflective markers to obtain the original image data.

[0035] To photograph reflective markers under complex lighting conditions using a DSLR camera, a mainstream high-end DSLR camera such as Canon or Nikon can be used, equipped with an auxiliary flash that supports high-speed sync. The photographs should be taken in sunlight, low light, or LED lighting environments, focusing on the reflective markers affixed to the object being measured. To prevent camera shake from affecting the image, the default exposure setting should be at least 180 frames per second (ensuring the DSLR camera and flash support the specified frame rate synchronization). The image should be set to capture the raw image, and all image data modification / optimization functions, such as lens aberration correction and image brightness correction, should be disabled. The aperture should be set to f / 11 to ensure sufficient depth of field. Finally, based on the camera's resolution and the size of the marker, the optimal shooting distance should be determined. After autofocusing, the camera's autofocus function should be disabled, and the raw image data should be captured. Since reflective markers are more sensitive to direct light from the flash, these settings will result in high-quality images with a dark background and bright reflective points.

[0036] In this step, a high-end DSLR camera is used. The camera should support an auxiliary flash and high-speed shooting at least 180 frames per second. The raw image data should support shooting 10-bit, 12-bit, or other high-bit images. The aperture should also be adjustable. This ensures that high-quality raw image data with dark backgrounds and bright reflections can be captured even under complex lighting conditions.

[0037] Step S2: Obtain an image with a grayscale value of 10 bits or more based on the original image data.

[0038] Parse the raw image data of DSLR cameras (such as the.CR2 format of Cannon cameras) through the Dcraw open-source library (commercial DSLR cameras all have color photosensitive chips, and the actual grayscale images provided are also 8-bit images obtained by merging RGB data). Perform Demosaicing to remove the mosaic, convert the Bayer pattern image into RGB data, and then only extract the data of the green channel as grayscale image data. Perform basic sharpening (Capture Sharpening) on the image to enhance the sharpness of the light spot edges of the reflective marking points; thus, obtain a 12-bit grayscale image with a darker background and more prominent light spot brightness of the reflective marking points. It should be noted that the 12-bit grayscale image here refers to a high-bit grayscale image, and the 12-bit grayscale image is only a preferred embodiment.

[0039] Step S3: Calculate the gradient map and binary map of the grayscale image.

[0040] The gradient map is obtained by calculating with a Gaussian gradient operator, and the binary map is obtained based on a gradient threshold and a grayscale threshold.

[0041] Specifically, calculate the gradient map of the image through a Gaussian gradient operator, and obtain a binary image based on the gradient threshold and the grayscale threshold. That is, let i represent the i-th row of the image, j represent the j-th column of the image, Value(i, j) represent the grayscale value of the pixel at the i-th row and j-th column of the image; Grad(i, j) represent the gradient value of the pixel at the i-th row and j-th column of the image; minValue is the minimum grayscale threshold, maxValue is the maximum grayscale threshold, and minGrad is the minimum gradient threshold; the pixels where Grad(i, j) < minGrad and Value(i, j) > minValue or Value(i, j) > maxValue are tentatively defined as the pixels of the marked point elliptical light spot, and the remaining pixels are tentatively defined as background pixels.

[0042] Step S4: Remove the light spot areas with too small and too large pixel areas in the binary map, and perform a maximum gradient area range search on the remaining light spot area point set to obtain an effective light spot area.

[0043] Removing the light spot areas with too small and too large pixel areas in the binary map includes removing the light spots with light spot pixel areas larger than the maximum light spot pixel area and the light spots with light spot pixel areas smaller than the minimum light spot pixel area.

[0044] The camera with a fixed resolution takes pictures of the marking points with a diameter of 6 - 12 mm at the maximum and minimum shooting distances respectively, and estimates the ideal light spot area range, that is, obtains the maximum light spot pixel area and the minimum light spot pixel area. Photogrammetry equipment has a certain depth of field range, which also determines the farthest and nearest shooting distances.

[0045] Based on the maximum and minimum pixel areas of the light spot, the light spot regions are searched, i.e., light spot regions with pixel areas that are too small or too large are removed. The size of the obtained light spot regions must be smaller than the maximum pixel area and larger than the minimum pixel area. The light spot region point sets that may be marker point elliptical light spots are obtained: Set1_A0{point(i1_0, j1_0), ..., point(i1_K, j1_K)}, Set2_B0{point(i2_0, j2_0), ..., point(i2_P, j2_P)}, ..., SetN_C0{point(iN_0, jN_0), ..., point(iN_Q, jN_Q)}; where point(i, j) represents the pixel point in the i-th row and j-th column of the image; A0, B0, ..., C0 represent the number of points in each region point set.

[0046] For the remaining spot region point set, perform a maximum gradient region range search. This involves marking all pixels in the entire spot region point set as inliers. For each inlier, query other pixels in eight directions. Specifically, each inlier queries other pixels along the eight directions: up, down, left, right, upper left, lower left, upper right, and lower right. If the gradient value of a queried pixel other than the inlier is greater than the gradient value of the inlier, then that pixel is temporarily marked as an inlier. This recursive search continues until no new pixels can be obtained as inliers, thus obtaining a new spot region point set Set1_A1{point(i1_0, j1_0), ..., po int(i1_K, j1_K)..., point(i1_T, j1_T)}, Set2_B1{point(i2_0, j2_0),..., point(i2_P, j2_P)..., point(i2_Y, j2_Y)},..., SetN_C1{point(iN_0, jN_0),..., point(iN_Q, jN_Q)..., point(iN_L, jN_L)}; where point(i, j) represents the pixel in the i-th row and j-th column of the image; A1, B1,..., C1 represent the number of points in each region point set.

[0047] Step S5: Calculate the centroid of the effective spot region and obtain the minimum bounding rectangle of the effective spot region. Find the maximum gradient point on the line connecting the mass point and the point on the side length of the minimum bounding rectangle. Collect the points between the maximum gradient point and the mass point to obtain the point set of the spot region.

[0048] Calculate the centroid of the effective spot region, where the centroid is an interior point of the effective spot region point set; otherwise, mark it as a non-reflective marker spot region.

[0049] Specifically, for each spot region point set, the centroid of each spot contour point set is calculated using the pixel coordinate value point(i,j). For example, the centroid of the Set1_A1 point set is calculated as follows:

[0050] Center_i = (i1_0 + ... + i1_K + ... + i1_T) / A1, taking the integer value;

[0051] Center_j = (j1_0 + ... + j1_K + ... + j1_T) / A1, taking the integer value;

[0052] Where point(Center_i, Center_j) represents the pixel in the Center_i row and Center_j column of the image, which is the centroid of the Set1_A1 spot region point set.

[0053] After obtaining the centroid of the effective spot region, calculate the minimum bounding rectangle based on the spot region; then, along the side length of the rectangle, with a step size of 1 pixel, find the point of maximum gradient between all pixels on the side length and the mass point, and set the pixels between the maximum gradient point and the mass point as new intra-region points, thus obtaining a new set of spot region points:

[0054] Set1_A2{point(i1_A0,j1_A0),…,point(i1_AK,j1_AK)},

[0055] Set2_B2{point(i2_A0,j2_A0),…,point(i2_AP,j2_AP)},

[0056] …,

[0057] SetN_C2{point(iN_A0,jN_A0),…,point(iN_AQ,jN_AQ)}.

[0058] Step S6: Extract the outermost contour points from the spot area point set to obtain the boundary contour point set, and remove the effective spot areas corresponding to the non-closed boundary contour point set.

[0059] The outermost contour is extracted based on the point set of the spot region, and the outermost contour points are set as boundary points to obtain the boundary contour point set:

[0060] BSet1_A2{point(i1_B0,j1_B0),…,point(i1_BF,j1_BF)},

[0061] BSet2_B2{point(i2_B0,j2_B0),…,point(i2_BG,j2_BG)},

[0062] …,

[0063] BSetN_C2{point(iN_B0,jN_B0),…,point(iN_BT,jN_BT)}.

[0064] Based on the boundary contour point set, starting from the first boundary point, connect the boundary points in a clockwise direction and detect whether the boundary contour is adjacent to each other to form a closed contour. If it is not a closed contour, it is a non-reflective marker spot area.

[0065] Step S7: Calculate the average gradient of the boundary contour point set and filter the remaining effective spot areas.

[0066] Calculate the average gradient of all boundary points in the boundary contour point set. The boundary contour point set with an average gradient greater than or equal to minGrad is the reflective marker spot region, where minGrad is the minimum gradient threshold in step S3. Similarly, the boundary contour point set with an average gradient less than minGrad is the non-reflective marker spot region.

[0067] Step S8: Extract the sub-pixel points of the boundary points of the remaining effective light spot region boundary contour point set, and perform ellipse fitting to obtain the ideal ellipse. Then calculate the average distance residual from the corresponding boundary contour point set to the ideal ellipse, screen out the effective reflective marker points, and record the center of the ideal ellipse as the region center of the reflective marker point.

[0068] Sub-pixel extraction is performed on the boundary points of all valid boundary contour point sets. This involves finding the sub-pixel point with the largest gradient based on the gradient direction and gradient values ​​in the X and Y directions of the boundary point. Then, ellipse fitting is performed based on the coordinates of the sub-pixel point set to obtain the ideal ellipse equation. The average residual distance from the boundary point to the ellipse equation is calculated, and a maximum residual threshold is set. Boundary contour point sets with an average residual less than the maximum residual threshold are considered the boundaries of the reflective marker spot region. The center point of the ideal ellipse equation of these points is then the center of the reflective marker region.

[0069] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting the center of a reflective marker area, characterized in that, Includes the following steps: S1. Use a DSLR camera to photograph the reflective markers to obtain the original image data; S2. Based on the original image data, obtain an image with a grayscale level of 10 bits or more; S3. Calculate the gradient map and binarized map of the grayscale image. The gradient map is obtained by calculating the Gaussian gradient operator, and the binarized map is obtained based on the gradient threshold and the grayscale threshold. Let i represent the i-th row of the image, j represent the j-th column of the image, Value(i,j) represent the grayscale value of the pixel in the i-th row and j-th column of the image; Grad(i,j) represent the gradient value of the pixel in the i-th row and j-th column of the image; minValue is the minimum grayscale threshold, maxValue is the maximum grayscale threshold, and minGrad is the minimum gradient threshold; pixels where Grad(i,j) < minGrad and Value(i,j) > minValue or Value(i,j) > maxValue are temporarily designated as pixels of the elliptical light spot of the marker point, and the remaining pixels are temporarily designated as background pixels; S4. Remove the spot areas with pixel areas that are too small or too large in the binarized image, and perform a maximum gradient region range search on the remaining spot area point set to obtain the effective spot area. The maximum gradient region range search on the remaining spot area point set includes marking all pixels in all spot area point sets as interior points. Each interior point queries other pixels in eight directions. If the gradient value of the queried pixel other than the interior point is greater than the gradient value of the interior point, then the point is marked as an interior point. Repeated recursive search is performed until no new pixel can be obtained as an interior point. S5. Calculate the centroid of the effective spot area and obtain the minimum bounding rectangle of the effective spot area. Find the maximum gradient point on the line connecting the mass point and the point on the side length of the minimum bounding rectangle. Collect the points between the maximum gradient point and the mass point to obtain the spot area point set. The centroid is the interior point of the effective spot area point set. Otherwise, it is marked as a non-reflective marker spot area. S6. Extract the outermost contour points from the point set of the spot area to obtain the boundary contour point set, and remove the effective spot areas corresponding to the non-closed boundary contour point set. S7. Calculate the average gradient of the boundary contour point set and filter the remaining effective spot areas. The filtering of the remaining effective spot areas includes comparing the average gradient of the outermost contour points with the minimum gradient threshold. If the average gradient of the outermost contour points is less than the minimum gradient threshold, it is recorded as a non-reflective marker spot area. S8. Extract the sub-pixel points of the boundary points of the remaining effective light spot region boundary contour point set, and perform ellipse fitting to obtain the ideal ellipse. Then calculate the average residual distance from the corresponding boundary contour point set to the ideal ellipse, set a maximum residual threshold, and confirm the boundary contour point set whose average residual is less than the maximum residual threshold as the effective reflective marker point. Record the center of the ideal ellipse as the region center of the reflective marker point.

2. The method for detecting the center of a reflective marker area as described in claim 1, characterized in that, The DSLR camera is a full-frame DSLR camera with a resolution of 12 megapixels or higher, and is equipped with an auxiliary flash with high-speed sync functionality.

3. The method for detecting the center of a reflective marker area as described in claim 1, characterized in that, Step S2 further includes parsing the original image data of the SLR camera using the Dcraw open-source library, performing de-mosaic processing, converting the Bayerpattern image into RGB data, extracting only the green channel data as grayscale image data, and performing basic sharpening on the image.

4. The method for detecting the center of a reflective marker area as described in claim 1, characterized in that, In step S4, removing spot areas with excessively small or large pixel areas in the binarized image includes removing spots with pixel areas larger than the largest pixel area and spots with pixel areas smaller than the smallest pixel area.

Images