Position sensing mark with anti-deformation capability and identification method thereof

Abstract

The invention provides a position sensing mark with anti-deformation capability and an identification method thereof, the position sensing mark is a triangular grid structure formed by arranging a plurality of dots with different colors, in the triangular grid structure, a triangle formed by three dots is taken as a minimum grid unit, and the minimum grid unit is taken as a minimum grid unit. A hexagon formed by six dots surrounding a central dot is used as a self-recognition unit; each self-identification unit obtains a self-identification unit ID according to a preset ID reading sequence and the color polarity of adjacent dots in a given reading direction; each self-identification unit ID in the position sensing mark is unique so as to carry out ID identification; the ID reading sequence is a preset reading sequence for traversing all 12 edges of the self-identification unit and a reading direction of the adjacent dots of each edge by taking a connecting line between two adjacent dots in the self-identification unit as an edge. According to the invention, identification is carried out according to the relative colors of the adjacent elements, and the method has higher deformation resistance and brightness non-uniformity resistance.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly relates to a position sensing marker with anti-deformation ability and an identification method thereof. Background Art

[0002] The method based on visual positioning markers is to attach visual positioning markers such as ARTag and CALTag on the surface, and track the marker features point by point to achieve surface tracking. This method benefits from the unique coding method of visual positioning markers, making the reading process simple and efficient, and having the advantages of low cost, fast calculation, high precision, etc. At present, this method has been successfully applied to many application scenarios.

[0003] In the past few years, the rapid development of the field of visual positioning markers and its profound impact on the entire computer vision field. At present, visual positioning markers are divided into four types: repetitive markers, landmark markers, self-identifying markers, and position sensing markers. The current visual positioning markers are mainly designed for planar or approximately planar application scenarios. The anti-deformation ability of the markers is insufficient. In application scenarios where the surface is bent, wrinkled, folded, stretched or even torn, such as tracking deformable surfaces such as the human body and flexible materials, visual positioning markers often fail due to surface deformation. The fundamental reasons for deformation failure mainly focus on the following two aspects: 1) Boundary loss. Boundary loss at the folding part is one of the problems that need to be studied for current visual positioning markers; 2) Uneven brightness. When deformation occurs, due to different incident angles of light, shadows appear on the surface, and the brightness of the shadow part of the marker is inconsistent with that of other parts, resulting in misidentification.

[0004] Regarding boundary loss, the position sensing marker can contain several times more positioning features than other markers under the same area, and can reduce the influence brought by a large part of boundary loss compared with other markers. However, the position sensing marker will also fail under the influence of uneven brightness; two different position sensing markers also misidentified under the exposure environment; the reason is that under the influence of uneven brightness, if the threshold is not adjusted for identification, it will be very challenging to identify different colors simultaneously. However, it is much easier to identify their relative colors; the most direct method to achieve relative color identification is to detect the relationship of the RGB three-channel values.

[0005] In view of this, aiming at the problem that the design and identification of existing visual markers rely on the independent color or shape of features and are sensitive to deformation, the present invention designs a position sensing marker that is identified based on relative color. This marker is no longer limited to the color of a single element, but is identified according to the color polarity of adjacent elements, and has stronger anti-deformation ability, providing a strong technical support for surface tracking. Summary of the Invention

[0006] The object of the present invention is to provide a position - sensing marker with anti - deformation ability and its recognition method, which can be used for calibration, registration, robot navigation, surgical navigation, visual servo, AR, motion tracking, etc.

[0007] To achieve the above object, the technical solution of the present invention is as follows:

[0008] On the one hand, the present invention provides a position - sensing marker with anti - deformation ability. The position - sensing marker is a triangular grid structure composed of a number of dots of different colors. In the triangular grid structure, a triangle composed of three dots is the smallest grid unit, and a hexagon composed of 6 dots surrounding a central dot is a self - identification unit.

[0009] Each self - identification unit obtains a self - identification unit ID according to a preset ID reading order and the color polarity of adjacent dots in a given reading direction. Each self - identification unit ID in the position - sensing marker is unique for ID recognition.

[0010] The ID reading order uses the connection line between two adjacent dots in the self - identification unit as an edge, and presets the reading order of traversing all 12 edges of the self - identification unit and the reading direction of adjacent dots for each edge.

[0011] Preferably, in the position - sensing marker, the value of each dot's color in the R, G, and B channels is greater than or equal to 0.2, and the marker background is selected as white to distinguish the background of the position - sensing marker.

[0012] Preferably, the color polarity of adjacent dots in a given direction is defined as a three - dimensional vector [r, g, b] composed of +1 or -1, where r, g, and b respectively represent the color polarities of the R, G, and B channels of adjacent dots in the given direction. For each of the R, G, and B channels, if the value of the adjacent dot increases in the given direction, the color polarity of the corresponding channel is represented as +1, and if the value of the adjacent dot decreases in the given direction, the color polarity of the corresponding channel is represented as -1. The colors of adjacent dots are different in the R, G, and B channels.

[0013] Preferably, in each self-identification unit, the side pointing from the central dot to the adjacent dot on the left is defined as the first reading order direction, the side pointing from the central dot to the adjacent dot in the upper left is defined as the second reading order direction, the side pointing from the central dot to the adjacent dot in the upper right is defined as the third reading order direction, the side pointing from the central dot to the adjacent dot on the right is defined as the fourth reading order direction, the side pointing from the central dot to the adjacent dot in the lower right is defined as the fifth reading order direction, and the side pointing from the central dot to the adjacent dot in the lower left is defined as the sixth reading order direction; taking the dot adjacent to the central dot on the left as the starting point and the ending point, the six sides around the self-identification unit are respectively set as the seventh reading order direction, the eighth reading order direction, the ninth reading order direction, the tenth reading order direction, the eleventh reading order direction, and the twelfth reading order direction along the clockwise direction.

[0014] On the other hand, the present invention proposes an identification method for the above-mentioned position perception marker with anti-deformation ability, which specifically includes the following steps:

[0015] S1. Obtain the original image of the position perception marker in the application scenario;

[0016] S2. Perform feature detection on the obtained original image to screen out the centroid of the dots existing in the image;

[0017] S3. Divide the screened center points into grids, screen the hexagonal grids, and reconstruct the screened hexagonal grids;

[0018] S4. Perform ID identification on each hexagon in the grid.

[0019] Preferably, the S2 is specifically as follows:

[0020] S21. Apply a Gaussian filter to blur the original image;

[0021] S22. Perform segmentation processing on the blurred image to obtain a binary image;

[0022] S23. Perform morphological opening operation on the binary image, and the morphological opening operation is the processing of first eroding and then dilating the image; then use regionprops to calculate the centroid of the connected components in the image to complete the feature detection.

[0023] Preferably, the blurred image is converted into a grayscale image, and then the Sauvola algorithm is used for image binarization processing.

[0024] Preferably, the S3 is specifically as follows:

[0025] S31. Perform triangulation on the points in the feature detection matrix, divide the points in the image into a series of connected but non-overlapping triangles, and the circumcircle of the triangle does not contain any other points in its area;

[0026] S32. According to the triangulation result, eliminate the detection points in the feature detection matrix that are not connected to six points by one point, and then screen the structure of six points connected to each point on the edge to screen out the hexagonal grid structure;

[0027] S33. Integrate the screened hexagonal grids to complete the grid reconstruction.

[0028] Preferably, the S4 is specifically: for each hexagonal grid structure, according to 6 possible initial positions, read according to the preset ID reading order and the color polarity of adjacent dots in the given reading direction, and compare the obtained 6 ID reading results with the self-identification unit ID in the marking field to complete the position identification of the hexagonal grid structure in the marking field; if multiple corresponding results are obtained after comparing the 6 ID reading results with the self-identification unit ID in the marking field, the optimal ID is selected through a voting mechanism.

[0029] Preferably, the selection of the optimal ID through the voting mechanism is specifically:

[0030] Select a vertex of the hexagonal grid structure to be voted, obtain the ID corresponding results of adjacent multiple hexagonal grid structures containing the vertex, vote on the label of the vertex in the marking field according to the ID corresponding results of multiple hexagonal grid structures, determine the correct label of the vertex in the marking field according to the voting result, and use the ID containing the correct label among the multiple corresponding results as the optimal ID.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention defines the positivity and negativity of the first-order difference of the RGB values of two colors in a given direction as color polarity, and based on this, proposes a position-aware marker to be recognized according to the relative colors of adjacent elements, which has stronger anti-deformation and anti-uneven brightness capabilities and can be used in research fields such as surgical instrument positioning, clothing surface capture, and surface tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the position-aware marker of the present invention;

[0034] Figure 2 It is a schematic diagram of the grid reconstruction process of the present invention;

[0035] Figure 3 It is a schematic diagram of the process of selecting the optimal ID through the voting mechanism of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0036] The technical solutions of the present invention will be specifically described below with reference to the accompanying drawings.

[0037] The present invention provides a position - sensing marker with anti - deformation ability. The position - sensing marker is a triangular grid structure composed of a number of dots of different colors. The dots are positioning features. In the triangular grid structure, a triangle composed of three dots is the smallest grid unit, and a hexagon composed of 6 dots surrounding a central dot is a self - identification unit;

[0038] Each of the self - identification units obtains a self - identification unit ID according to a preset ID reading order and the color polarities of adjacent dots in a given reading direction; each self - identification unit ID in the position - sensing marker is unique for ID identification;

[0039] The ID reading order uses the connection line between two adjacent dots in the self - identification unit as an edge, and presets the reading order of traversing all 12 edges of the self - identification unit and the reading directions of adjacent dots on each edge.

[0040] In this embodiment, in the position - sensing marker, the value of the color of each dot in the R, G, and B channels is greater than or equal to 0.2, and the background of the marker is selected as white to distinguish the background of the position - sensing marker.

[0041] In this embodiment, the color polarity of adjacent dots in a given direction is defined as a three - dimensional vector [r, g, b] composed of +1 or -1, where r, g, and b respectively represent the color polarities of the R, G, and B channels of adjacent dots in the given direction; for each of the R, G, and B channels, if the value of the adjacent dot increases in the given direction, the color polarity of the corresponding channel is represented as +1, and if the value of the adjacent dot decreases in the given direction, the color polarity of the corresponding channel is represented as -1; the colors of adjacent dots are different in the R, G, and B channels.

[0042] In this embodiment, it is preset that in each self - identification unit, the edge from the central dot to the adjacent left - hand dot is the first reading order direction, the edge from the central dot to the adjacent upper - left dot is the second reading order direction, the edge from the central dot to the adjacent upper - right dot is the third reading order direction, the edge from the central dot to the adjacent right - hand dot is the fourth reading order direction, the edge from the central dot to the adjacent lower - right dot is the fifth reading order direction, and the edge from the central dot to the adjacent lower - left dot is the sixth reading order direction; taking the adjacent left - hand dot of the central dot as the starting point and the ending point, the six edges around the self - identification unit are respectively set as the seventh reading order direction, the eighth reading order direction, the ninth reading order direction, the tenth reading order direction, the eleventh reading order direction, and the twelfth reading order direction along the clockwise direction.

[0043] As Figure 1As shown in the figure, (a) is an anti-deformation position sensing marker, the circle is the positioning feature, and the hexagon is the self-identification unit; (b) shows the RGB values of the colors at both ends of the color polarity vector and the given direction, and the obtained polarity vector is [-1, 1, 1]; (c) shows the self-identification unit and the ID reading order. Each ID contains n = 12 sets of color polarities, and there are α = 2 3 = 8 polarities. Therefore, the upper bound of the total number of IDs is α n = 8 12 , which is much larger than most position sensing markers.

[0044] The present invention also proposes a recognition method for the above-mentioned position sensing marker with anti-deformation ability, which specifically includes the following steps:

[0045] S1. Obtain the original image of the position sensing marker in the application scenario;

[0046] S2. Perform feature detection on the obtained original image to screen out the centroid of the dot existing in the image;

[0047] S3. Perform grid division on the screened center of the circle, screen the hexagonal grid, and reconstruct the screened hexagonal grid;

[0048] S4. Perform ID recognition on each hexagon in the grid.

[0049] In this embodiment, the specific content of S2 is as follows:

[0050] S21. Apply a Gaussian filter to blur the original image; reduce the noise of the image;

[0051] S22. Perform segmentation processing on the blurred image to obtain a binary image;

[0052] S23. Perform morphological opening operation on the binary image. The morphological opening operation is the process of first eroding and then dilating the image, which can not only eliminate the small noise points in the image, but also smooth the boundary of the dot without significantly changing its area and shape; then use regionprops to calculate the centroid of the connected components in the image to complete the feature detection; the regionprops function returns the centroid in the form of a structure array, and stores the x and y coordinates of the centroid into a two-column matrix to complete the feature detection.

[0053] In this embodiment, the blurred image is converted into a grayscale image, and then the Sauvola algorithm is used for image binarization processing.

[0054] In this embodiment, as Figure 2 shown, the specific content of S3 is as follows:

[0055] S31. Triangulate the points in the feature detection matrix, divide the points in the image into a series of connected but non-overlapping triangles, and the circumcircle of the triangle does not contain any other points in its region;

[0056] S32. According to the triangulation result, eliminate the detection points in the feature detection matrix that are not connected to six points by one point, and then screen the structure where each point is connected to six points on the edge to screen out the hexagonal grid structure; store the x and y coordinates of the 7 points included in each hexagon into a fourteen-column matrix.

[0057] S33. Integrate the screened hexagonal grids to complete grid reconstruction.

[0058] In this embodiment, the specific content of S4 is as follows: For each hexagonal grid structure, according to 6 possible initial positions, read according to the preset ID reading order and the color polarity of adjacent dot points in the given reading direction, and compare the 6 obtained ID reading results with the self-identification unit ID in the marking field to complete the position identification of the hexagonal grid structure in the marking field; if multiple corresponding results are obtained after comparing the 6 ID reading results with the self-identification unit ID in the marking field, the optimal ID is selected through a voting mechanism.

[0059] In this embodiment, the specific method of selecting the optimal ID through the voting mechanism is as follows:

[0060] Select a vertex of the hexagonal grid structure to be voted, obtain the ID corresponding results of adjacent multiple hexagonal grid structures containing the vertex, vote on the label of the vertex in the marking field according to the ID corresponding results of multiple hexagonal grid structures, determine the correct label of the vertex in the marking field according to the voting result, and use the ID containing the correct label among the multiple corresponding results as the optimal ID.

[0061] As Figure 3 shown, for the 6 ID reading results of the hexagonal grid in the red part, two groups of corresponding results on the right side of the figure are obtained by comparing with the marking field, and then the optimal ID is selected through the voting mechanism; obtain the ID reading results of adjacent hexagonal grid structures containing the vertex in the blue frame, determine that the label of the vertex in the blue frame in the marking field is 7, and thus determine Figure 3 the ID reading result corresponding to the middle part as the optimal ID.

[0062] The above are the preferred embodiments of the present invention. All changes made according to the technical solution of the present invention, when the functions and effects generated do not exceed the scope of the technical solution of the present invention, fall within the protection scope of the present invention.

Claims

1. A position sensing marker with anti-deformation capability, characterized in that: The position sensing mark is a triangular grid structure composed of a plurality of dots of different colors, in which a triangle composed of three dots is the smallest grid unit, and a hexagon composed of six dots surrounding a central dot is a self-identification unit; Each of the self-identification units obtains a self-identification unit ID according to a preset ID reading sequence and the color polarity of adjacent dots in a given reading direction; each self-identification unit ID in the position sensing mark is unique for ID identification; The ID reading order is based on the line between two adjacent dots in the self-identification unit as an edge, and the preset reading order of traversing all 12 edges of the self-identification unit and the reading direction of the adjacent dots of each edge.

2. A position sensing marker with anti-deformation capability according to claim 1, characterized in that: In the position sensing mark, the color values ​​of each dot in the three channels of R, G, and B are all greater than or equal to 0.2, and the mark background is selected to be white to distinguish the background of the position sensing mark.

3. A position sensing marker with anti-deformation capability according to claim 2, characterized in that: The color polarity of adjacent dots in a given direction is defined as a three-dimensional vector [r, g, b] composed of +1 or -1, wherein r, g, and b respectively represent the color polarity of the R, G, and B channels of the adjacent dots in the given direction; for each channel of R, G, and B, if the value of the adjacent dots in the given direction increases, the color polarity of the corresponding channel is expressed as +1, and if the value of the adjacent dots in the given direction decreases, the color polarity of the corresponding channel is expressed as -1; the values ​​of the colors of the adjacent dots in the three channels of R, G, and B are all different.

4. The position sensing marker with anti-deformation capability according to claim 1, characterized in that: It is preset that in each self-identification unit, the edge from the center dot to the adjacent left dot is the first reading sequence direction, the edge from the center dot to the adjacent upper left dot is the second reading sequence direction, the edge from the center dot to the adjacent upper right dot is the third reading sequence direction, the edge from the center dot to the adjacent right dot is the fourth reading sequence direction, the edge from the center dot to the adjacent lower right dot is the fifth reading sequence direction, and the edge from the center dot to the adjacent lower left dot is the sixth reading sequence direction; with the adjacent left dot of the center dot as the starting point and the end point, the six edges around the self-identification unit are set in a clockwise direction as the seventh reading sequence direction, the eighth reading sequence direction, the ninth reading sequence direction, the tenth reading sequence direction, the eleventh reading sequence direction, and the twelfth reading sequence direction, respectively.

5. A method for identifying a position sensing marker with anti-deformation capability according to any one of claims 1 to 4, characterized in that: The specific steps include: S1, obtaining the original image of the application scene location awareness mark; S2, performing feature detection on the acquired original image to screen out the center of mass of the dots in the image; S3, meshing the selected circle centers, screening the hexagonal meshes, and reconstructing the screened hexagonal meshes; S4. Perform ID identification on each hexagon in the grid.

6. A method for identifying a position sensing marker with anti-deformation capability according to claim 5, characterized in that: The S2 is specifically: S21, applying a Gaussian filter to blur the original image; S22, segmenting the blurred image to obtain a binary image; S23, performing a morphological opening operation on the binary image, wherein the morphological opening operation is an erosion-then-expansion process on the image; and then using regionprops to calculate the centroid of connected components in the image to complete feature detection.

7. A method for identifying a position sensing marker with anti-deformation capability according to claim 6, characterized in that: The blurred image is converted into a grayscale image and then the Sauvola algorithm is used to perform image binarization.

8. A method for identifying a position sensing marker with anti-deformation capability according to claim 5, characterized in that: The S3 is specifically: S31, triangulate the points in the feature detection matrix to divide the points in the image into a series of connected but non-overlapping triangles, and the circumscribed circles of the triangles do not contain any other points in their surface areas; S32, according to the triangulation result, remove the detection points in the feature detection matrix that are not one point connected to six points, and then screen the structure of the six points on the edge connecting each point to screen out a hexagonal grid structure; S33. Integrate the selected hexagonal grids to complete grid reconstruction.

9. A method for identifying a position sensing marker with anti-deformation capability according to claim 5, characterized in that: Specifically, S4 includes: for each hexagonal grid structure, according to 6 possible initial positions, reading is performed in accordance with a preset ID reading order and the color polarity of adjacent dots in a given reading direction, and the obtained 6 ID reading results are compared with the self-identification unit ID in the marking field to complete the position identification of the hexagonal grid structure in the marking field; If multiple corresponding results are obtained after comparing the 6 ID reading results with the self-identification unit ID in the tag field, the optimal ID is selected through a voting mechanism.

10. A method for identifying a position sensing marker with anti-deformation capability according to claim 9, characterized in that: The specific method of selecting the optimal ID through voting mechanism is as follows: Select a vertex of the hexagonal grid structure to be voted, obtain the ID correspondence results of multiple adjacent hexagonal grid structures containing the vertex, vote on the label of the vertex in the label field according to the ID correspondence results of multiple hexagonal grid structures, determine the correct label of the vertex in the label field according to the voting result, and use the ID containing the correct label in the multiple correspondence results as the optimal ID.

Images