Cone-cylindrical surface type marker and its recognition method

Abstract

An embodiment of the present application provides a conical-cylindrical marker and a method for identifying the same. Among them, the conical-cylindrical marker includes: a cylindrical part, on the cylindrical surface of which multiple layers of two-dimensional codes are printed, and each layer of the two-dimensional codes in the multiple layers of two-dimensional codes is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of each two-dimensional code in the multiple layers of two-dimensional codes form positioning corner points with a centrosymmetric pattern; a conical part, the bottom surface of the conical part is attached to the top surface of the cylindrical part, and a two-dimensional code is printed on the conical surface of the conical part, and the two-dimensional code printed on the conical surface is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of the two-dimensional code printed on the conical surface form positioning corner points with a centrosymmetric pattern. This solution can effectively improve the recognition and positioning accuracy of surgical navigation.

Description

Technical Field

[0001] The present application relates to the field of surgical robots, and particularly to a conical-cylindrical marker, a recognition method for the conical-cylindrical marker, an electronic device, and a computer-readable medium. Background Art

[0002] Automation devices such as robots have been widely used in the industrial field and have shown obvious advantages in terms of operation flexibility, stability, and accuracy. To solve problems existing in surgical operations such as insufficient precision, excessive radiation, large incisions, and operation fatigue, people have begun to explore how to introduce robotic methods in surgical operations, leveraging the unique advantages of high-tech devices such as robots and sensors to provide surgeons with new treatment methods and systems, solve the above problems, and improve the surgical effect, that is, to implement surgical operations with the help of medical robots, including those related to nerves, the head, and so on.

[0003] Surgical navigation technology is a key link in the operation process of medical robots. The structure on which the implementation of surgical navigation technology depends includes a marker, which is a visual module with a specific pattern. By recognizing the specific pattern on the visual module, the marker is automatically located, thereby determining the position of the patient relative to the robot. The robot system completes surgical navigation according to the surgical plan planned by the doctor. The processing precision of the marker body, the processing precision of the recognition pattern, the assembly precision, etc. all directly affect the precision of surgical navigation. This marker is an important accessory for the entire robotic surgery. However, due to certain reasons such as cost considerations, there are usually cases of counterfeiting of the marker, which thus has a negative impact on the precision of surgical navigation and causes direct harm to the patient.

[0004] Therefore, it can be seen that how to effectively improve the recognition and positioning precision of surgical navigation has become an urgent technical problem to be solved currently. Summary of the Invention

[0005] The purpose of the present application is to propose a conical-cylindrical marker, a recognition method for the conical-cylindrical marker, an electronic device, and a computer-readable medium, which are used to solve the technical problem of how to effectively improve the recognition and positioning precision of surgical navigation existing in the prior art.

[0006] According to the first aspect of the embodiments of the present application, a conical-cylindrical marker is provided, which is applied to surgical navigation. The conical-cylindrical marker includes: a cylindrical portion, on the cylindrical surface of which multiple layers of two-dimensional codes are printed, and each layer of the two-dimensional codes in the multiple layers of two-dimensional codes is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of each two-dimensional code in the multiple layers of two-dimensional codes form positioning corner points with a centrosymmetric pattern; a conical portion, the bottom surface of the conical portion is attached to the top surface of the cylindrical portion, and a two-dimensional code is printed on the conical surface of the conical portion. The two-dimensional code printed on the conical surface is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of the two-dimensional code printed on the conical surface form positioning corner points with a centrosymmetric pattern.

[0007] According to the second aspect of the embodiments of the present application, a method for identifying a conical-cylindrical marker is provided, which is applied to surgical navigation. The method includes: based on the surface equation corresponding to the grayscale image to be identified and the position data of the pixel points in the grayscale image, determining the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical marker in the grayscale image, where the conical-cylindrical marker is the conical-cylindrical marker according to the first aspect of the embodiments of the present application; based on the position data of the positioning corner points, finding all eight-neighborhood connected regions in the grayscale image, and traversing all eight-neighborhood connected regions in the grayscale image to find all eight-neighborhood connected regions adjacent to four adjacent positioning corner points; determining all eight-neighborhood connected regions adjacent to four adjacent positioning corner points as candidate regions of the two-dimensional code, and based on the position data of the four adjacent positioning corner points and the position data of the four vertices of a preset unit square two-dimensional code, determining projection transformation data; based on the projection transformation data, performing projection transformation on the candidate regions to obtain the region where the unit square two-dimensional code corresponding to the candidate regions is located, and based on the region where the unit square two-dimensional code corresponding to the candidate regions is located, identifying the conical-cylindrical marker in the grayscale image.

[0008] According to a third aspect of an embodiment of the present application, there is provided an identification device for a conical-cylindrical surface type marker, which is applied to surgical navigation. The device includes: a first determination module, configured to determine position data of positioning corner points of a two-dimensional code on the conical-cylindrical surface type marker in the grayscale image based on a surface equation corresponding to the grayscale image to be identified and position data of pixel points in the grayscale image, wherein the conical-cylindrical surface type marker is the conical-cylindrical surface type marker according to the first aspect of the embodiment of the present application; a traversal module, configured to find all eight-neighborhood connected regions of the grayscale image based on the position data of the positioning corner points, and traverse all eight-neighborhood connected regions of the grayscale image to find all eight-neighborhood connected regions adjacent to four adjacent positioning corner points; a second determination module, configured to determine that all eight-neighborhood connected regions adjacent to four adjacent positioning corner points are candidate regions of the two-dimensional code, and determine projection transformation data based on the position data of the four adjacent positioning corner points and position data of four vertices of a preset unit square two-dimensional code; an identification module, configured to perform projection transformation on the candidate regions based on the projection transformation data to obtain a region where the unit square two-dimensional code corresponding to the candidate regions is located, and identify the conical-cylindrical surface type marker in the grayscale image based on the region where the unit square two-dimensional code corresponding to the candidate regions is located.

[0009] According to a fourth aspect of an embodiment of the present application, there is provided an electronic device, including: one or more processors; a computer-readable medium configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the identification method of the conical-cylindrical surface type marker according to the second aspect of the embodiment of the present application.

[0010] According to a fifth aspect of an embodiment of the present application, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the identification method of the conical-cylindrical surface type marker according to the second aspect of the embodiment of the present application is implemented.

[0011] The conical-cylindrical marker provided by the embodiment of the present application includes: a cylindrical portion, on the cylindrical surface of which multiple layers of two-dimensional codes are printed, and each layer of the two-dimensional codes in the multiple layers of two-dimensional codes is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of each two-dimensional code in the multiple layers of two-dimensional codes form positioning corner points with a centrosymmetric pattern; a conical portion, the bottom surface of which is attached to the top surface of the cylindrical portion, and on the conical surface of which a two-dimensional code is printed, and the two-dimensional code printed on the conical surface is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of the two-dimensional code printed on the conical surface form positioning corner points with a centrosymmetric pattern. Compared with other existing methods, the conical-cylindrical marker provided by the embodiment of the present application includes a conical portion, which effectively increases the recognition range of the conical-cylindrical marker, and the corner points of the two-dimensional code printed on the cylindrical surface of the cylindrical portion of the conical-cylindrical marker are positioning corner points with a centrosymmetric pattern, and the corner points of the two-dimensional code printed on the conical surface of the conical portion of the conical-cylindrical marker are positioning corner points with a centrosymmetric pattern, which can effectively improve the recognition and positioning accuracy of surgical navigation. Description of the Drawings

[0012] Some specific embodiments of the embodiments of the present application will be described in detail hereinafter with reference to the drawings in an exemplary but not restrictive manner. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0013] Figure 1A Schematic diagram of the cylindrical portion of the conical-cylindrical marker provided in the first embodiment;

[0014] Figure 1B Schematic diagram of the positioning corner points of the two-dimensional code on the conical-cylindrical marker provided in the first embodiment;

[0015] Figure 1C Schematic diagram of the conical portion of the conical-cylindrical marker provided in the first embodiment;

[0016] Figure 1D Schematic diagram of the contour of the two-dimensional code on the conical portion of the conical-cylindrical marker provided in the first embodiment;

[0017] Figure 1E Schematic diagram of the projection transformation of the two-dimensional code provided in the first embodiment;

[0018] Figure 2 Flowchart of the steps of the method for recognizing the conical-cylindrical marker provided in the second embodiment;

[0019] Figure 3 Schematic diagram of the structure of the device for recognizing the conical-cylindrical marker provided in the third embodiment;

[0020] Figure 4 Schematic structural diagram of the electronic device provided in the fourth embodiment;

[0021] Figure 5 Hardware structure of the electronic device provided in the fifth embodiment. Specific implementation manners

[0022] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art shall fall within the protection scope of the embodiments of the present application.

[0023] A detailed description is given to the conical-cylindrical marker provided in the first embodiment. Specifically, the conical-cylindrical marker provided in the first embodiment is applied to surgical navigation and includes: a cylindrical portion, on the cylindrical surface of which multiple layers of two-dimensional codes are printed, and each layer of the two-dimensional codes in the multiple layers of two-dimensional codes is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of each two-dimensional code in the multiple layers of two-dimensional codes form positioning corner points with a centrosymmetric pattern; a conical portion, the bottom surface of which is attached to the top surface of the cylindrical portion, and on the conical surface of which a two-dimensional code is printed, and the two-dimensional code printed on the conical surface is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of the two-dimensional code printed on the conical surface form positioning corner points with a centrosymmetric pattern. Thereby, the conical-cylindrical marker includes a conical portion, effectively increasing the recognition range of the conical-cylindrical marker, and the corner points of the two-dimensional code printed on the cylindrical surface of the cylindrical portion of the conical-cylindrical marker are positioning corner points with a centrosymmetric pattern, and the corner points of the two-dimensional code printed on the conical surface of the conical portion of the conical-cylindrical marker are positioning corner points with a centrosymmetric pattern, which can effectively improve the recognition and positioning accuracy of surgical navigation.

[0024] In this embodiment, each two-dimensional code in the multi-layer two-dimensional code has an identification area and a background area located outside the identification area. The identification area is composed of small squares of different colors, and the background area is composed of small squares of the same color. According to the order of the small squares in the identification area from left to right and from top to bottom, determine whether the color of the small squares in the identification area is the same as the color of the small squares in the background area, and encode the small squares in the identification area according to the similarities and differences between the color of the small squares in the identification area and the color of the small squares in the background area, so as to obtain the code corresponding to the two-dimensional code where the identification area is located. The codes corresponding to each two-dimensional code in the multi-layer two-dimensional code are different, and the codes corresponding to each two-dimensional code in the multi-layer two-dimensional code after being rotated by a preset degree are also different.

[0025] In a specific example, the cone-cylinder-shaped marker proposed in this embodiment has a conical part, and the recognition range along the rotation direction of the cone-cylinder-shaped marker axis is larger. And because the style of the positioning corner points is centrosymmetric, the recognition and positioning will be more accurate. The cone-cylinder-shaped marker is composed of multiple two-dimensional codes. Recognizing any one of the two-dimensional codes can determine the position of the entire marker. The unfolded view of the cylindrical part is as Figure 1A shown, which is composed of two or more layers of black two-dimensional codes and white two-dimensional codes arranged at intervals. The purpose is to form black and white cross positioning corner points around each two-dimensional code for easy recognition and positioning, as Figure 1B shown. Each two-dimensional code is composed of 8×8 small squares. The outermost circle of small squares is the background grid, and the color of the background grid is the same within the same two-dimensional code. The two-dimensional code with the background grid defined as black is the black two-dimensional code, and the two-dimensional code with the background grid defined as white is the white two-dimensional code. The 6×6 grid inside the two-dimensional code is the identification area. The small squares in the identification area can be black or white. Define the small squares in the identification area with the same color as the background grid as 0, and those with different colors from the background grid as 1. Therefore, according to the order from left to right and from top to bottom, a two-dimensional code can be represented by a group of 36-bit binary numbers. For example, Figure 1A the upper left two-dimensional code can be represented by 001100000000010111101011000100001100. And each two-dimensional code must satisfy the anisotropic property, which means that after the same two-dimensional code is rotated by 90 degrees, the binary numbers read in the same order from left to right and from top to bottom are different. Such a property ensures that no matter how the two-dimensional code rotates in space, it can be uniquely confirmed, and the order of its four positioning corner points is also uniquely confirmed. Finally, all the two-dimensional codes that make up a cone-cylinder-shaped marker must satisfy that their corresponding binary codes are different.

[0026] In this embodiment, the shape of the conical surface is an annular shape obtained by subtracting two sectors with different radii and the same center. Moreover, for the two-dimensional code printed on the conical surface, each contour line along the radial direction intersects at the center of the sector, and the angle between each adjacent pair of contour lines is equal. The shape of the two-dimensional code printed on the conical surface is trapezoidal, and the spacing of the contour lines of the two-dimensional code printed on the conical surface in the radial direction is not equal. The trapezoidal two-dimensional code printed on the conical surface is obtained by a projection transformation of a unit square two-dimensional code, and the spacing is determined by the unit square two-dimensional code and the projection transformation.

[0027] In a specific example, Figure 1C is the developed view of the conical surface part, which is an annular shape obtained by subtracting two sectors with different radii and the same center. Similarly, a two-dimensional code also needs to be printed on the annular shape to facilitate identification in the image. The contour of the two-dimensional code pattern is as Figure 1D shown. Each contour line along the radial direction intersects at the center of the sector, and the angles between them are equal. For the contour lines in the circumferential direction, except for the outermost and innermost two contour lines which are arcs, the grid lines of the internal two-dimensional code are broken lines. Therefore, the developed view of a single two-dimensional code is a trapezoid, and the spacing of their internal contour lines in the radial direction is not equal. The calculation method of the spacing will be described below. Moreover, the black two-dimensional codes and the white two-dimensional codes are also arranged at intervals, and it is ensured that the four vertices of each two-dimensional code are the positioning corner points of black and white crosses.

[0028] In a specific example, as Figure 1E shown, the trapezoidal two-dimensional code is obtained by a projection transformation of a square two-dimensional code. As Figure 1E shown, the trapezoidal two-dimensional code is obtained by a projection transformation H of a square two-dimensional code with a side length of one, and H is a 3×3 matrix. If H is obtained, the corresponding point of any point in the square two-dimensional code in the trapezoidal two-dimensional code can be obtained, and the spacing of each line in the trapezoidal two-dimensional code can be obtained. Assuming that the vertex at the lower left corner of the two-dimensional code is the coordinate origin, the coordinates of its four vertices are as Figure 1E shown, which are (0,0), (1,0), (1,1), (0,1). We obtain the coordinates of the four vertices of the trapezoidal two-dimensional code, which are (0,0), (a,0), (a - c, h), (b, h), according to the external dimensions of the trapezoidal two-dimensional code (a, b, c, h in the figure). Then, H can be obtained through the coordinates of the four vertices of the two-dimensional code before and after the transformation. The method is as follows:

[0029] Because of the special property of H being a homogeneous matrix, the last element of H can be set to 1, and the first 8 elements of H are arranged into an 8×1 column vector If a set of corresponding points before and after the projection transformation are (x, y) and (x′, y′), the following equation can be obtained:

[0030]

[0031] Substituting the above four pairs of corresponding vertices into the equations, the following equations can be obtained:

[0032]

[0033] It is possible to Solve it, and then H can be obtained. After obtaining H, the interval between the rows of the trapezoidal two-dimensional code can be calculated. Taking the last row of the two-dimensional code as an example, before the projective transformation H, that is, the coordinates of the leftmost vertex of the last row of the square two-dimensional code are Converted to homogeneous coordinates as Through the projective transformation H, the homogeneous coordinates of the leftmost vertex of the last row of the trapezoidal two-dimensional code can be obtained as q′(x, y, w) = H * p′, and the obtained y is the interval distance of the last row of the trapezoidal two-dimensional code.

[0034] Referring to Figure 2 , the flowchart of the steps of the recognition method of the conical and cylindrical marker provided in the second embodiment is shown.

[0035] Specifically, the recognition method of the conical and cylindrical marker provided in this embodiment is applied to surgical navigation and includes the following steps:

[0036] In step S201, based on the surface equation corresponding to the grayscale image to be recognized and the position data of the pixel points in the grayscale image, determine the position data of the positioning corner points of the two-dimensional code on the conical and cylindrical marker in the grayscale image.

[0037] Wherein, the conical and cylindrical marker is the conical and cylindrical marker according to the first embodiment of the present invention.

[0038] In some alternative embodiments, when determining the position data of the positioning corner points of the two-dimensional code on the conical and cylindrical marker in the grayscale image, based on the surface equation and the position data of the pixel points in the grayscale image, solve the Hessian matrix centered on the pixel points in the grayscale image; in response to determining that the first eigenvalue of the Hessian matrix is positive, the second eigenvalue of the Hessian matrix is negative, and the product of the first eigenvalue and the second eigenvalue is the minimum in the local area centered on the pixel points in the grayscale image, then determine that the pixel points in the grayscale image are the saddle points of the surface equation; determine the position data of the saddle points of the surface equation as the position data of the positioning corner points of the two-dimensional code on the conical and cylindrical marker in the grayscale image. Thereby, the position data of the positioning corner points of the two-dimensional code on the conical and cylindrical marker in the grayscale image can be accurately determined.

[0039] In a specific example, the definition of a grayscale image is that the brightness of each pixel in the image is represented by an integer from 0 to 255, where 0 is the darkest and 255 is the brightest. Locate the corner points of all black and white crosses in the image. The shape of the corner points is as shown in Figure 1B If the image is regarded as a surface represented by a two-dimensional equation r(x, y), the positions of the cross corner points are the positions of the saddle points on the surface. The characteristic of the saddle point in the surface equation is that the first eigenvalue λ1 of the Hessian matrix (Hessian matrix) obtained with this point as the center is positive, and the second eigenvalue λ2 is negative.

[0040] The Hessian matrix can be expressed as:

[0041]

[0042] where r xx , r xy , r yy are the second-order derivatives of the two-dimensional equation at this point.

[0043] Therefore, find all the pixel points in the image that satisfy λ1 > 0, λ2 < 0 and λ1 * λ2 is locally minimum, which are all the cross corner points in the image.

[0044] In step S202, based on the position data of the positioning corner points, find all the eight-neighborhood connected regions in the grayscale image, and traverse all the eight-neighborhood connected regions in the grayscale image to find all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points.

[0045] In some alternative embodiments, when finding all the eight-neighborhood connected regions in the grayscale image based on the position data of the positioning corner points, perform binarization processing on the grayscale image to obtain the corresponding binary image of the grayscale image, and assign pixel values to the pixel points in the peripheral region centered on the position data of the positioning corner points in the binary image to obtain the assigned binary image; in response to determining that the pixel values of a group of pixel points in the assigned binary image are the same, and the pixel points in the group of pixel points satisfy the eight-neighborhood definition, determine that the region formed by connecting the pixel points in the group of pixel points is the eight-neighborhood connected region. Thereby, all the eight-neighborhood connected regions in the grayscale image can be accurately found.

[0046] In a specific example, image binarization is to set the pixel values higher than a certain brightness threshold to 255 and those lower than the threshold to 0 according to the local brightness of the image. Usually, the Otsu algorithm is used to obtain the binarized image. In the binarized image, the 3×3 area around the pixels corresponding to the located corner positions is assigned the value 127. In the binary image, find all the eight-neighbor connected regions with values of 0 and 255, and skip the pixels with the value 127. Definition of the eight-neighbor: If a pixel p(u p , v p ) is within the eight-neighbor of a pixel q(u q , v q ), then max(|u p - u q |, |v p - v q |) ≤ 1. The eight-neighbor connected region means that a group of pixels in the image with the same pixel value and connected through the eight-neighbor definition is a connected region. After finding all the connected regions, assign a label value to each of them. The label values are positive integers, ensuring that the label values of each connected region are different.

[0047] In step S203, determine all the eight-neighbor connected regions adjacent to the four adjacent located corner points as the candidate regions of the two-dimensional code, and based on the position data of the four adjacent located corner points and the position data of the four vertices of the preset unit square two-dimensional code, determine the projective transformation data.

[0048] In some alternative embodiments, when determining the projective transformation data based on the position data of the four adjacent located corner points and the position data of the four vertices of the preset unit square two-dimensional code, establish the correspondence between the four adjacent located corner points and the four vertices of the unit square two-dimensional code, and based on this correspondence, substitute the coordinates of the four adjacent located corner points and the coordinates of the four vertices of the unit square two-dimensional code into the projective transformation equation to solve for the projective transformation matrix for projecting the candidate region to the region where the corresponding unit square two-dimensional code of the candidate region is located. Thereby, the projective transformation matrix for projecting the candidate region to the region where the corresponding unit square two-dimensional code of the candidate region is located can be accurately determined.

[0049] In a specific example, traverse all the connected regions to find all the connected regions adjacent to the four adjacent corner points. These connected regions are the candidate regions of the two-dimensional code. Arrange the four cross corner points in a clockwise direction, which can correspond one-to-one with Figure 1E the four vertices of the unit (with side length of one) square two-dimensional code in c .

[0050] In step S204, based on the projection transformation data, perform a projection transformation on the candidate region to obtain the region where the unit square two-dimensional code corresponding to the candidate region is located, and based on the region where the unit square two-dimensional code corresponding to the candidate region is located, identify the conical and cylindrical marker in the grayscale image.

[0051] In some alternative embodiments, when identifying the conical and cylindrical marker in the grayscale image based on the region where the unit square two-dimensional code corresponding to the candidate region is located, determine whether the color of the small square in the recognition region in the region is the same as the color of the small square in the background region in the region in the order of the small squares in the recognition region from left to right and from top to bottom; based on the similarities and differences between the color of the small square in the recognition region and the color of the small square in the background region, decode the small square in the recognition region to obtain the code corresponding to the unit square two-dimensional code corresponding to the candidate region; in response to determining that the code corresponding to the unit square two-dimensional code corresponding to the candidate region is the same as the code corresponding to the unit square two-dimensional code on the conical and cylindrical marker entity, determine the position data of the conical and cylindrical marker in the grayscale image as the position data of the conical and cylindrical marker entity. Thereby, the position data of the conical and cylindrical marker in the grayscale image can be accurately identified.

[0052] In a specific example, within the 6×6 recognition region in the region where the unit square two-dimensional code is located after the projection transformation corresponding to the two-dimensional code candidate region, perform decoding and compare it with the code of the two-dimensional code on the conical and cylindrical marker entity. If they are the same, determine the position data of the conical and cylindrical marker in the grayscale image as the position data of the conical and cylindrical marker entity. If they are not the same, determine that the position data of the conical and cylindrical marker in the grayscale image is not the position data of the conical and cylindrical marker entity. c In the 6×6 recognition region within the region where the unit square two-dimensional code is located after the projection transformation corresponding to the two-dimensional code candidate region, perform decoding and compare it with the code of the two-dimensional code on the conical and cylindrical marker entity. If they are the same, determine the position data of the conical and cylindrical marker in the grayscale image as the position data of the conical and cylindrical marker entity. If they are not the same, determine that the position data of the conical and cylindrical marker in the grayscale image is not the position data of the conical and cylindrical marker entity.

[0053] Based on the first embodiment, by using the position data of the positioning corner points of the two-dimensional code on the conical and cylindrical marker in the grayscale image, find all the eight-neighborhood connected regions in the grayscale image, and find all the eight-neighborhood connected regions adjacent to the four adjacent positioning corner points. Then perform a projection transformation on all the eight-neighborhood connected regions adjacent to the four adjacent positioning corner points to obtain the corresponding region where the unit square two-dimensional code is located. Then, based on the region where the unit square two-dimensional code corresponding to the candidate region is located, identify the conical and cylindrical marker in the grayscale image, and the conical and cylindrical marker in the grayscale image can be accurately identified.

[0054] The recognition method of the conical-cylindrical surface type marker provided in this embodiment can be executed by any suitable device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment devices, advertising devices, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices, etc.

[0055] Referring to Figure 3 , a schematic structural diagram of the recognition device of the conical-cylindrical surface type marker provided in the third embodiment of this application is shown.

[0056] The recognition device of the conical-cylindrical surface type marker provided in this embodiment is applied to surgical navigation. The device includes: a first determination module 301, configured to determine the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical surface type marker in the grayscale image based on the surface equation corresponding to the grayscale image to be recognized and the position data of the pixel points in the grayscale image, where the conical-cylindrical surface type marker is the conical-cylindrical surface type marker according to the first embodiment of this application; a traversal module 302, configured to find all the eight-neighborhood connected regions of the grayscale image based on the position data of the positioning corner points, and traverse all the eight-neighborhood connected regions of the grayscale image to find all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points; a second determination module 303, configured to determine that all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points are candidate regions of the two-dimensional code, and determine projection transformation data based on the position data of the four adjacent positioning corner points and the position data of the four vertices of a preset unit square two-dimensional code; an identification module 304, configured to perform a projection transformation on the candidate region based on the projection transformation data to obtain the region where the unit square two-dimensional code corresponding to the candidate region is located, and identify the conical-cylindrical surface type marker in the grayscale image based on the region where the unit square two-dimensional code corresponding to the candidate region is located.

[0057] Optionally, the first determination module 301 is specifically configured to: solve the Hessian matrix centered on the pixel point in the grayscale image based on the surface equation and the position data of the pixel points in the grayscale image; in response to determining that the first eigenvalue of the Hessian matrix is positive, the second eigenvalue of the Hessian matrix is negative, and the product of the first eigenvalue and the second eigenvalue is the minimum in the local region centered on the pixel point in the grayscale image, determine that the pixel point in the grayscale image is a saddle point of the surface equation; determine the position data of the saddle point of the surface equation as the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical surface type marker in the grayscale image.

[0058] Optionally, the traversal module 302 is specifically configured to: binarize the grayscale image to obtain a binary image corresponding to the grayscale image, and assign pixel values to the pixel points in the peripheral area centered on the position data of the positioning corner points in the binary image to obtain an assigned binary image; in response to determining that the pixel values of a group of pixel points in the assigned binary image are the same, and the pixel points in the group of pixel points satisfy the eight-neighborhood definition, determine that the area formed by connecting the pixel points in the group of pixel points is the eight-neighborhood connected area.

[0059] Optionally, the second determination module 303 is specifically configured to: establish a correspondence relationship between the four adjacent positioning corner points and the four vertices of the unit square two-dimensional code, and based on the correspondence relationship, substitute the coordinates of the four adjacent positioning corner points and the coordinates of the four vertices of the unit square two-dimensional code into the projective transformation equation to solve for the projective transformation matrix for projecting and transforming the candidate area into the area where the corresponding unit square two-dimensional code of the candidate area is located.

[0060] Optionally, the recognition module 304 is specifically configured to: determine whether the color of the small squares in the recognition area of the area is the same as the color of the small squares in the background area of the area in the order of the small squares in the recognition area of the area from left to right and from top to bottom; decode the small squares in the recognition area based on the similarities and differences between the color of the small squares in the recognition area and the color of the small squares in the background area to obtain the code corresponding to the unit square two-dimensional code corresponding to the candidate area; in response to determining that the code corresponding to the unit square two-dimensional code corresponding to the candidate area is the same as the code corresponding to the unit square two-dimensional code on the conical-cylindrical marker entity, determine that the position data of the conical-cylindrical marker in the grayscale image is the position data of the conical-cylindrical marker entity.

[0061] The recognition device for the conical-cylindrical marker provided in this embodiment is used to implement the corresponding conical-cylindrical marker recognition method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0062] Figure 4 It is a schematic structural diagram of the electronic device in the fourth embodiment; the electronic device may include:

[0063] One or more processors 401;

[0064] A computer-readable medium 402, which can be configured to store one or more programs,

[0065] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for identifying a conical-cylindrical marker as described in the second embodiment above.

[0066] Figure 5 This is the hardware structure of the electronic device in the fifth embodiment; as Figure 5 shown, the hardware structure of the electronic device may include: a processor 501, a communication interface 502, a computer-readable medium 503, and a communication bus 504;

[0067] Among them, the processor 501, the communication interface 502, and the computer-readable medium 503 complete mutual communication through the communication bus 504;

[0068] Optionally, the communication interface 502 may be an interface of a communication module, such as an interface of a GSM module;

[0069] Among them, the processor 501 may be specifically configured to: based on the surface equation corresponding to the grayscale image to be recognized and the position data of the pixel points in the grayscale image, determine the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical marker in the grayscale image, where the conical-cylindrical marker is the conical-cylindrical marker described in the first embodiment of this application; based on the position data of the positioning corner points, find all the eight-neighborhood connected regions in the grayscale image, and traverse all the eight-neighborhood connected regions in the grayscale image to find all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points; determine all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points as the candidate regions of the two-dimensional code, and based on the position data of the four adjacent positioning corner points and the position data of the four vertices of a preset unit square two-dimensional code, determine the projective transformation data; based on the projective transformation data, perform projective transformation on the candidate regions to obtain the region where the unit square two-dimensional code corresponding to the candidate regions is located, and based on the region where the unit square two-dimensional code corresponding to the candidate regions is located, identify the conical-cylindrical marker in the grayscale image.

[0070] The processor 501 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0071] The computer-readable medium 503 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0072] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code configured to execute the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the methods of the present application are performed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable medium can, for example, but is not limited to, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. And in the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program configured to be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0073] Computer program code configured to perform the operations of the present application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions configured to implement a specified logical function. There are specific sequential relationships in the above specific embodiments, but these sequential relationships are only exemplary. In specific implementations, these steps may be fewer, more, or the execution order may be adjusted. That is, in some alternative implementations, the functions marked in the blocks may also occur in a different order than that marked in the accompanying drawings. For example, two consecutively represented blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0075] The modules involved in the embodiments described in the present application can be implemented in software or in hardware. The described modules can also be provided in a processor. For example, it can be described as: a processor includes a first determination module, a traversal module, a second determination module, and an identification module. Among them, the names of these modules do not constitute a limitation to the module itself in some cases. For example, the first determination module can also be described as "a module that determines the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical marker in the grayscale image based on the surface equation corresponding to the grayscale image to be recognized and the position data of the pixel points in the grayscale image".

[0076] As another aspect, the present application also provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for identifying the conical-cylindrical marker as described in the second embodiment above.

[0077] As another aspect, the present application also provides a computer-readable medium. The computer-readable medium may be included in the device described in the above embodiment; or it may exist alone without being assembled into the device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by the device, the device: based on the surface equation corresponding to the grayscale image to be identified and the position data of the pixel points in the grayscale image, determines the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical marker in the grayscale image, where the conical-cylindrical marker is the conical-cylindrical marker described in the first embodiment of the present application; based on the position data of the positioning corner points, finds all the eight-neighborhood connected regions in the grayscale image, and traverses all the eight-neighborhood connected regions in the grayscale image to find all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points; determines all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points as the candidate regions of the two-dimensional code, and based on the position data of the four adjacent positioning corner points and the position data of the four vertices of the preset unit square two-dimensional code, determines the projection transformation data; based on the projection transformation data, performs a projection transformation on the candidate regions to obtain the region where the unit square two-dimensional code corresponding to the candidate regions is located, and based on the region where the unit square two-dimensional code corresponding to the candidate regions is located, identifies the conical-cylindrical marker in the grayscale image.

[0078] In the various embodiments of the present disclosure, the expressions "first", "second", "the first" or "the second" used may modify various components without regard to order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, the first user device and the second user device represent different user devices, although both are user devices. For example, without departing from the scope of the present disclosure, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element.

[0079] When an element (e.g., a first element) is referred to as being “(operatively or communicatively) coupled” or “(operatively or communicatively) coupled to” or “connected to” another element (e.g., a second element), it should be understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it can be understood that when an element (e.g., a first element) is referred to as being “directly connected” or “directly coupled” to another element (a second element), no element (e.g., a third element) is inserted therebetween.

[0080] The above description is only the preferred embodiments of the present application and the illustration of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) disclosed in the present application that have similar functions.

Claims

1. A conical-cylindrical marker, characterized in that, Applied to surgical navigation, the conical-cylindrical marker includes: A cylindrical part, on the cylindrical surface of which multiple layers of two-dimensional codes are printed, and each layer of the two-dimensional codes in the multiple layers of two-dimensional codes is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of each two-dimensional code in the multiple layers of two-dimensional codes form positioning corner points with a centrosymmetric pattern; A conical part, the bottom surface of which is attached to the top surface of the cylindrical part, and a two-dimensional code is printed on the conical surface of the conical part. The two-dimensional code printed on the conical surface is composed of black two-dimensional codes and white two-dimensional codes arranged at intervals, so that the corner points of the two-dimensional code printed on the conical surface form positioning corner points with a centrosymmetric pattern.

2. The conical-cylindrical marker according to claim 1, wherein Each two-dimensional code in the multiple layers of two-dimensional codes has an identification area and a background area located outside the identification area, and the identification area is composed of small squares of different colors, and the background area is composed of small squares of the same color.

3. The conical-cylindrical marker according to claim 2, wherein According to the order of the small squares in the identification area from left to right and from top to bottom, determine whether the colors of the small squares in the identification area are the same as the colors of the small squares in the background area, and encode the small squares in the identification area according to the similarities and differences between the colors of the small squares in the identification area and the colors of the small squares in the background area, so as to obtain the code corresponding to the two-dimensional code where the identification area is located.

4. The conical-cylindrical marker according to claim 3, wherein The codes corresponding to each two-dimensional code in the multiple layers of two-dimensional codes are different, and the codes corresponding to each two-dimensional code in the multiple layers of two-dimensional codes after being rotated by a preset degree are also different.

5. The conical-cylindrical marker according to claim 1, wherein The shape of the conical surface is an annulus obtained by subtracting two sectors with different radii and the same center of the circle, and each contour line of the two-dimensional code printed on the conical surface along the radial direction intersects at the center of the circle of the sector, and the included angle between each two adjacent contour lines is equal.

6. The conical-cylindrical marker according to claim 5, characterized in that, The shape of the two-dimensional code printed on the conical surface is trapezoidal, and the spacing of the contour lines of the two-dimensional code printed on the conical surface in the radial direction along the circumferential direction is not equal.

7. The conical-cylindrical marker according to claim 6, characterized in that, The trapezoidal two-dimensional code printed on the conical surface is obtained by a projection transformation of a unit square two-dimensional code, and the interval is determined by the unit square two-dimensional code and the projection transformation.

8. A recognition method for the conical columnar marker according to any one of claims 1 to 7, characterized in that, Applied to surgical navigation, the method includes: Based on the surface equation corresponding to the grayscale image to be recognized and the position data of the pixel points in the grayscale image, determine the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical marker in the grayscale image; Based on the position data of the positioning corner points, find all the eight-neighborhood connected regions in the grayscale image, and traverse all the eight-neighborhood connected regions in the grayscale image to find all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points; Determine that all the eight-neighborhood connected regions adjacent to four adjacent positioning corner points are candidate regions of the two-dimensional code, and based on the position data of the four adjacent positioning corner points and the position data of the four vertices of a preset unit square two-dimensional code, determine the projection transformation data; Based on the projection transformation data, perform a projection transformation on the candidate region to obtain the region where the unit square two-dimensional code corresponding to the candidate region is located, and based on the region where the unit square two-dimensional code corresponding to the candidate region is located, identify the conical-cylindrical marker in the grayscale image.

9. The recognition method of the conical cylinder surface type marker according to claim 8, characterized in that, The one based on the to-be-identified gray-scale image's corresponding surface equation and the position data of the pixel points in the gray-scale image to determine the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical marker in the gray-scale image includes: Based on the surface equation and the position data of the pixel points in the gray-scale image, solve the Hessian matrix centered on the pixel points in the gray-scale image; In response to determining that the first eigenvalue of the Hessian matrix is positive, the second eigenvalue of the Hessian matrix is negative, and the product of the first eigenvalue and the second eigenvalue is the minimum in the local region centered on the pixel points in the gray-scale image, determine that the pixel points in the gray-scale image are the saddle points of the surface equation; Determine the position data of the saddle points of the surface equation as the position data of the positioning corner points of the two-dimensional code on the conical-cylindrical marker in the gray-scale image.

10. The recognition method of the conical-cylindrical surface type marker according to claim 8, characterized in that, The one based on the position data of the positioning corner points to find all the eight-neighborhood connected regions in the gray-scale image includes: Perform a binarization process on the gray-scale image to obtain the binarized image corresponding to the gray-scale image, and assign pixel values to the pixel points in the peripheral region centered on the position data of the positioning corner points in the binarized image to obtain the binarized image after assignment; In response to determining that the pixel values of a group of pixel points in the binarized image after assignment are the same, and the pixel points in the group of pixel points satisfy the eight-neighborhood definition, determine that the region formed by connecting the pixel points in the group of pixel points is the eight-neighborhood connected region.

11. The identification method of the conical cylinder surface type marker according to claim 8, characterized in that, The one based on the position data of the four adjacent positioning corner points and the position data of the four vertices of the preset unit square two-dimensional code to determine the projection transformation data includes: Establish a correspondence relationship between the four adjacent positioning corner points and the four vertices of the unit square two-dimensional code, and based on the correspondence relationship, substitute the coordinates of the four adjacent positioning corner points and the coordinates of the four vertices of the unit square two-dimensional code into the projection transformation equation to solve the projection transformation matrix for projecting the candidate region into the region where the unit square two-dimensional code corresponding to the candidate region is located.

12. The recognition method of the cone-columnar marker according to claim 8, characterized in that, The one based on the region where the unit square two-dimensional code corresponding to the candidate region is located to identify the conical-cylindrical marker in the gray-scale image includes: Determine whether the color of the small squares in the recognition region is the same as the color of the small squares in the background region in the region in the order of the small squares in the recognition region from left to right and from top to bottom; Based on the similarities and differences between the color of the small squares in the recognition region and the color of the small squares in the background region, decode the small squares in the recognition region to obtain the code corresponding to the unit square two-dimensional code corresponding to the candidate region. In response to determining that the code corresponding to the unit square two-dimensional code corresponding to the candidate region is the same as the code corresponding to the unit square two-dimensional code on the conical cylinder-shaped marker entity, determine the position data of the conical cylinder-shaped marker in the grayscale image as the position data of the conical cylinder-shaped marker entity.

Images