Sequential recognition method of marker points, surgical robot system and storage medium

Abstract

The present application relates to a method for sequentially identifying marker points, a surgical robot system, and a storage medium. The method for sequentially identifying marker points includes: acquiring a two-dimensional image obtained by shooting a calibration target and a target object; detecting marker points in the two-dimensional image to obtain a plurality of marker point sets; for any marker point set, constructing a coordinate system corresponding to any marker point set based on any marker point set, mapping any marker point set to the coordinate system to obtain a mapping point set corresponding to any marker point set; determining a reconstruction point set corresponding to any marker point set based on the coordinate system and mapping point set corresponding to any marker point set; determining a plurality of target marker points corresponding to any marker point set, and a label for each target marker point based on the reconstruction point set corresponding to any marker point set. This method can reconstruct undetected marker points, eliminate marker points outside the boundary of the two-dimensional image, and then detect all marker points within the two-dimensional image, thereby improving the accuracy of sequential marker point detection.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a method for sequentially recognizing marker points, a surgical robot system, and a storage medium. Background Art

[0002] C-arm X-ray machines are widely used in orthopedic surgical robot systems to obtain real-time X-ray images of the hip joint during surgery. In order to achieve accurate real-time navigation during surgery, it is necessary to Figure 1 The calibration target shown is exposed to the imaging field of view of the C-arm X-ray machine along with the patient. This is used to obtain an X-ray image containing the calibration target and the marker balls, which is then used for camera calibration. The calibration process requires sorting the coordinates of the marker points in the X-ray image, so the X-ray image-based marker point sequence recognition algorithm is crucial for the entire orthopedic intraoperative navigation system. However, during the intraoperative X-ray imaging process, since the calibration target is in an arbitrary position within the imaging field of view of the C-arm X-ray machine, there are often complex situations such as marker points being blocked by bone tissue or located outside the boundary, resulting in inaccurate calibration. Summary of the Invention

[0003] Based on this, it is necessary to provide a sequential recognition method for marker points, a surgical robot system and a storage medium to address the above technical problems, which can reconstruct occluded marker points and eliminate marker points located outside the image boundary.

[0004] In a first aspect, the present application provides a method for sequentially identifying markers. The method comprises:

[0005] Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0006] Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image;

[0007] For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set;

[0008] Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points;

[0009] Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

[0010] In one embodiment, the any one marking point set includes a plurality of marking points; and constructing a coordinate system corresponding to the any one marking point set based on the any one marking point set includes:

[0011] Determine the nearest neighboring marker point of each marker point in any marker point set, and select a coordinate system origin and a plurality of reference points in any marker point set based on the nearest neighboring marker point of each marker point;

[0012] A coordinate system corresponding to any one of the marking point sets is constructed according to a coordinate system origin and a plurality of reference points selected from the any one of the marking point sets.

[0013] In one embodiment, the mapping point set corresponding to any one of the marking point sets includes at least: a mapping origin and a plurality of mapping reference points, wherein the mapping origin is a point obtained by mapping the origin of the coordinate system to the coordinate system, and the mapping reference points are points obtained by mapping the reference points to the coordinate system; determining the reconstructed point set corresponding to any one of the marking point sets based on the coordinate system and the mapping point set corresponding to the any one of the marking point sets includes:

[0014] Determining a pseudo point in the mapping point set based on the coordinate system corresponding to any one of the marking point sets and the mapping point set;

[0015] Eliminating pseudo points and the plurality of mapping reference points from the mapping point set to obtain a candidate point set;

[0016] A reconstruction point set corresponding to any one of the marked point sets is determined based on the candidate point set.

[0017] In one embodiment, determining the pseudo points in the mapping point set based on the coordinate system and the mapping point set corresponding to any one of the marking point sets includes:

[0018] For any mapping point in the mapping point set corresponding to any marking point set, multiple lines are determined based on the mapping point set. If there are no two lines in the multiple lines that are respectively parallel to the two direction axes of the coordinate system, then the any mapping point is determined to be a pseudo point, wherein each line includes the any mapping point.

[0019] In one embodiment, determining the reconstruction point set corresponding to any one of the marked point sets based on the candidate point set includes:

[0020] Performing clustering processing on the candidate point set to obtain a row direction step length and a column direction step length;

[0021] Reconstructing a reference point set in the coordinate system corresponding to any one of the marking point sets according to the mapping origin, the row direction step size, and the column direction step size;

[0022] The reference point set is converted into the image coordinate system of the two-dimensional image to obtain a reconstruction point set corresponding to any one of the marking point sets.

[0023] In one embodiment, determining a plurality of target marking points corresponding to any marking point set and a label of each target marking point based on the reconstructed point set corresponding to any marking point set includes:

[0024] Labeling the reconstruction point set corresponding to any one of the marking point sets according to a preset order corresponding to the any one of the marking point sets to obtain a plurality of labeled reconstruction points corresponding to the any one of the marking point sets;

[0025] Among the plurality of labeled reconstructed points, a plurality of target marking points in the two-dimensional image are selected.

[0026] In one embodiment, the multiple calibration points include multiple first preset calibration points and multiple second preset calibration points, the calibration target includes a first target surface and a second target surface parallel to each other, the first target surface is provided with multiple first preset calibration points, the second target surface is provided with multiple second preset calibration points, the radius of the first preset calibration point is different from the radius of the second preset calibration point, the multiple first preset calibration points include a first preset origin and multiple first preset reference points, and the multiple second preset calibration points include a second preset origin and multiple second preset reference points.

[0027] In one embodiment, detecting the marker points in the two-dimensional image to obtain a plurality of marker point sets includes:

[0028] Determining a segmentation mask of the two-dimensional image, and detecting marker points based on the segmentation mask to obtain an initial marker point set;

[0029] The initial mark point set is divided into a first mark point set and a second mark point set based on a preset radius interval, wherein a radius of a first mark point in the first mark point set is different from a radius of a second mark point in the second mark point set.

[0030] In a second aspect, the present application further provides a surgical robot system. The system comprises:

[0031] 2D image acquisition equipment and processor;

[0032] The two-dimensional image acquisition device is used to acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0033] The processor is configured to detect marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image; for any marker point set, construct a coordinate system corresponding to the any marker point set based on the any marker point set, map the any marker point set to the coordinate system, and obtain a mapping point set corresponding to the any marker point set; determine a reconstruction point set corresponding to the any marker point set based on the coordinate system and mapping point set corresponding to the any marker point set, wherein the reconstruction point set includes undetected marker points; determine a plurality of target marker points corresponding to the any marker point set and a label of each target marker point based on the reconstruction point set corresponding to the any marker point set, wherein the plurality of target marker points are all within the two-dimensional image.

[0034] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0035] Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0036] Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image;

[0037] For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set;

[0038] Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points;

[0039] Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0041] Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0042] Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image;

[0043] For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set;

[0044] Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points;

[0045] Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

[0046] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0047] Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0048] Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image;

[0049] For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set;

[0050] Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points;

[0051] Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

[0052] In the above-mentioned method for sequential recognition of marker points, the calibration target includes multiple target surfaces, each target surface includes multiple calibration points, a two-dimensional image obtained by shooting the calibration target and the target object is obtained, the marker points in the two-dimensional image are detected, and multiple marker point sets are obtained, and the multiple marker point sets correspond one-to-one to the multiple target surfaces. The coordinate system of the multiple calibration points on the target surface is determined according to the marker point set corresponding to the target surface, and the marker point set is mapped to the coordinate system to obtain a mapping point set. According to the coordinate values ​​of the multiple mapping points in the mapping point set and the coordinate system, a reconstructed point set in the image coordinate system is reconstructed, and the multiple reconstruction points included in the reconstructed point set correspond one-to-one to the multiple calibration points on the target surface; the reconstructed point set includes undetected marker points, and all target marker points in the two-dimensional image and the order of each target marker point are determined according to the reconstruction point set; according to the above-mentioned method for sequential recognition of marker points, the undetected marker points can be reconstructed, and the marker points outside the boundary of the two-dimensional image can be eliminated, and then all marker points in the two-dimensional image can be detected, thereby improving the accuracy of marker point sequential detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A diagram illustrating an application environment of a method for sequentially identifying marker points in one embodiment;

[0054] Figure 2 1 is a flow chart of a method for sequentially identifying marker points in one embodiment;

[0055] Figure 3 is a schematic diagram of a calibration target in one embodiment;

[0056] Figure 4 A schematic diagram of a plurality of first preset calibration points on a first target surface in one embodiment;

[0057] Figure 5 A schematic diagram of a plurality of second preset calibration points on a second target surface in one embodiment;

[0058] Figure 6 is a schematic diagram of a two-dimensional image acquired by a processor in one embodiment;

[0059] Figure 7 for Figure 6 The segmentation mask of

[0060] Figure 8 is a schematic diagram of a first coordinate system in one embodiment;

[0061] Figure 9 is a schematic diagram of a second coordinate system in one embodiment;

[0062] Figure 10 A schematic diagram of marker points including pseudo points detected based on a two-dimensional image according to an embodiment;

[0063] Figure 11In one embodiment, according to the sequential recognition method of the marking points, the Figure 10 A schematic diagram showing all target marking points in the two-dimensional image and the label of each target marking point;

[0064] Figure 12 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0066] The method for sequentially identifying marking points provided in the embodiment of the present application can be applied to Figure 1 The surgical robot system shown is used for real-time navigation during surgery. A camera calibration algorithm is required during real-time surgical navigation, and the accuracy of the camera calibration algorithm affects the accuracy of real-time surgical navigation. The surgical robot system includes: a two-dimensional imaging device 102 and a processor 104. A target object is positioned on an operating table. The position of the two-dimensional imaging device 102 is adjusted so that the two-dimensional imaging device 102 can capture a lesion on the target object. A calibration target is placed between the camera of the two-dimensional imaging device 102 and the target object. The two-dimensional imaging device 102 captures a two-dimensional image containing the calibration target and the lesion on the target object. The processor 104 captures the two-dimensional image, determines all target markers in the two-dimensional image, and labels each target marker. The resulting labeled target markers are used for camera calibration during real-time surgical navigation. The processor 104 can be a personal computer, a laptop computer, a smartphone, a tablet computer, or the like.

[0067] In one embodiment, Figure 2 As shown in FIG, a sequential recognition method of marker points is provided, which is applied to Figure 1 The following steps are taken as an example to illustrate the processor in the example:

[0068] S101 , obtaining a two-dimensional image obtained by photographing a calibration target and a target object.

[0069] The calibration target includes multiple target surfaces, which include at least two target surfaces. Through holes are opened on the target surfaces, and the through holes are used to install multiple calibration points. The calibration points can be metal marking balls. The target object is a human body. The two-dimensional image is obtained by shooting the calibration target and the target object with a two-dimensional imaging device. The two-dimensional imaging device can be a C-arm X-ray machine, direct digital radiography (DDR), etc.

[0070] Specifically, when a calibration target is located between a target object and a camera of a 2D imaging device, the 2D imaging device captures the calibration target and the target object to obtain a 2D image. The calibration target can be placed between the target object and the camera of the 2D imaging device in any desired position, provided that the target surface of the calibration target is parallel to the imaging surface of the 2D image. The processor can retrieve the 2D image from a memory of the 2D imaging device or in real time.

[0071] S102: Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets.

[0072] Specifically, the marking point is a point obtained by imaging the calibration point on the calibration target in the two-dimensional image; the processor determines the segmentation mask of the two-dimensional image, uses the existing circle detection algorithm to detect the segmentation mask, obtains multiple marking points, and divides the multiple marking points into multiple marking point sets.

[0073] The multiple marker point sets correspond one-to-one to the multiple target surfaces of the calibration target. The radius of the calibration points on different target surfaces is different, and thus the radius of the marker points obtained by imaging the calibration points on different target surfaces in the two-dimensional image is also different. The multiple marker points detected can be divided into multiple marker point sets based on the radius of the multiple marker points. The multiple marker point sets include at least two marker point sets.

[0074] Since the calibration target is placed in an arbitrary posture between the target object and the camera of the two-dimensional imaging device, some calibration points may overlap with human tissues and medical devices in the body and cannot be imaged in the two-dimensional image. Some calibration points may not be within the field of view of the two-dimensional imaging device and cannot be imaged in the two-dimensional image. Some calibration points may have been imaged in the two-dimensional image but failed to be detected. The above points that cannot be imaged in the two-dimensional image, or have been imaged but failed to be detected, can be regarded as undetected marking points in the two-dimensional image. The following steps are used to reconstruct all marking points that cannot be detected in the two-dimensional image, and determine multiple target marking points in the two-dimensional image and the label of each target marking point.

[0075] S103 . For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set.

[0076] Specifically, the processor determines the coordinate system origin and multiple reference points used to construct the coordinate system in any marking point set, constructs the coordinate system corresponding to any marking point set based on the determined coordinate system origin and multiple reference points, determines the mapping matrix between the coordinate system and the image coordinate system of the two-dimensional image, and maps any marking point set to the coordinate system according to the determined mapping matrix to obtain the mapping point set corresponding to any marking point.

[0077] S104: Determine a reconstruction point set corresponding to any one of the marked point sets based on the coordinate system and the mapping point set corresponding to any one of the marked point sets.

[0078] Specifically, mapping points corresponding to multiple reference points are eliminated from the mapping point set. Based on the multiple remaining mapping points in the mapping point set, the row distance between two adjacent remaining mapping points in the row direction is obtained to obtain multiple row distances. The column distance between two adjacent remaining mapping points in the column direction is obtained to obtain multiple column distances. The row direction step size is determined based on the multiple row distances. The column direction step size is determined based on the multiple column distances. Based on the coordinate system origin, the row direction step size, and the column direction step size, the reconstruction point set corresponding to any marked point set in the image coordinate system is determined. The reconstruction point set corresponding to any marked point set includes multiple reconstruction points, which correspond one-to-one to the multiple calibration points on the target surface corresponding to the any marked point set.

[0079] The reconstructed point set includes undetected marker points, that is, marker points that cannot be imaged in the two-dimensional image because the calibration points overlap with human tissues and medical devices in the body, or the calibration points are not within the field of view of the two-dimensional imaging device, and marker points that have been imaged in the two-dimensional image but cannot be detected.

[0080] S105 : Based on the reconstructed point set corresponding to any one of the marker point sets, determine a plurality of target marker points corresponding to the any one of the marker point sets, and a label of each target marker point.

[0081] The multiple target marking points are all in the two-dimensional image, and the labels of the target marking points are used to reflect the order of the target marking points.

[0082] Specifically, the reconstructed point set corresponding to any marker point set is numbered according to the preset order corresponding to that set. Reconstructed points within the 2D image are selected from the numbered reconstructed point set and used as target marker points. Because the reconstructed point set is numbered first, and then the reconstructed points within the 2D image are used as target marker points, the numbers of the target marker points are determined according to the preset order of the marker point set and are not changed by other reconstructed points outside the bounds of the 2D image.

[0083] In the above-mentioned method for sequential recognition of marker points, the calibration target includes multiple target surfaces, each target surface includes multiple calibration points, a two-dimensional image obtained by shooting the calibration target and the target object is obtained, the marker points in the two-dimensional image are detected, and multiple marker point sets are obtained, and the multiple marker point sets correspond one-to-one to the multiple target surfaces. The coordinate system of the multiple calibration points on the target surface is determined according to the marker point set corresponding to the target surface, and the marker point set is mapped to the coordinate system to obtain a mapping point set. According to the coordinate values ​​of the multiple mapping points in the mapping point set and the coordinate system, a reconstructed point set in the image coordinate system is reconstructed, and the multiple reconstruction points included in the reconstructed point set correspond one-to-one to the multiple calibration points on the target surface; the reconstructed point set includes undetected marker points, and all target marker points in the two-dimensional image and the order of each target marker point are determined according to the reconstruction point set; according to the above-mentioned method for sequential recognition of marker points, the undetected marker points can be reconstructed, and the marker points outside the boundary of the two-dimensional image can be eliminated, and then all marker points in the two-dimensional image can be detected, thereby improving the accuracy of marker point sequential detection.

[0084] In one embodiment, for example, Figure 3 As shown, the calibration target includes a first target surface B1 and a second target surface B2, and the multiple calibration points include multiple first preset calibration points and multiple second preset calibration points. Multiple first preset calibration points are set on the first target surface B1, and multiple second preset calibration points are set on the second target surface B2. The radius of the first preset calibration point is different from the radius of the second preset calibration point. The multiple first preset calibration points include a first preset origin and multiple first preset reference points, and the multiple second preset calibration points include a second preset origin and multiple second preset reference points.

[0085] The first preset origin and multiple first preset reference points are used to construct a first coordinate system, and the second preset origin and multiple second preset reference points are used to construct a second coordinate system.

[0086] For the sake of convenience, the calibration points among the multiple first preset calibration points except the multiple first preset reference points are recorded as first common calibration points (including the first preset origin), and the calibration points among the multiple second preset calibration points except the multiple second preset reference points are recorded as second common calibration points (including the second preset origin).

[0087] Multiple first common calibration points are arranged on the first target surface according to the preset first row distance and first column distance, and are distributed in a matrix, and the first row distance and the first column distance can be the same; multiple second common calibration points are arranged on the second target surface according to the preset second row distance and second column distance, and are distributed in a matrix, and the second row distance and the second column distance can be the same, the second row distance and the first row distance can also be the same, and the second column distance and the first column distance can also be the same.

[0088] A number of first preset reference points are respectively set on the first axial direction and the second axial direction of the first preset origin, and the first axial direction and the second axial direction are orthogonal. The number of the first preset reference points set on the first axial direction can be set to be greater than the number of the first preset reference points set on the second axial direction; the first preset reference points set on the first axial direction of the first preset origin are between the first preset origin and the first common calibration point adjacent to the first preset origin on the first axial direction, that is, the distance between any two first preset reference points on the first axial direction is less than the distance between any two adjacent first common calibration points (first row distance or first column distance); the first preset reference points set on the second axial direction of the first preset origin are between the first preset origin and the first common calibration point adjacent to the first preset origin on the second axial direction, that is, the distance between any two first preset reference points on the second axial direction is less than the distance between any two adjacent first common calibration points (first row distance or first column distance).

[0089] Several second preset reference points are respectively set on the third axial direction and the fourth axial direction of the second preset origin. The third axial direction and the fourth axial direction are orthogonal to each other. The number of the several second preset reference points set on the third axial direction can be set to be greater than the number of the several second preset reference points set on the fourth axial direction; the several second preset reference points set on the third axial direction of the second preset origin are located between the second preset origin and the second common calibration point adjacent to the second preset origin on the third axial direction, that is, the distance between any two second preset reference points on the third axial direction is less than the distance between any two adjacent second common calibration points (second row distance or second column distance); the several second preset reference points set on the fourth axial direction of the second preset origin are located between the second preset origin and the second common calibration point adjacent to the second preset origin on the fourth axial direction, that is, the distance between any two second preset reference points on the fourth axial direction is less than the distance between any two adjacent second common calibration points (second row distance or second column distance).

[0090] A plurality of first common calibration points on the first target surface are numbered according to a first preset sequence, and a plurality of second common calibration points on the second target surface are numbered according to a second preset sequence.

[0091] For example, Figure 4 As shown, 30 first preset calibration points are set on the first target surface ( Figure 4 ), the 30 first preset calibration points include 1 first preset origin and 5 first preset reference points; Figure 4As shown, the 25 first common calibration points on the first target surface are distributed in a matrix of 5 rows and 5 columns, and the 25 first common calibration points are numbered according to a first preset order, which is used to reflect the order of the first common calibration points numbered 1 to 25.

[0092] For example, Figure 5 As shown, 21 second preset calibration points are set on the second target surface ( Figure 5 ); the 21 first preset calibration points include 1 second preset origin and 5 second preset reference points; Figure 5 As shown, the 16 second common calibration points on the second target surface are distributed in a matrix of 4 rows and 4 columns, and the 16 second common calibration points are numbered according to a second preset order, which is used to reflect the order of the second common calibration points numbered 1 to 16.

[0093] It can be understood that the above example is only one possible way. The number of first preset calibration points on the first target surface, the number of first preset reference points, the position of the first preset origin, the position of the first preset reference point, the first preset order, the number of second preset calibration points on the second target surface, the number of second preset reference points, the position of the second preset origin, the position of the second preset reference point and the second preset order can all be adjusted according to actual needs.

[0094] In one embodiment, when the calibration target includes a first target surface and a second target surface, S102 includes:

[0095] S211 : Determine a segmentation mask for the two-dimensional image, and detect marker points based on the segmentation mask to obtain an initial marker point set.

[0096] Specifically, a two-dimensional image is obtained by projecting and superimposing three-dimensional objects onto the imaging surface based on the degree to which X-rays are absorbed by the objects when X-rays pass through human tissue, medical devices in the body, calibration targets, and other objects within the field of view of the two-dimensional imaging device. It can be seen that the imaging environment of a two-dimensional image is very complex. If the marker point is used as the foreground of the two-dimensional image, the marker point is seriously interfered by the background noise, and the grayscale difference in different areas of the two-dimensional image is large. Therefore, in this embodiment, the processor uses a local threshold segmentation method to determine the segmentation mask of the two-dimensional image to filter out the background noise in the two-dimensional image; the existing circle detection algorithm is used to detect the segmentation mask to obtain multiple marker points, and the initial marker point set includes the multiple marker points detected.

[0097] It should be noted that some calibration points may overlap with human tissues and medical devices in the body and cannot be imaged in the two-dimensional image. Some calibration points may not be within the field of view of the two-dimensional imaging device and cannot be imaged in the two-dimensional image. There may also be calibration points that have been imaged in the two-dimensional image but have not been detected. Therefore, multiple marking points are detected, which may correspond to some of the calibration points set on the calibration target, not all of the calibration points.

[0098] For example, the two-dimensional image acquired by the processor is as follows: Figure 6 As shown, the local threshold segmentation method is used to determine the segmentation mask of the two-dimensional image. The segmentation mask is as follows Figure 7 shown.

[0099] S212: Divide the initial punctuation point set into a first marked point set and a second marked point set based on a preset radius interval.

[0100] The radius of the first marking point in the first marking point set is different from the radius of the second marking point in the second marking point set.

[0101] Specifically, the radius of the calibration point (first preset calibration point) set on the first target surface of the calibration target is different from the radius of the calibration point set on the second target surface. Therefore, the radius of the mark point imaged by the calibration point set on the first target surface in the two-dimensional image is different from the radius of the mark point imaged by the calibration point (second preset calibration point) set on the second target surface in the two-dimensional image. Figure 6 The two-dimensional image shown, and Figure 7 In the segmentation masks shown, it can be seen that there are markers with different radii, for example, Figure 6 The radius of the middle marker r1 is larger than the radius of the marker r2.

[0102] The processor obtains the radius of each marking point in the initial marking point set, takes the marking point whose radius is within the preset radius interval as the first marking point, and takes the marking point whose radius is not within the preset radius interval as the second marking point; the preset radius interval can be determined based on the radius of the calibration point set on the first target surface and the radius of the calibration point set on the second target surface.

[0103] In one embodiment, in S103, constructing a coordinate system corresponding to any one of the marker point sets based on the any one of the marker point sets includes:

[0104] S311 , determining the nearest neighboring marker point of each marker point in any marker point set, and selecting a coordinate system origin and a plurality of reference points in any marker point set based on the nearest neighboring marker point of each marker point.

[0105] Specifically, when the calibration target includes a first target surface and a second target surface, any marking point set can be a first marking point set or a second marking point set; the first coordinate system origin and multiple first reference points are selected in the first marking point set, and the second coordinate system origin and multiple second reference points are selected in the second marking point set.

[0106] Determine the nearest neighboring marker point for each first marker point in the first marker point set, obtain the distance between each first marker point and its nearest neighboring marker point, and use any first marker point as a first tie point if the distance between the first marker point and its nearest neighboring marker point is within a first interval. The first interval includes a first minimum value and a first maximum value, which is less than the distance between any two adjacent first common calibration points. The first minimum value and the first maximum value can be set as needed.

[0107] Determine the line between any two adjacent first system establishment points to obtain multiple first lines, extend the multiple first lines so that the first lines on the same line are merged to obtain two orthogonal first merged lines, and use the intersection of the two first merged lines among the multiple first system establishment points as the origin of the first coordinate system; and use the first system establishment points other than the origin of the first coordinate system among the multiple first system establishment points as the first reference points.

[0108] Determine the nearest neighboring marker point for each second marker point in the second marker point set, obtain the distance between each second marker point and its nearest neighboring marker point, and use any second marker point as a second tie point if the distance between the second marker point and its nearest neighboring marker point is within a second interval. The second interval includes a second minimum value and a second maximum value, which is less than the distance between any two adjacent second common calibration points. The second minimum value and the second maximum value can be set as needed, and the second interval can be the same as the first interval.

[0109] Determine the line between any two adjacent second system establishment points to obtain multiple second lines, extend the multiple second lines so that the second lines on the same line are merged to obtain two orthogonal second merged lines, and use the intersection of two second merged lines among the multiple second system establishment points as the origin of the second coordinate system; and use the second system establishment points other than the origin of the second coordinate system among the multiple second system establishment points as the second reference points.

[0110] S312: Construct a coordinate system corresponding to any one of the marking point sets according to the coordinate system origin and multiple reference points selected from the any one of the marking point sets.

[0111] Specifically, based on the first coordinate system origin and multiple first reference points selected in the first mark point set, a first coordinate system corresponding to the first mark point set is constructed. The line between any two adjacent first reference points is determined, and the determined line is extended so that the lines on the same line are merged to obtain two orthogonal merged lines. The two directional axes of the first coordinate are determined based on the two orthogonal merged lines, wherein one directional axis passes through the first coordinate system origin and multiple first reference points on a merged line, and the other directional axis passes through the first coordinate system origin and multiple first reference points on another merged line; the first horizontal coordinate axis and the first vertical coordinate axis are determined based on the number of first reference points included in the two directional axes. The directional axis including the larger number of first reference points can be used as the first horizontal coordinate axis x1, and the directional axis including the smaller number of first reference points can be used as the first vertical coordinate axis y1. It is also possible to set the direction from the origin of the first coordinate system to the first reference point on the directional axis as the positive direction.

[0112] The number of the plurality of first preset reference points set on the first target surface is an odd number, and the number of the first preset reference points set on one side of the first preset origin is greater than the number of the first preset reference points set on the other side of the first preset origin. For example, Figure 4 As shown, the calibration point labeled 9 is the first preset origin, three first preset reference points are set on the left side of the first preset origin, and two first preset reference points are set on the lower side of the first preset origin; among the multiple first reference points corresponding to the first target surface, there are three first reference points on one merged line and two first reference points on another merged line. The direction axis passing through the three first reference points is taken as the x1 axis, and the direction axis passing through the two first reference points is taken as the y1 axis, which can be determined as follows Figure 8 The first coordinate system shown.

[0113] Similarly, a second coordinate system corresponding to the second marker point set is constructed based on the second coordinate system origin and multiple second reference points selected from the second marker point set. The second coordinate system includes a second horizontal axis x2 and a second vertical axis y2.

[0114] The number of the plurality of second preset reference points set on the second target surface is an odd number, and the number of the second preset reference points set on one side of the second preset origin is greater than the number of the second preset reference points set on the other side of the second preset origin. For example, Figure 5 As shown, the calibration point labeled 7 is the second preset origin, three second preset reference points are set on the right side of the second preset origin, and two second preset reference points are set on the upper side of the second preset origin; among the multiple second reference points corresponding to the second target surface, there are three second reference points on one merged line and two second reference points on another merged line. The direction axis passing through the three second reference points is taken as the x2 axis, and the direction axis passing through the two first reference points is taken as the y2 axis, which can be determined as follows Figure 9 The second coordinate system shown.

[0115] In one embodiment, a first mapping matrix T1 between the image coordinate system of the two-dimensional image and the first coordinate system is determined, and the first marking point set is mapped to the first coordinate system through T1 to obtain a first mapping point set. The first mapping point set includes multiple first mapping points, and the multiple first mapping points include at least a first mapping origin obtained by mapping the origin of the first coordinate system, and a first mapping reference point obtained by mapping multiple first reference points.

[0116] Determine a second mapping matrix T2 between the image coordinate system and the second coordinate system of the two-dimensional image, and map the second marking point set to the second coordinate system through T2 to obtain a second mapping point set, where the second mapping point set includes multiple second mapping points, and the multiple second mapping points include at least a second mapping origin obtained by mapping the origin of the second coordinate system, and second mapping reference points obtained by mapping the multiple second reference points.

[0117] In one embodiment, S104 includes:

[0118] S411: Determine a pseudo point in the mapping point set based on the coordinate system corresponding to any one of the marking point sets and the mapping point set.

[0119] Specifically, there may be some false detection points among the detected multiple marking points. These false detection points are not obtained by imaging the calibration points on the calibration target in the two-dimensional image. These false detection points are called pseudo points.

[0120] For any mapping point in the mapping point set corresponding to any marking point set, multiple lines are determined based on the mapping point set. If there are no two lines in the multiple lines that are parallel to the two direction axes of the coordinate system, the any mapping point is determined to be a pseudo point, wherein each line includes the any mapping point.

[0121] In the case where the calibration target includes a first target surface and a second target surface, the pseudo points in the mapping point set are determined based on the coordinate system and mapping point set corresponding to any one of the marking point sets, including: determining the pseudo points in the first mapping point set based on the first coordinate system and the first mapping point set corresponding to the first marking point set, and determining the pseudo points in the second mapping point set based on the second coordinate system and the second mapping point set corresponding to the second marking point set.

[0122] Determining pseudo points in the first mapping point set based on the first coordinate system and the first mapping point set corresponding to the first marking point set includes: for any first mapping point, determining a connecting line between the any first mapping point and each of the remaining first mapping points to obtain multiple connecting lines; if the multiple connecting lines of the any first mapping point do not include a connecting line parallel to the first horizontal axis x1 and another connecting line parallel to the first vertical axis y1, then determining that the any first mapping point is a pseudo point; if there are two connecting lines among the multiple connecting lines of the any first mapping point, and the two connecting lines are parallel to the first horizontal axis x1 and the first vertical axis y1, respectively, then determining that the any first mapping point is not a pseudo point.

[0123] Based on the second coordinate system and the second mapping point set corresponding to the second marking point set, the pseudo points in the second mapping point set are determined, including: for any second mapping point, determining the connection line between the any second mapping point and each of the remaining second mapping points to obtain multiple connection lines. If the multiple connection lines of the any second mapping point do not include a connection line parallel to the second horizontal axis x2 and another connection line parallel to the second vertical axis y2, then the any second mapping point is determined to be a pseudo point; if there are two connection lines in the multiple connection lines of the any second mapping point, and the two connection lines are parallel to the second horizontal axis x2 and the second vertical axis y2 respectively, then the any second mapping point is determined not to be a pseudo point. Figure 10 As shown, e1, e2, e3, and e4 are pseudo points.

[0124] S412: Eliminate pseudo points and the plurality of mapping reference points in the mapping point set to obtain a candidate point set, and determine a reconstructed point set corresponding to any one of the marked point sets based on the candidate point set.

[0125] Specifically, when the calibration target includes a first target surface and a second target surface, pseudo points and a plurality of first mapping reference points are eliminated from the first mapping point set to obtain a first candidate point set, and a first reconstruction point set corresponding to the first marked point set is determined based on the first candidate point set. Pseudo points and a plurality of second mapping reference points are eliminated from the second mapping point set to obtain a second candidate point set, and a second reconstruction point set corresponding to the second marked point set is determined based on the second candidate point set.

[0126] The first reconstruction point set includes multiple first reconstruction points, and the multiple first reconstruction points correspond one-to-one to multiple first common calibration points on the first target surface; the second reconstruction point set includes multiple second reconstruction points, and the multiple second reconstruction points correspond one-to-one to multiple second common calibration points on the second target surface; that is, the first reconstruction point set and the second reconstruction point set include marking points that are not detected through two-dimensional images.

[0127] In one embodiment, the reconstruction point set corresponding to any marker point set is determined by the following steps:

[0128] S4121. Perform clustering processing on the candidate point set to obtain a row direction step length and a column direction step length.

[0129] Specifically, in the case where the calibration target includes a first target surface and a second target surface, the first reconstruction point set corresponding to the first marker point set is determined as an example for explanation. The multiple first common calibration points on the first target surface are arranged according to the preset first row distance and first column distance intervals and are distributed in a matrix; ideally, the multiple first marker points obtained by imaging the multiple first common calibration points in the two-dimensional image are also arranged according to the preset first row distance and first column distance intervals and are distributed in a matrix, and then the first marker point set is mapped to the first coordinate system, and pseudo points and multiple first mapping reference points are eliminated to obtain multiple first candidate points. The multiple first candidate points are also arranged according to the preset first row distance and first column distance intervals and are distributed in a matrix; however, in reality, some calibration points may not be detected, and therefore, the multiple first candidate points may not be distributed in a complete matrix.

[0130] Clustering is performed on the plurality of first candidate points according to their coordinate values ​​to obtain a plurality of clusters, wherein the abscissas of the first candidate points in each cluster are the same, or the ordinates of the first candidate points in each cluster are the same.

[0131] In each cluster with the same horizontal coordinate, the vertical distance between each first candidate point and its adjacent candidate point is determined to obtain multiple vertical distances corresponding to each cluster with the same horizontal coordinate, and the vertical distance with the largest proportion among the multiple vertical distances is used as the column direction step size.

[0132] In each cluster with the same vertical coordinate, the horizontal distance between each first candidate point and its adjacent candidate point is determined to obtain multiple horizontal distances corresponding to each cluster with the same vertical coordinate, and the horizontal distance with the largest proportion among the multiple horizontal distances is used as the row direction step size.

[0133] S4122: Reconstruct a reference point set in the coordinate system corresponding to any one of the marking point sets according to the mapping origin, the row direction step size, and the column direction step size.

[0134] Specifically, in the case where the calibration target includes a first target surface and a second target surface, the reconstruction of a first reference point set in a first coordinate system corresponding to a first marking point set is taken as an example for description.

[0135] The first reference point set includes multiple first reference points, each of which corresponds one-to-one to multiple first common calibration points on the first target surface. The matrix obtained by arranging the multiple first reference points is the same as the matrix obtained by arranging the multiple first common calibration points on the first target surface. The position of the first mapping origin in the matrix corresponding to the multiple first reference points is the same as the position of the first preset origin in the matrix corresponding to the multiple first common calibration points. The number of rows and columns of the matrix corresponding to the multiple first common calibration points on the first target surface, as well as the position of the first coordinate origin in the matrix, are obtained. Based on the obtained number of rows, columns, and positions, as well as the first mapping origin, row direction step size, and column direction step size, the first reference point set is reconstructed.

[0136] For example, the number of rows and columns of the matrix obtained by arranging multiple first common calibration points on the first target surface are both 5, that is, the matrix is ​​a matrix with 5 rows and 5 columns, and the position of the first coordinate origin in the matrix is ​​(2, 4), that is, the position of the first coordinate origin in the matrix is ​​the second row and fourth column; thus, it can be seen that the matrix obtained by arranging multiple first reference points is also a matrix with 5 rows and 5 columns, and the position of the first mapping origin in the matrix obtained by arranging the multiple first reference points is also (2, 4); according to the number of rows and columns of the matrix corresponding to the first reference point set being 5, the position of the first mapping origin in the matrix being (2, 4), the row direction step size and the column direction step size, all first reference points with positions from (1, 1) to (5, 5) in the matrix corresponding to the first reference point set can be determined.

[0137] S4123: Convert the reference point set to the image coordinate system of the two-dimensional image to obtain a reconstructed point set corresponding to any one of the marking point sets.

[0138] Specifically, when the calibration target includes a first target surface and a second target surface, the first reference point set is converted to the image coordinate system to obtain a first reconstruction point set corresponding to the first calibration point set, and the second reference point set is converted to the image coordinate system to obtain a second reconstruction point set corresponding to the second calibration point set.

[0139] According to the first mapping matrix T1 of the first coordinate system and the image coordinate system, the first inverse mapping matrix T1 of the first coordinate system and the image coordinate system is determined -1 , according to the first inverse mapping matrix T1 -1 The first reference point set is converted to the image coordinate system to obtain the first reconstructed point set; according to the second mapping matrix T2 of the second coordinate system and the image coordinate system, the second inverse mapping matrix T2 of the second coordinate system and the image coordinate system is determined -1 , according to the second inverse mapping matrix T2 -1 The second reference point set is converted to the image coordinate system to obtain a second reconstructed point set.

[0140] In one embodiment, S105 includes:

[0141] S511 : Labeling a reconstruction point set corresponding to any one of the marking point sets according to a preset order corresponding to any one of the marking point sets to obtain a plurality of labeled reconstruction points corresponding to the any one of the marking point sets.

[0142] Specifically, when any of the marked point sets is a first marked point set, the preset order corresponding to the marked point set is the first preset order. When any of the marked point sets is a second marked point set, the preset order corresponding to the marked point set is the second preset order. Multiple first reconstruction points in the first reconstruction point set are numbered according to the first preset order to obtain multiple numbered first reconstruction points. Multiple second reconstruction points in the second reconstruction point set are numbered according to the second preset order to obtain multiple numbered second reconstruction points.

[0143] The first preset order is the order pre-set for multiple first common calibration points on the first target surface, and the second preset order is the order pre-set for multiple second common calibration points on the second target surface. Therefore, the order of the multiple first reconstruction points after labeling is the same as the order of the multiple first common calibration points, and the order of the multiple second reconstruction points after labeling is the same as the order of the multiple second common calibration points.

[0144] S512: Select a plurality of target marking points in the two-dimensional image from the plurality of labeled reconstructed points.

[0145] Specifically, when the calibration target includes a first target surface and a second target surface, among multiple labeled first reconstruction points, the first reconstruction point in the two-dimensional image is selected, and the selected first reconstruction point is used as the target marking point; among multiple labeled second reconstruction points, the second reconstruction point in the two-dimensional image is selected, and the selected second reconstruction point is used as the target marking point.

[0146] After determining the order of the reconstruction points (after labeling the reconstruction points), the reconstruction points outside the two-dimensional image boundary are eliminated. This ensures that all target marker points within the two-dimensional image are determined while ensuring that the order detection of the target marker points will not be incorrect due to undetected marker points.

[0147] For example, Figure 10 As shown in , some calibration points on the calibration target are not within the field of view of the two-dimensional imaging device, so some calibration points are not displayed in the two-dimensional image; Figure 11 As shown, according to the above-mentioned sequential identification method of the marking points, the points located at Figure 10 All target marking points in the two-dimensional image shown, as well as the label of each target marking point. Figure 11From the black characters on the white background, it can be seen that the first preset calibration points numbered 3, 4, and 5 on the first target surface are not within the field of view of the two-dimensional imaging device, and the first preset calibration point numbered 11 is blocked by the bone screw; Figure 11 From the white text on a black background, it can be seen that the second preset calibration point numbered 12 on the second target surface is partially blocked by the bone screw. This example shows that, according to the above-mentioned method for sequential recognition of marker points, the undetected marker points can be reconstructed, and the marker points outside the two-dimensional image can be eliminated to obtain all the target marker points within the two-dimensional image. In addition, the undetected marker points do not affect the order of other target marker points.

[0148] In the above-mentioned method for sequential recognition of marking points, the calibration target includes multiple target surfaces, each target surface includes multiple calibration points, a two-dimensional image obtained by photographing the calibration target and the target object is acquired, the marking points in the two-dimensional image are detected to obtain multiple marking point sets, the multiple marking point sets correspond one-to-one to the multiple target surfaces, a coordinate system of the multiple calibration points on the target surface is determined based on the marking point sets corresponding to the target surfaces, the marking point sets are mapped to the coordinate system to obtain a mapping point set, and a reconstructed point set in the image coordinate system is reconstructed based on the coordinate values ​​of the multiple mapping points in the mapping point set and the coordinate system, the multiple reconstruction points included in the reconstructed point set correspond one-to-one to the multiple calibration points on the target surface; The reconstructed point set includes undetected marker points, and all target marker points in the two-dimensional image and the order of each target marker point are determined based on the reconstructed point set; the undetected marker points can be reconstructed according to the above-mentioned marker point sequence recognition method, and the marker points outside the two-dimensional image boundary are eliminated. Therefore, the calibration target does not need to be fixed under the camera of the two-dimensional imaging device. The calibration target can be placed between the target object and the camera of the two-dimensional imaging device in any posture, making the use of the calibration target more casual. The above-mentioned marker point sequence recognition method can accurately detect all target marker points in the two-dimensional image, thereby improving the accuracy of marker point sequence detection.

[0149] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0150] Based on the same inventive concept, the present application also provides a surgical robot system for implementing the aforementioned method for sequentially identifying marker points. The solution provided by this system is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more surgical robot system embodiments provided below can be found in the aforementioned limitations on the method for sequentially identifying marker points, and will not be repeated here.

[0151] In one embodiment, a surgical robot system is provided, comprising: a two-dimensional image acquisition device and a processor;

[0152] The two-dimensional image acquisition device is used to capture a two-dimensional image of a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0153] The processor is configured to acquire the two-dimensional image, detect marker points in the two-dimensional image, and obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image; for any marker point set, construct a coordinate system corresponding to the any marker point set based on the any marker point set, map the any marker point set to the coordinate system, and obtain a mapping point set corresponding to the any marker point set; determine a reconstruction point set corresponding to the any marker point set based on the coordinate system and mapping point set corresponding to the any marker point set, wherein the reconstruction point set includes undetected marker points; determine a plurality of target marker points corresponding to the any marker point set and a label of each target marker point based on the reconstruction point set corresponding to the any marker point set, wherein the plurality of target marker points are all within the two-dimensional image.

[0154] Each module in the aforementioned sequential landmark recognition system may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0155] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 12As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for sequentially identifying marked points is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0156] Those skilled in the art will understand that Figure 12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0157] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0158] Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0159] Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image;

[0160] For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set;

[0161] Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points;

[0162] Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

[0163] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0164] Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0165] Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image;

[0166] For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set;

[0167] Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points;

[0168] Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

[0169] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0170] Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points;

[0171] Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image;

[0172] For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set;

[0173] Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points;

[0174] Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

[0175] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0176] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0177] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for sequential identification of marker points, characterized in that: The method comprises: Acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points; the calibration target is placed between the target object and a camera of a two-dimensional imaging device in an arbitrary posture; Detecting marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image; For any marking point set, construct a coordinate system corresponding to the any marking point set based on the any marking point set, map the any marking point set to the coordinate system, and obtain a mapping point set corresponding to the any marking point set; Determining a reconstruction point set corresponding to any one of the marker point sets based on the coordinate system and the mapping point set corresponding to the any one of the marker point sets, wherein the reconstruction point set includes undetected marker points; Based on the reconstructed point set corresponding to any one of the marker point sets, a plurality of target marker points corresponding to the any one of the marker point sets and a label of each target marker point are determined, wherein the plurality of target marker points are all within the two-dimensional image.

2. The method according to claim 1, characterized in that The any one marking point set includes a plurality of marking points; and constructing a coordinate system corresponding to the any one marking point set based on the any one marking point set includes: Determine the nearest neighboring marker point of each marker point in any marker point set, and select a coordinate system origin and a plurality of reference points in any marker point set based on the nearest neighboring marker point of each marker point; A coordinate system corresponding to any one of the marking point sets is constructed according to a coordinate system origin and a plurality of reference points selected from the any one of the marking point sets.

3. The method according to claim 2, characterized in that The mapping point set corresponding to any one of the marking point sets includes at least: a mapping origin and a plurality of mapping reference points, wherein the mapping origin is a point obtained by mapping the origin of the coordinate system to the coordinate system, and the mapping reference points are points obtained by mapping the reference points to the coordinate system; determining the reconstructed point set corresponding to any one of the marking point sets based on the coordinate system and the mapping point set corresponding to the any one of the marking point sets includes: Determining a pseudo point in the mapping point set based on the coordinate system corresponding to any one of the marking point sets and the mapping point set; Eliminating pseudo points and the plurality of mapping reference points from the mapping point set to obtain a candidate point set; A reconstruction point set corresponding to any one of the marked point sets is determined based on the candidate point set.

4. The method according to claim 3, characterized in that The determining of the pseudo points in the mapping point set based on the coordinate system corresponding to any one of the marking point sets and the mapping point set includes: For any mapping point in the mapping point set corresponding to any marking point set, multiple lines are determined based on the mapping point set. If there are no two lines in the multiple lines that are respectively parallel to the two direction axes of the coordinate system, then the any mapping point is determined to be a pseudo point, wherein each line includes the any mapping point.

5. The method according to claim 3, characterized in that The determining, based on the candidate point set, a reconstruction point set corresponding to any one of the marked point sets comprises: Performing clustering processing on the candidate point set to obtain a row direction step length and a column direction step length; Reconstructing a reference point set in the coordinate system corresponding to any one of the marking point sets according to the mapping origin, the row direction step size, and the column direction step size; The reference point set is converted into the image coordinate system of the two-dimensional image to obtain a reconstruction point set corresponding to any one of the marking point sets.

6. The method according to claim 1, wherein The determining, based on the reconstructed point set corresponding to the any one marker point set, a plurality of target marker points corresponding to the any one marker point set and a label of each target marker point includes: Labeling the reconstruction point set corresponding to any one of the marking point sets according to a preset order corresponding to the any one of the marking point sets to obtain a plurality of labeled reconstruction points corresponding to the any one of the marking point sets; Among the plurality of labeled reconstructed points, a plurality of target marking points in the two-dimensional image are selected.

7. The method according to claim 1, characterized in that The multiple calibration points include multiple first preset calibration points and multiple second preset calibration points. The calibration target includes a first target surface and a second target surface parallel to each other. The first target surface is provided with multiple first preset calibration points, and the second target surface is provided with multiple second preset calibration points. The radius of the first preset calibration point is different from the radius of the second preset calibration point. The multiple first preset calibration points include a first preset origin and multiple first preset reference points. The multiple second preset calibration points include a second preset origin and multiple second preset reference points.

8. The method according to claim 7, characterized in that The detecting of the marker points in the two-dimensional image to obtain a plurality of marker point sets includes: Determining a segmentation mask of the two-dimensional image, and detecting marker points based on the segmentation mask to obtain an initial marker point set; The initial marker point set is divided into a first marker point set and a second marker point set based on a preset radius interval, wherein a radius of a first marker point in the first marker point set is different from a radius of a second marker point in the second marker point set.

9. A surgical robot system, characterized in that: The system includes: a two-dimensional image acquisition device and a processor; The two-dimensional image acquisition module is used to acquire a two-dimensional image obtained by photographing a calibration target and a target object, wherein the calibration target includes a plurality of calibration points; the calibration target is placed between the target object and the camera of the two-dimensional imaging device in an arbitrary posture; The processor is configured to detect marker points in the two-dimensional image to obtain a plurality of marker point sets, wherein the marker points are points where the calibration points are imaged in the two-dimensional image; for any marker point set, construct a coordinate system corresponding to the any marker point set based on the any marker point set, map the any marker point set to the coordinate system, and obtain a mapping point set corresponding to the any marker point set; determine a reconstruction point set corresponding to the any marker point set based on the coordinate system and mapping point set corresponding to the any marker point set, wherein the reconstruction point set includes undetected marker points; determine a plurality of target marker points corresponding to the any marker point set and a label of each target marker point based on the reconstruction point set corresponding to the any marker point set, wherein the plurality of target marker points are all within the two-dimensional image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images