Surgical robot landmark identification and positioning method, apparatus and device

Abstract

The present disclosure relates to a surgical robot landmark recognition positioning method, device and equipment, the method comprising: acquiring a surgical image shot through a preset calibration ruler; performing feature point detection based on the surgical image to obtain a point set comprising multiple feature points; combining feature points in the point set that satisfy three-point collinear arrangement, and obtaining a first straight line set comprising three-point collinear line segments based on the combination result; obtaining a line segment template according to the calibration ruler; wherein the line segment template comprises multiple numbered landmarks and multiple sample line segments composed of collinear landmarks; finding multiple target line segments matched with the sample line segments based on the first straight line set; and determining target landmark points in the surgical image for surgical robot positioning according to the landmarks and feature points in the target line segments. The present disclosure can improve the accuracy of landmark recognition positioning.

Description

Technical Field

[0001] This disclosure relates to the field of surgical robot technology, and in particular to a method, apparatus and equipment for identifying and locating markers in a surgical robot. Background Technology

[0002] Surgical robots utilize medical imaging for surgical positioning. The principle behind this is to use medical imaging equipment to capture images of the patient during surgery and calculate the mapping relationship between the surgical robot and the surgical space based on specific landmarks on the images. This allows for precise surgical path planning, assisting surgeons in achieving robot-assisted surgery with less fluoroscopic radiation, higher surgical precision, and better surgical outcomes. Among these, the automatic and effective identification of specific landmarks on medical images is a crucial step affecting surgical time and results. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a method, apparatus, and device for identifying and locating markers on a surgical robot.

[0004] According to one aspect of this disclosure, a method for identifying and locating markers in a surgical robot is provided, comprising:

[0005] Acquire surgical images captured using a preset calibration scale;

[0006] Based on the surgical image, feature point detection is performed to obtain a point set including multiple feature points;

[0007] The feature points in the set of points that satisfy the collinear arrangement of three points are combined, and a first set of straight lines including collinear line segments is obtained based on the combination result.

[0008] A line segment template is obtained according to the calibration scale; wherein, the line segment template includes multiple numbered markers and multiple sample line segments composed of collinear markers;

[0009] Based on the first set of lines, multiple target line segments that match the sample line segments are searched;

[0010] Based on the marker position and the feature points in the target line segment, the target marker point for surgical robot positioning in the surgical image is determined.

[0011] According to another aspect of this disclosure, a surgical robot marker recognition and positioning device is also provided, comprising:

[0012] The image acquisition module is used to acquire surgical images captured through a preset calibration scale;

[0013] The point set acquisition module is used to perform feature point detection based on the surgical image to obtain a point set including multiple feature points;

[0014] The first straight line set acquisition module is used to combine feature points in the point set that satisfy the collinear arrangement of three points, and obtain a first straight line set including collinear line segments based on the combination result;

[0015] The template acquisition module is used to acquire a line segment template according to the calibration scale; wherein, the line segment template includes multiple numbered flags and multiple sample line segments composed of collinear flags;

[0016] The line segment matching module is used to find multiple target line segments that match the sample line segments based on the first set of lines;

[0017] The marker point determination module is used to determine the target marker points in the surgical image for surgical robot positioning based on the marker positions and feature points in the target line segment.

[0018] According to another aspect of this disclosure, an electronic device is also provided, the electronic device comprising:

[0019] processor;

[0020] Memory used to store the processor's executable instructions;

[0021] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the above method.

[0022] According to another aspect of this disclosure, a computer-readable storage medium is also provided, the storage medium storing a computer program for performing the above-described method.

[0023] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0024] The surgical robot marker recognition and positioning method, apparatus, and device provided in this disclosure include: acquiring a surgical image captured through a preset calibration ruler; performing feature point detection based on the surgical image to obtain a point set including multiple feature points; combining the feature points in the point set that satisfy the collinear arrangement of three points to obtain a first set of straight lines including collinear line segments based on the combination result; obtaining a line segment template according to the calibration ruler; wherein the line segment template includes multiple numbered markers and multiple sample line segments composed of collinear markers; searching for multiple target line segments that match the sample line segments based on the first set of straight lines; and determining target marker points in the surgical image for surgical robot positioning based on the markers and feature points in the target line segments.

[0025] In this embodiment, the line segment template includes sample line segments and markers as matching standards. In this case, on the one hand, the length ratios between sample line segments and the length ratios of sub-segments within a sample line segment are used to perform matching calculations. Therefore, it is compatible with various imaging devices, allowing for less restriction on the angle between the imaging device and the calibration scale. Furthermore, it can accurately and effectively identify feature points on surgical images of different imaging qualities, without requiring high image quality. Additionally, using the length ratios between sample line segments for matching calculations avoids the angle errors caused by projection in existing technologies that use feature triangles, thereby improving the accuracy of positioning and recognition. On the other hand, using markers to extract target markers for surgical robot positioning further improves the accuracy of positioning and recognition. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0027] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a surgical robot marker recognition and positioning method according to an embodiment of the present disclosure;

[0029] Figure 2 This is a schematic diagram of the calibration scale described in the embodiments of this disclosure;

[0030] Figure 3 This is a schematic diagram of the line segment template described in an embodiment of this disclosure;

[0031] Figure 4 This is a flowchart of another surgical robot marker recognition and positioning method according to an embodiment of this disclosure;

[0032] Figure 5 This is a structural block diagram of a surgical robot marker recognition and positioning device according to an embodiment of this disclosure;

[0033] Figure 6 This is a schematic diagram of the structure of the electronic device described in an embodiment of this disclosure. Detailed Implementation

[0034] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0035] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0036] Surgical robots typically require the use of medical images, such as 2D X-ray fluoroscopy, for surgical positioning. In this process, the automatic and effective identification of specific landmarks on medical images is a crucial step affecting surgical time and outcome. Some methods involve automatically identifying landmarks by first calculating the distance and perpendicularity between them; then, identifying, sorting, and numbering the landmarks searchable within a limited threshold range. However, this method has two drawbacks: first, it is easily affected by background factors such as image grayscale distribution, guide needles, and bone, leading to either the inability to detect effective candidate points or the detection of too many candidate points, resulting in the inability to correctly identify effective landmarks; second, when applied to different types of medical imaging equipment (such as C-arms), different recognition thresholds are required due to limitations such as image size, making this method incompatible with different imaging devices. To address these issues, this disclosure provides a surgical robot landmark identification and positioning method, apparatus, and device. For ease of understanding, the embodiments of this disclosure are described below.

[0037] Figure 1 This is a flowchart illustrating a surgical robot marker identification and positioning method according to an embodiment of the present disclosure. This method can be executed by a surgical robot marker identification and positioning device, which can be implemented using software and / or hardware. Figure 1 As shown, the surgical robot marker identification and positioning method provided in this embodiment may include the following steps.

[0038] Step S101: Obtain surgical images captured using a preset calibration ruler.

[0039] This embodiment utilizes medical imaging equipment such as a C-arm to capture X-ray fluoroscopic images of a patient during surgery using a preset calibration scale, thereby obtaining surgical images. The calibration scale may include multiple markers arranged in a specified distribution, such as... Figure 2 The image shows an example of a calibration scale; the markers on this scale can be circular points with a preset radius. Depending on their arrangement, the radii of multiple markers can be the same or different, such as... Figure 2The example demonstrates two types of markers with different radii. A set of markers arranged in a collinear pattern of three or four points can form a sample line segment. In the calibration scale, there can be multiple sample line segments, and the sub-segments separated by the markers in each sample line segment can have a certain length ratio relationship.

[0040] Step S102: Perform feature point detection based on the surgical image to obtain a point set including multiple feature points.

[0041] In one implementation, feature point detection can be performed on surgical images using image processing techniques such as the blob algorithm, findContours algorithm, and Houghcircles algorithm to obtain a point set including multiple feature points. The point set obtained in this way can more comprehensively and completely cover the feature points in the surgical image and avoid missing feature points.

[0042] Alternatively, in another implementation, the surgical image is preprocessed, such as by performing grayscale iterative stretching or Gaussian filtering. Feature point detection is then performed on the preprocessed surgical image to obtain a set of points containing multiple feature points. This method increases the accuracy and clarity of the feature points, which is beneficial for improving the accuracy of subsequent processing.

[0043] Alternatively, the point sets obtained by the above two methods can be merged, and subsequent processing can be performed based on the merged point set.

[0044] Step S103: Combine the feature points in the point set that satisfy the collinear arrangement of three points, and obtain the first set of straight lines including the collinear line segments based on the combination result.

[0045] This embodiment, based on the condition that three points are collinear, combines every three feature points in the point set, and then uses the vector product to determine whether the three feature points in the combination are collinear, thus obtaining the first set of straight lines. In specific implementation, the vector product of the line segments formed by the three feature points in the combination result can be calculated; the smaller the vector product, the closer the line segment is to a straight line. Therefore, line segments with a vector product less than a preset value can be added to the first set of straight lines. The preset value can be set to zero, for example, that is, line segments with a vector product of zero are added to the first set of straight lines.

[0046] Step S104: Obtain a line segment template according to the calibration scale; wherein, the line segment template includes multiple numbered markers and multiple sample line segments composed of collinear markers.

[0047] It is understandable that the line segment template obtained from the calibration scale can accurately represent the calibration characteristics of the calibration scale. The calibration characteristics may include: the number of markers, their arrangement and distribution, the radius ratio between markers, their relative positional relationship, position coordinates, sample line segments composed of collinear markers, the length ratio between sample line segments, and the length ratio between sub-line segments separated by markers within a sample line segment, etc.

[0048] like Figure 3 The example is based on Figure 2 The line segment template is obtained from the calibration scale. For ease of understanding, each marker can be assigned a unique number, such as 1, 2, 3, etc. The method for determining sample line segments composed of collinear markers in the line segment template can include, for example, forming a first line segment by grouping four collinear markers, and determining the first number group corresponding to this first line segment based on the marker numbers; Figure 3 For example, the first sample line segment is formed by the four markers in the first numbered group (1, 2, 3, 4). Similarly, the second line segment corresponding to the second numbered group (4, 5, 6) is formed by three collinear markers numbered 4, 5, and 6; and the third line segment corresponding to the second numbered group (1, 7, 6) is formed by three collinear markers numbered 1, 7, and 6. Furthermore, to meet the actual surgical positioning needs and facilitate the identification of markers and sample line segments, the radius of some markers can be selectively set to be larger than the radius of other markers. For example, the radius of markers numbered 4 and 7 can be set to twice the radius of the other markers. In the line segment template, each marker can be assigned coordinates. These coordinates facilitate accurate marker positioning and can also be used to determine various information such as the length and angle of the sample line segment, and the proportion between sub-segments within the sample line segment.

[0049] Step S105: Based on the first set of straight lines, find multiple target line segments that match the sample line segments.

[0050] In one possible embodiment, any line segment in the first set of lines can be matched with each sample line segment in the line segment template, and this matching may include at least the following:

[0051] The length ratio matching specifically involves matching the length ratio of any line segment relative to other line segments with the length ratio of the current sample line segment relative to other sample line segments. The sub-segment ratio matching specifically involves matching the length ratio between sub-segments within any line segment with the length ratio between sub-segments within the current sample line segment. The position matching specifically involves matching the position and / or angle of any line segment relative to other line segments with the position and / or angle of the current sample line segment relative to other sample line segments. The flag matching specifically involves matching the information of feature points in any line segment with the information of flag points in the current sample line segment. The information used for matching feature points and flag points can include radius, number, and arrangement.

[0052] By matching line segments in the first set of lines with sample line segments in the line segment template, multiple target line segments that match the sample line segments are found from the first set of lines. Based on the above matching content, the target line segments found in this embodiment meet the matching error range corresponding to each matching content in terms of length ratio, length ratio between sub-line segments, position and / or angle, and flag position.

[0053] In the above embodiment of finding multiple target line segments that match the sample line segments, the matching calculation is performed using information such as the length ratio between sample line segments and the length ratio between sub-line segments. This not only ensures compatibility with various imaging devices of different models, allowing for less restriction on the angle between the imaging device and the calibration scale, but also enables accurate and effective identification of feature points on surgical images (i.e., X-ray fluoroscopic images) of different imaging qualities. Furthermore, the matching calculation using the length ratio between sample line segments avoids the angle error caused by projection in the existing technology that uses feature triangles for calculation, thereby improving the accuracy of positioning and identification.

[0054] Step S106: Determine the target marker points in the surgical image for surgical robot positioning based on the marker positions and feature points in the target line segment.

[0055] According to the above embodiments, matching the target line segment with the sample line segment indicates that positioning has been achieved in the direction of the straight line. Simultaneously, the target line segment contains feature points that can be accurately located and reflect coordinate positions. However, considering the potential errors in the matching process of feature points and line segments, it is necessary to determine whether any feature points in the target line segment are missing. Based on this, the method for determining the target marker points in this embodiment can be referred to as follows.

[0056] Determine if the number of non-repeating feature points in the target line segment is the same as the number of flag bits. If yes, the feature points in the target line segment are identified as target flag points for surgical robot localization in the surgical image; otherwise, the missing feature points are calculated inversely based on the flag bits and the currently existing feature points to obtain the predicted feature points.

[0057] Specifically, we can determine the first marker position that corresponds one-to-one with the existing feature points in the target line segment, and the second marker position that does not correspond to a feature point; it can be understood that the second marker position corresponds to the missing feature point. The relative position, arrangement, and radius between the first and second marker positions are known. Therefore, based on the above information between the first and second marker positions, as well as the position, arrangement, and radius of the existing feature points, we can reverse-calculate the missing feature points to obtain the predicted feature points.

[0058] Then, target markers for surgical robot localization are determined in the surgical image based on existing feature points and predicted feature points.

[0059] This embodiment can directly merge the existing and predicted feature points as target markers for surgical robot positioning in the surgical image. Alternatively, to further improve accuracy, this embodiment can first verify whether the existing and predicted feature points match the corresponding markers in the line segment template. That is, it merges the existing and predicted feature points and verifies whether the merged feature points match the markers in terms of relative position, arrangement, and radius. If a match is verified, the existing and predicted feature points are determined as target markers for surgical robot positioning in the surgical image. Of course, if a mismatch is verified, new predicted feature points can be recalculated, or missing feature points can be manually added.

[0060] Based on the above embodiments, this embodiment can provide another method for surgical robot marker recognition and positioning, which can be referred to as follows. Figure 4 As shown, the steps include the following.

[0061] Step S201: Obtain surgical images captured using a preset calibration ruler.

[0062] Step S202: Obtain a line segment template according to the calibration scale; wherein, the line segment template includes multiple numbered markers and multiple sample line segments composed of collinear markers.

[0063] Step S203: Perform feature point detection based on the surgical image to obtain a point set including multiple feature points.

[0064] Step S204: Combine the feature points in the point set that satisfy the collinear arrangement of three points, and obtain the first set of straight lines including the collinear line segments based on the combination result.

[0065] Step S205: Combine the feature points in the point set that satisfy the condition of collinear arrangement of four points, and obtain a second set of straight lines including collinear line segments based on the combination results. In this embodiment, based on the condition of collinearity of four points, every four feature points in the point set are combined, and then the vector product is used to determine whether the four feature points of the combination are collinear, so as to obtain the second set of straight lines.

[0066] Step S206: Obtain the first sample line segment composed of four flag bits of the first number group from the sample line segments of the line segment template.

[0067] Step S207: Obtain the first length ratio of the sub-segment between every two adjacent marker positions in the first sample line segment. Figure 3 For example, the first sample line segment is a line segment composed of the first numbered group (1, 2, 3, 4); the sub-line segment between every two adjacent flag positions in the first sample line segment can be represented as L. ij Let ij be the flag number. Therefore, based on the position coordinates of the flag, the first length ratio between the sub-segments can be calculated as L. 12 :L 23 :L 34 = 1:2:3.

[0068] Step S208: Find a first line segment from the line segments of the second set of lines that matches the first sample line segment and the first length ratio. Each line segment in the second set of lines, consisting of four collinear feature points, can be matched with the first sample line segment and the first length ratio in the line segment template to obtain a matching error. If the matching error is within a preset error range, it means that a first line segment matching the first sample line segment and the first length ratio can be found. If a first line segment is found from the second set of lines, the embodiment corresponding to step S209 can be executed. If the matching errors of multiple line segments are not within the preset error range, it means that a first line segment matching the first sample line segment and the first length ratio could not be found. In this case, the embodiment corresponding to step S214 can be executed.

[0069] Step S209: Delete the collinear line segments consisting of three collinear points formed by feature points in the first line segment from the first line segment set, thus obtaining the third line set. The line segments in the first line set and... Figure 3 Medium sample line segment L 123 L 234 L 456 L 176 Correspondingly, the first line segment and Figure 3 Sample line segment L 1234Correspondingly, by deleting the collinear line segments consisting of three collinear points from the feature points of the first line segment from the first line segment set, a third line set is obtained. The line segments included in the third line set are... Figure 3 Medium sample line segment L 456 L 176 Correspondingly.

[0070] Step S210: Obtain a second sample line segment composed of three flag bits from the sample line segments of the line segment template, and a third sample line segment composed of three flag bits from the third number group. For example, the second number group is (4, 5, 6), and the second sample line segment formed by it is L. 456 The second numbered group is (1, 7, 6), and the third sample line segment formed by it is L. 176 .

[0071] Step S211: Obtain the second and third length ratios of the sub-segments between every two adjacent markers in the second and third sample line segments, respectively. The specific calculation method for the first length ratio between sub-segments in the first sample line segment can be referred to, and will not be elaborated further here.

[0072] Step S212: From the line segments of the third set of straight lines, find the second line segment that matches the ratio of the second sample line segment and the second length, and find the third line segment that matches the ratio of the third sample line segment and the third length.

[0073] Step S213: Determine the first line segment, the second line segment, and the third line segment as the target line segment.

[0074] According to the above embodiments, based on the different line segment division methods of three-point collinearity and four-point collinearity, the first line segment is first found from the second line set using the feature of four-point collinearity. Then, based on this, the line segments in the first line set are reduced. The resulting third line set can reduce the amount of line segment data. Thus, in the process of finding the second and third line segments from the third line set using the feature of three-point collinearity, the number of interfering line segments and the computational complexity of line segment matching can be reduced, thereby improving matching efficiency and accuracy.

[0075] Step S214: Find multiple target line segments that match the sample line segments from the first set of straight lines. Matching the sample line segments includes: matching the first sample line segment and the first length ratio, matching the second sample line segment and the second length ratio, and matching the third sample line segment and the third length ratio.

[0076] In this embodiment, if the first line segment cannot be found from the second line set, multiple target line segments that match the sample line segment can be directly searched based on the first line set. The specific implementation method can be referred to the embodiment of step S105 above.

[0077] After obtaining the target line segment according to the above embodiments, the target marker point can be determined according to the following steps based on the marker position and the feature points in the target line segment.

[0078] Step S215: Determine whether the number of non-repeating feature points in the target line segment is the same as the number of flag bits; if yes, proceed to step S216; if no, proceed to steps S217 and S218.

[0079] Step S216: The feature points in the target line segment are determined as target markers in the surgical image for the positioning of the surgical robot.

[0080] Step S217: Based on the flag bit and the currently existing feature points, perform inverse calculation on the missing feature points to obtain the predicted feature points.

[0081] Step S218: Verify whether the predicted feature point matches the corresponding marker in the line segment target, and if the match is verified, determine the existing feature point and the predicted feature point as the target marker in the surgical image for surgical robot localization.

[0082] In summary, the surgical robot marker recognition and localization method provided in this embodiment first obtains a set of points based on the surgical image, and then combines the three points from the set of points based on collinearity to obtain a first set of straight lines including multiple line segments; next, using sample line segments with length ratios in the line segment template, multiple target line segments that match the sample line segments are searched based on the first set of straight lines; finally, the target marker points in the surgical image used for surgical robot localization are determined according to the marker points and feature points in the target line segments.

[0083] In the above technical solution, the line segment template contains sample line segments and marker positions as matching standards. In this case, on the one hand, the length ratios between sample line segments and the length ratios of sub-segments within a sample line segment are used to perform matching calculations. Therefore, it is compatible with various imaging devices of different models, allowing for less restriction on the angle between the imaging device and the calibration scale. Furthermore, it can accurately and effectively identify feature points on surgical images (i.e., X-ray fluoroscopy images) of different imaging qualities, without requiring high image quality. Additionally, using the length ratios between sample line segments for matching calculations avoids the angular errors caused by projection in existing technologies that use feature triangles for calculations, thereby improving the accuracy of positioning and recognition. On the other hand, using marker positions to extract target marker points for surgical robot positioning further improves the accuracy of positioning and recognition.

[0084] Figure 5This is a structural block diagram of a surgical robot marker recognition and positioning device provided in an embodiment of this disclosure. This device can be used to implement the aforementioned surgical robot marker recognition and positioning method, and can be implemented using software and / or hardware. Figure 5 As shown, the surgical robot marker recognition and positioning device provided in this embodiment may include:

[0085] Image acquisition module 301 is used to acquire surgical images captured through a preset calibration scale;

[0086] The point set acquisition module 302 is used to perform feature point detection based on the surgical image to obtain a point set including multiple feature points;

[0087] The first straight line set acquisition module 303 is used to combine feature points in the point set that satisfy the collinear arrangement of three points, and obtain a first straight line set including collinear line segments based on the combination result;

[0088] The template acquisition module 304 is used to acquire a line segment template according to the calibration scale; wherein, the line segment template includes multiple numbered flags and multiple sample line segments composed of collinear flags;

[0089] The line segment matching module 305 is used to find multiple target line segments that match the sample line segments based on the first set of straight lines;

[0090] The marker point determination module 306 is used to determine the target marker points in the surgical image for surgical robot positioning based on the marker positions and feature points in the target line segment.

[0091] The device provided in this embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0092] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Figure 4 As shown, the electronic device 400 includes one or more processors 401 and memory 402.

[0093] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0094] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the surgical robot marker identification and positioning method of the embodiments of this disclosure described above, and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0095] In one example, the electronic device 400 may further include an input device 403 and an output device 404, these components being interconnected via a bus system and / or other forms of connection mechanism (not shown). Furthermore, the input device 403 may also include, for example, a keyboard, a mouse, etc.

[0096] The output device 404 can output various information to the outside, including determined distance information, direction information, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0097] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device 400 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 400 may include any other suitable components depending on the specific application.

[0098] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program for executing the above-described surgical robot marker identification and positioning method.

[0099] The present disclosure provides a computer program product for a surgical robot marker identification and positioning method, device, electronic device, and medium, which includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0100] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying and locating markers in a surgical robot, characterized in that, include: Acquire surgical images captured using a preset calibration scale; Based on the surgical image, feature point detection is performed to obtain a point set including multiple feature points; The feature points in the set of points that satisfy the collinear arrangement of three points are combined, and a first set of straight lines including collinear line segments is obtained based on the combination result. A line segment template is obtained according to the calibration scale; wherein, the line segment template includes multiple numbered markers and multiple sample line segments composed of collinear markers; Based on the first set of lines, multiple target line segments that match the sample line segments are searched; Based on the marker position and the feature points in the target line segment, determine the target marker point in the surgical image for surgical robot positioning; The step of determining the target marker points for surgical robot positioning in the surgical image based on the marker positions and feature points in the target line segment includes: Determine whether the number of non-repeating feature points in the target line segment is the same as the number of flag bits; If so, the feature points in the target line segment are determined as target markers in the surgical image for surgical robot positioning; If not, then the missing feature points are calculated inversely based on the flag bit and the currently existing feature points to obtain the predicted feature points; The target markers for surgical robot localization in the surgical image are determined based on the existing feature points and the predicted feature points.

2. The method according to claim 1, characterized in that, The method further includes: The feature points in the point set that satisfy the collinear arrangement of four points are combined, and a second set of straight lines including collinear line segments is obtained based on the combination result. Obtain a first sample line segment composed of four flag bits from the first number group from the sample line segments of the line segment template; Obtain the first length ratio of the sub-segment between every two adjacent flag positions in the first sample line segment; Find the first line segment from the second set of line segments that matches the ratio of the first sample line segment to the first length.

3. The method according to claim 2, characterized in that, The step of finding multiple target line segments that match the sample line segments based on the first set of lines includes: If the first line segment is found in the second set of lines, delete the three collinear line segments composed of feature points in the first line segment from the first set of lines to obtain the third set of lines; From the sample line segments of the line segment template, obtain a second sample line segment composed of three flag bits from the second number group and a third sample line segment composed of three flag bits from the third number group; Obtain the second length ratio and the third length ratio of the sub-segment between every two adjacent flag positions in the second sample line segment and the third sample line segment, respectively; From the line segments of the third set of straight lines, find a second line segment that matches the ratio of the second sample line segment to the second length, and find a third line segment that matches the ratio of the third sample line segment to the third length. The first line segment, the second line segment, and the third line segment are identified as the target line segment.

4. The method according to claim 3, characterized in that, The step of finding multiple target line segments that match the sample line segments based on the first set of lines includes: Find multiple target line segments that match the sample line segments from the first set of lines, wherein matching the sample line segments includes: matching the first sample line segment and the first length ratio, matching the second sample line segment and the second length ratio, and matching the third sample line segment and the third length ratio.

5. The method according to claim 1, characterized in that, The step of determining the target marker points for surgical robot localization in the surgical image based on the currently existing feature points and the predicted feature points includes: Verify whether the predicted feature point matches the corresponding flag in the target line segment; In the case of a verified match, the currently existing feature points and the predicted feature points are determined as target markers in the surgical image for the positioning of the surgical robot.

6. The method according to claim 1, characterized in that, The first set of straight lines obtained based on the combination result includes three collinear line segments, including: Calculate the vector product of the line segments formed by the three feature points in the combined result; Add the line segments whose vector product is less than a preset value to the first set of lines.

7. A surgical robot marker recognition and positioning device, characterized in that, include: The image acquisition module is used to acquire surgical images captured through a preset calibration scale; The point set acquisition module is used to perform feature point detection based on the surgical image to obtain a point set including multiple feature points; The first straight line set acquisition module is used to combine feature points in the point set that satisfy the collinear arrangement of three points, and obtain a first straight line set including collinear line segments based on the combination result; The template acquisition module is used to acquire a line segment template according to the calibration scale; wherein, the line segment template includes multiple numbered flags and multiple sample line segments composed of collinear flags; The line segment matching module is used to find multiple target line segments that match the sample line segments based on the first set of lines; The marker point determination module is used to determine the target marker points in the surgical image for surgical robot positioning based on the marker positions and feature points in the target line segment; The marker point determination module is also used for: Determine whether the number of non-repeating feature points in the target line segment is the same as the number of flag bits; If so, the feature points in the target line segment are determined as target markers in the surgical image for surgical robot positioning; If not, then the missing feature points are calculated inversely based on the flag bit and the currently existing feature points to obtain the predicted feature points; The target markers for surgical robot localization in the surgical image are determined based on the existing feature points and the predicted feature points.

8. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1-6.

Images