Registration method, surgical navigation system spatial registration method, and registration device

Abstract

The application relates to a registration method, a surgical navigation system space registration method and a registration device. The method is applied to an optical surgical navigation system; the optical surgical navigation system comprises an optical navigator and one or more marker assemblies; the optical navigator is used for capturing the marker assemblies; the marker assemblies comprise a plurality of optical markers, and the relative positions among the optical markers are invariable; the method comprises the following steps: acquiring a target image sequence; the target image sequence comprises a plurality of target images output by the optical navigator based on the optical markers in a field of view; processing the target images to obtain position information of each optical marker in the target images; generating geometric feature information among the optical markers based on the position information in the target image sequence; dividing the optical markers according to the geometric feature information among the optical markers to obtain an identification result of each marker assembly; and registering the marker assemblies according to the identification result. The method can improve the registration efficiency of the surgical navigation.

Description

Technical Field

[0001] This application relates to the field of surgical navigation technology, and in particular to a registration method, a spatial registration method for a surgical navigation system, and a registration device. Background Technology

[0002] With the development of surgical navigation technology, optical navigation systems have emerged. The target being navigated typically includes markers that can be captured by the optical navigation instrument. For example, multiple markers may be distributed on the target in a specific geometric shape. This specific geometry serves as prior knowledge for the optical surgical navigation system. Only after acquiring the specific geometric shape information of a target can the system accurately and stably identify and navigate to the target. Currently, in existing systems, this specific geometry, as prior knowledge, is usually measured by a corresponding measuring device and pre-loaded into the optical surgical navigation system in the form of a configuration file. The optical surgical navigation system then performs surgical navigation on the target.

[0003] However, current registration methods or traditional approaches suffer from problems such as low efficiency in surgical navigation registration. Summary of the Invention

[0004] Therefore, it is necessary to provide a registration method, a spatial registration method for surgical navigation systems, and a registration device that can improve the efficiency of surgical navigation registration, in order to address the aforementioned technical problems.

[0005] In a first aspect, this application provides a registration method applied to an optical surgical navigation system; the optical surgical navigation system includes an optical navigator and one or more marker components; wherein the optical navigator is used to capture the marker components; the marker components include multiple optical markers, the relative positions of the optical markers remaining unchanged; the method includes:

[0006] Acquire a target image sequence; the target image sequence includes multiple target images output by the optical navigator based on optical markers in the field of view;

[0007] Process the target image to obtain the position information of each optical marker in the target image;

[0008] Based on the location information in the target image sequence, generate geometric feature information between each optical marker;

[0009] Based on the geometric feature information between each optical marker, each optical marker is divided to obtain the recognition result of each marker component;

[0010] Based on the identification results, the tagged components are registered.

[0011] In one embodiment, the step of dividing the optical markers based on the geometric feature information between them to obtain the recognition result of each marker component includes:

[0012] Based on the geometric feature information between each optical marker, the segmentation result for the optical marker is obtained; the geometric feature information between each optical marker includes the relative positional relationship between each optical marker in the target image sequence; the segmentation result includes one or more groups of optical markers in the target image sequence whose relative positional relationship remains unchanged;

[0013] Based on the segmentation results, the recognition results of each marker component are obtained; the recognition results include the geometric feature information of the marker component.

[0014] In one embodiment, obtaining the recognition results of each marker component further includes:

[0015] Based on the geometric feature information of the marker component and the component registry, query the registration status of the marker component;

[0016] If the registration status of the tag component is unregistered, a new entry for the tag component will be added to the component registry. The entry information includes the tag component's identification number and geometric feature information.

[0017] In one embodiment, obtaining the recognition results of each marker component further includes:

[0018] If the registration status of the tag component is "registered", then the geometric feature information of the tag component is matched with the component registry to obtain the entry information of the tag component.

[0019] In one embodiment, the entry information also includes a component configuration file; the component configuration file is generated based on the geometric feature information of the tagged components;

[0020] Based on the identification results, the steps for registering the tagged components include:

[0021] Register the marked components according to the component configuration file.

[0022] In one embodiment, the method further includes:

[0023] If the entry information of the marker component does not contain navigation target information, then based on the entry information, the application object prompt information for the marker component is output; the application object prompt information is used to guide the user to input navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to.

[0024] If navigation target information is received, add the navigation target information to the entry information;

[0025] If the entry information of the marker component includes navigation target information, then the navigation target information is displayed.

[0026] In one embodiment, the method further includes:

[0027] If a user input instruction is received, the user input information is obtained; the user input instruction includes at least one of adding information to the entry information of the marker component and overwriting information on the entry information of the marker component; the user input information includes navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to; based on the user input instruction and the user input information, the entry information of the marker component is updated.

[0028] In one embodiment, the target image sequence includes multiple frames of target images acquired by the optical navigator while the marker component is in relative motion with respect to the optical navigator.

[0029] Secondly, this application provides a spatial registration method for a surgical navigation system that includes the above-described registration method, the method further comprising:

[0030] Retrieve entry information for registered tag components;

[0031] Based on the entry information of the registered tag components, spatial registration is performed on the tag components.

[0032] Thirdly, this application provides a registration device applied to an optical surgical navigation system; the optical surgical navigation system includes an optical navigator and one or more marker components; wherein the optical navigator is used to capture the marker components; the marker components include a plurality of optical markers, the relative positions of the optical markers remaining unchanged; the device includes:

[0033] The image acquisition module is used to acquire a target image sequence; the target image sequence includes multiple target images output by the optical navigator based on optical markers in the field of view;

[0034] The location acquisition module is used to process the target image and obtain the location information of each optical marker in the target image;

[0035] The marker recognition module is used to generate geometric feature information between optical markers based on the location information in the target image sequence;

[0036] The component segmentation module is used to segment each optical marker based on the geometric feature information between them, and to obtain the recognition results of each marker component.

[0037] The component registration module is used to register the tagged components based on the recognition results.

[0038] Fourthly, this application provides an optical surgical navigation system, which includes:

[0039] A marking assembly comprising a plurality of optical markers; the optical markers being rigidly connected to each other; the optical markers comprising at least one of luminescent microspheres and reflective microspheres;

[0040] An optical navigator is used to capture marker components; the optical navigator includes one or more of a depth camera, an infrared camera, a visible light camera, or a near-infrared camera.

[0041] The navigation target is fixedly connected to the marking component; the navigation target includes one or more of the following: robotic arm moving carrier, robotic arm end effector, surgical instrument, registration probe, target site, and registration target;

[0042] The display unit is connected to the processing unit and is used to display scene guidance information to guide the user to input navigation target information; the display unit is also used to display one or more of the following: target images, location information, geometric feature information of marker components, and identification serial numbers of optical markers;

[0043] The processing unit is connected to the optical navigation device and includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0044] Fifthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0045] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0046] The aforementioned registration method, surgical navigation system spatial registration method, and registration device acquire and process multiple target images from a target image sequence output by an optical navigator based on optical markers in the field of view. This allows them to obtain the positional information of each optical marker in the target image. Based on the positional information in the target image sequence, geometric feature information between each optical marker can be generated, eliminating the need for time-consuming offline measurement of the geometric feature information of the marker components, as well as the need for loading and importing the geometric feature information of the marker components. By dividing each optical marker according to the geometric feature information between them, the recognition results of each marker component can be obtained, without manually specifying the marker components or manually matching the marker components with pre-acquired geometric feature information. Based on the recognition results, the marker components can be registered. This registration method does not rely on user pre-measurement or system loading and import of the geometric feature information of the marker components. It only requires processing multiple target images output by the optical navigator to complete the acquisition of geometric feature information between optical markers, the recognition of each marker component, and the registration process. This significantly improves the registration speed, greatly simplifies the registration process and operations, improves system efficiency, streamlines the system workflow, and makes the registration process simple, fast, and efficient. On the other hand, the aforementioned registration method eliminates the traditional process of acquiring geometric feature information of the marker components using a measurement system, fundamentally avoiding the accumulation of systematic errors between the measurement system and the optical surgical navigation system. The optical surgical navigation system can also be used for the navigation process; using the same system for registration and navigation improves the accuracy of applying the registration method to the optical surgical navigation system. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the registration method in one embodiment;

[0048] Figure 2 This is a schematic diagram of the structure of the marking component in one embodiment;

[0049] Figure 3 This is a flowchart illustrating the registration steps in one embodiment;

[0050] Figure 4 This is a flowchart illustrating the registration steps in another embodiment;

[0051] Figure 5 This is a flowchart illustrating the registration steps in yet another embodiment;

[0052] Figure 6 This is a flowchart illustrating the registration method in another embodiment;

[0053] Figure 7 This is a flowchart illustrating the registration method in yet another embodiment;

[0054] Figure 8 A flowchart illustrating the process of registering a workflow for a labeled component and registering a navigation workflow in an example;

[0055] Figure 9 This is a structural block diagram of the registration device in one embodiment;

[0056] Figure 10 This is a schematic diagram of the structure of an optical surgical navigation system in one embodiment;

[0057] Figure 11 This is an internal structural diagram of the processing unit in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] Currently, the registration of navigation arrays often requires obtaining specific geometric information of the navigation array as prior knowledge. Only after registration can the navigation array be accurately and stably identified and navigated. In existing systems, this "prior knowledge" is usually measured by a coordinate measuring machine or other measuring device and pre-loaded into the navigation system in the form of a configuration file. The system then navigates the navigation array and the navigation targets connected to it. This process of obtaining specific geometry offline and registering the navigable target in the system is usually called "offline registration." Offline registration typically involves extremely time-consuming offline measurement work, and the errors of the offline measurement system inevitably contribute to the navigation results. Furthermore, the registration results are difficult to maintain high stability; that is, deformation of the navigable target or other factors that alter its specific geometry before the navigation system is used cannot be effectively avoided or eliminated. This application proposes a registration method, a spatial registration method for a surgical navigation system, a registration device, an optical surgical navigation system, a storage medium, and a computer program product that can simplify the registration process and improve the efficiency of navigation array registration.

[0060] In one embodiment, such as Figure 1 As shown, this application provides a registration method applied to an optical surgical navigation system; the optical surgical navigation system includes an optical navigator and one or more marker components; wherein the optical navigator is used to capture the marker components; the marker components include multiple optical markers, the relative positions of the optical markers remaining unchanged; the method includes:

[0061] Step 110: Obtain the target image sequence; the target image sequence includes multiple target images output by the optical navigator based on optical markers in the field of view;

[0062] Specifically, the marking component may include multiple optical markers with fixed relative positions. For example, the optical markers may include at least one of luminescent spheres and reflective spheres. The marking component may be used to fixally connect to the navigation target. The navigation target may include one or more of a robotic arm moving carrier, a robotic arm end effector, a surgical instrument, a registration probe, a target site, and a registration target. The optical navigator may capture optical markers in its field of view and output multiple target images. It may acquire a target image sequence, which may include multiple target images. The target image sequence may be acquired during pre-shipment testing or during post-shipment pre-operative or surgical procedures. When an optical marker appears in the field of view of the optical navigator, the target image sequence can be acquired and output, and the registration process can begin.

[0063] In some examples, the optical navigator may include an optical navigation camera, such as one or more of a depth camera, an infrared camera, a visible light camera, or a near-infrared camera, wherein the camera can be one or more of a monocular camera, a binocular camera, a tricular camera, and a multi-view camera; the marking component can be, for example, Figure 2 The illustrated optical navigation array may include four reflective spheres with a reflective coating on their surfaces, rigidly connected to each other. For example, the four reflective spheres are arranged on a rigid support with specific geometric dimensions. The target image sequence may include multiple frames of target images acquired by the optical navigator when the marker components move relative to the optical navigator. For example, the marker components may be manually or automatically moved relative to the optical navigator within its field of view for a period of time, during which the optical navigator can acquire multiple frames of target images. The relative movement between the marker components and the optical navigator may include movement or rotation along regular or irregular paths, and the period of relative movement can be any time. The geometric feature information between the optical markers may include their relative positions, such as their relative coordinates.

[0064] Step 120: Process the target image to obtain the position information of each optical marker in the target image;

[0065] Specifically, processing the target image can obtain the position information of each optical marker in the target image. The position information can include the position information of each optical marker relative to the optical navigator, or the position information of each optical marker relative to other reference points.

[0066] In some examples, location information may include the three-dimensional coordinates of each optical marker relative to the optical navigator.

[0067] Step 130: Based on the positional information in the target image sequence, generate geometric feature information between each optical marker;

[0068] Specifically, geometric feature information between optical markers in a target image can be generated based on their positional information. For example, for each target image in a target image sequence, geometric feature information between optical markers in that target image can be generated separately. Furthermore, for a target image sequence, if multiple marker components are captured within the field of view of the optical navigator, then the target image includes optical markers of multiple marker components. During the acquisition of multiple target images by the optical navigator, the relative positions between the multiple marker components change, and the relative positions between optical markers belonging to different marker components also change. However, the relative positions between optical markers belonging to the same marker component remain unchanged in the target image sequence. Therefore, geometric feature information between optical markers can be generated based on their positional information in the target image sequence. This geometric feature information can include both the geometric feature information between optical markers that remains unchanged in the target image sequence and the geometric feature information between optical markers that changes with the target image in the target image sequence.

[0069] Step 140: Based on the geometric feature information between each optical marker, divide each optical marker into its constituent parts to obtain the recognition results of each marker component;

[0070] Specifically, based on the geometric feature information between optical markers, the geometric feature information between one or more sets of optical markers that remain unchanged in the target image sequence can be divided into a corresponding number of marker components. That is, the geometric feature information between each set of optical markers that remains unchanged in the target image sequence is determined as a marker component, and then the recognition result of each marker component is obtained. Each marker component can be distinguished based on the geometric feature information between optical markers belonging to the same marker component.

[0071] In some examples, the identification results may include geometric feature information of the marker components, which includes geometric feature information between the various optical markers of the marker components; the geometric feature information of the marker components can be used to distinguish different marker components.

[0072] Step 150: Register the tagging component based on the recognition results.

[0073] Specifically, the marker components can be registered based on the recognition results. Registration may include storing and registering the recognition results of each marker component, so that the optical navigator can recognize each marker component during navigation.

[0074] In this embodiment, by acquiring and processing multiple target images from a target image sequence output by an optical navigator based on optical markers in the field of view, the position information of each optical marker in the target image can be obtained. Based on the position information in the target image sequence, geometric feature information between each optical marker can be generated, eliminating the need for time-consuming offline measurement of the geometric feature information of the marker components, as well as the need for loading and importing the geometric feature information of the marker components. According to the geometric feature information between each optical marker, the optical markers are divided, and the recognition results of each marker component can be obtained, eliminating the need to manually specify the marker components or manually match the marker components with pre-acquired geometric feature information. Based on the recognition results, the marker components can be registered. The registration method provided in this embodiment does not rely on the user's pre-measurement or the system's loading and import of the geometric feature information of the marker components. It only requires processing multiple target images output by the optical navigator to complete the acquisition of geometric feature information between each optical marker, the recognition of each marker component, and the registration process. This significantly improves the registration speed, greatly simplifies the registration process and operations, improves system efficiency, streamlines the system workflow, and makes the registration process simple, fast, and efficient. On the other hand, the registration method of this application eliminates the traditional process of acquiring geometric feature information of the marker component using a measurement system, fundamentally avoiding the accumulation of systematic errors between the measurement system and the optical surgical navigation system. The optical surgical navigation system can also be used in the navigation process; using the same system for registration and navigation improves the accuracy of the registration method applied to the optical surgical navigation system.

[0075] In one embodiment, such as Figure 3 As shown, the steps for dividing each optical marker based on its geometric features to obtain the recognition results of each marker component include:

[0076] Step 310: Based on the geometric feature information between each optical marker, obtain the segmentation result for the optical marker; the geometric feature information between each optical marker includes the relative positional relationship between each optical marker in the target image sequence; the segmentation result includes one or more groups of optical markers in the target image sequence whose relative positional relationship remains unchanged;

[0077] Step 320: Based on the segmentation results, obtain the recognition results of each marker component; the recognition results include the geometric feature information of the marker component.

[0078] Specifically, the geometric feature information between each optical marker can include the relative positional relationship between each optical marker in the target image sequence. This relative positional relationship can include the relative positional relationship between one or more groups of optical markers that remain unchanged in the target image sequence. Then, based on the geometric feature information between each optical marker, a segmentation result for the optical markers can be obtained. This segmentation result can include one or more groups of optical markers in the target image sequence whose relative positional relationship remains unchanged. Based on the segmentation result, the recognition result of each marker component can be obtained. For example, based on one or more groups of optical markers in the target image sequence whose relative positional relationship remains unchanged, a corresponding number of marker components can be obtained. The recognition result of the marker component can include the geometric feature information of the marker component, which can include the geometric feature information between each optical marker of the marker component. By segmenting the geometric feature information between each optical marker, the optical markers of different marker components can be segmented, enabling the recognition of different marker components and the acquisition of the geometric feature information of the marker components, thus preparing for the registration of the marker components.

[0079] In some examples, the geometric feature information of the marker component may include at least one of the following: the Euclidean distance between any two optical marker pairs, the angle between the lines connecting the pairs, and the Euclidean distance difference between the pairs.

[0080] In one embodiment, such as Figure 4 As shown, the recognition results for each marker component also include:

[0081] Step 410: Based on the geometric feature information of the marker component and the component registry, query the registration status of the marker component;

[0082] Step 420: If the registration status of the tag component is unregistered, add a new entry for the tag component in the component registry; the entry information includes the identification number of the tag component and the geometric feature information of the tag component.

[0083] Specifically, based on the geometric feature information of the marker components and the component registry, the registration status of the marker components can be queried. The geometric feature information of the marker components can be obtained based on the segmentation results, and the component registry can be a pre-stored registry. The component registry can include entries for registered marker components. If a marker component's registration status is unregistered, a new entry for the marker component can be added to the component registry. The entry information can include the marker component's identification number and geometric feature information. The identification number can be generated by encoding the marker component (e.g., random encoding) to distinguish different marker components, and the identification number can correspond one-to-one with the geometric feature information of the marker component. By adding entries for unregistered marker components to the component registry, real-time registration of marker components captured by the optical navigator can be facilitated.

[0084] In one embodiment, obtaining the recognition results of each marker component further includes:

[0085] If the registration status of the tag component is "registered", then the geometric feature information of the tag component is matched with the component registry to obtain the entry information of the tag component.

[0086] Specifically, if a marker component's registration status is "registered," it indicates that the component registry contains an entry for that marker component. This entry information may include the marker component's geometric features and its identification number. By matching the marker component's geometric features with the component registry and retrieving and reading the marker component's entry information, the marker component's entry information can be obtained. By matching the geometric features of already registered marker components with the component registry, duplicate registration of marker components can be avoided, and the entry information of already registered marker components can be quickly retrieved.

[0087] In some examples, the geometric features of the marker components can also be used to encode the individual optical markers within the marker components. Obtaining the entry information for the marker components can be used to prompt the user with basic information about the marker components and guide the user to confirm this information.

[0088] In one embodiment, the entry information also includes a component configuration file; the component configuration file is generated based on the geometric feature information of the tagged components;

[0089] Based on the identification results, the steps for registering the tagged components include:

[0090] Register the marked components according to the component configuration file.

[0091] Specifically, if the registration status of a marker component is unregistered, a new entry for the marker component can be added to the component registry. This entry can include a component configuration file generated based on the geometric feature information of the marker component. The component configuration file can then be loaded into the optical surgical navigation system to register the marker component. By generating the component configuration file, the geometric feature information of the marker component can be converted into a file format that is easy for the optical surgical navigation system to load and read, thus completing the registration of the marker component.

[0092] In some examples, component configuration files may include files that require configuration file formatting when loaded in an Optical Tracking System (OTS).

[0093] In one embodiment, such as Figure 5 As shown, the method also includes:

[0094] Step 510: If the entry information of the marker component does not contain navigation target information, then based on the entry information, output application object prompt information for the marker component; the application object prompt information is used to guide the user to input navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to.

[0095] Step 520: If navigation target information is received, add navigation target information to the entry information;

[0096] Step 530: If the entry information of the marker component contains navigation target information, then the navigation target information is displayed.

[0097] Specifically, if the entry information of the marker component does not include navigation target information, application object prompts can be output based on the entry information. These prompts guide the user to input navigation target information for the marker component. Based on this navigation target information, spatial registration of the navigation target can be performed. While ensuring a fixed connection between the marker component and the navigation target, the relative pose increment of the navigation target can be obtained through the relative pose increment of the marker component. The navigation target can include one or more of the following: a robotic arm moving carrier (e.g., a robotic arm cart), a robotic arm end effector, surgical instruments, registration probes (e.g., handheld registration probes), target sites (e.g., patient bony structures), and registration targets. By adding navigation target information to the entry information, spatial registration of the navigation target can be performed, preparing the optical surgical navigation system for the navigation process.

[0098] In some examples, navigation target information may include a 3D model of the navigation target for spatial registration. If the marker component is rigidly connected to the robotic arm trolley, the navigation target information can be used to obtain the spatial pose of the robotic arm. If the marker component is fixedly connected to the end effector of the robotic arm, the navigation target information can be used to obtain the spatial pose of the end effector during spatial registration of the robotic arm. If the marker component is fixedly connected to a handheld registration probe, the navigation target information can be used for selecting bony or surface feature points of the patient during contact registration, and for selecting feature points on the calibration and verification fixture during instrument calibration and verification. For example, after fixing the marker component at the target site, a medical image scan is performed, and then the marker component is extracted from the image to obtain the coordinates of the feature points at the target site. The coordinate system is then registered using these feature points. If the marker component is fixedly connected to the patient's bony structure, the navigation target information can be used for compensating for and tracking the displacement of the patient's bony tissue during robot-assisted surgery. If the marker component is rigidly connected to the registration target, the navigation target information can be used to obtain the spatial pose of the registration target during image registration.

[0099] In one embodiment, such as Figure 6 As shown, the method also includes:

[0100] Step 610: If a user input instruction is received, then user input information is obtained; the user input instruction includes at least one of adding information to the entry information of the marker component and overwriting information on the entry information of the marker component; the user input information includes navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to.

[0101] Step 620: Update the entry information of the tag component based on the user input instructions and user input information.

[0102] Specifically, users can trigger input commands via buttons, touchscreens, etc. These commands can include at least one of the following: adding information to the entry information of the marker component, or overwriting existing entry information. If the optical surgical navigation system receives a user input command, it acquires the user input information, which may include navigation target information representing the navigation target connected to the marker component. If the user input command is to add information to the entry information of the marker component, the acquired user input information can be added to the entry information of the marker component. If the user input command is to overwrite existing entry information of the marker component, the acquired user input information can be overwritten onto the corresponding entry in the entry information of the marker component. By acquiring user input information, the entry information of the marker component can be manually modified in real time to ensure the updated status of the entry information.

[0103] In one embodiment, such as Figure 7 As shown, this application provides a spatial registration method for a surgical navigation system that includes the above-described registration method, the method further comprising:

[0104] Step 710: Obtain entry information for registered tag components;

[0105] Step 720: Based on the entry information of the registered tag components, perform spatial registration of the tag components.

[0106] Specifically, the entry information of the registered marker components can be obtained, and the registered marker components can be spatially registered based on the entry information of the registered marker components. The spatial registration can include establishing the positional relationship between the initial position coordinate system of the marker components and the coordinate system of the optical navigator.

[0107] In some examples, if the marking component is fixedly connected to the robotic arm, a transformation relationship can also be established between the initial position coordinate system of the marking component, the coordinate system of the optical navigator, and the coordinate system of the robotic arm.

[0108] In some examples, such as Figure 8As shown in (a), this is the marker component registration workflow, which can be the initial step in the navigation registration workflow. During marker component registration, the navigation marker component and the navigation target can first be assembled, for example, by making a fixed or rigid connection between them; the marker component is then adjusted to enter the field of view of the optical navigator. The optical surgical navigation system can identify the marker component, as detailed in steps 110 to 140; the identified marker component is matched in the component registry. If the marker component is not registered, it is added to the registry; if it is already registered, the successfully matched registered marker component is called; in the optical surgical navigation system, the marker component configuration file is loaded, which can be a configuration file generated based on the geometric feature information of the marker component. Figure 8 (b) shows the registration navigation workflow. After the registration of the marker components is completed, the robotic arm, target body part, and surgical instruments can be registered separately to perform surgical navigation. At the same time, the information of the registered navigation targets and the connected marker components is displayed to provide feedback on the position status of the navigation targets and the registration results of the marker components. In common navigation application scenarios, only the relative pose increment of the navigation target needs to be considered. In this case, it is not necessary to input the navigation target parameters in advance. On the basis of ensuring the rigid connection between the marker components and the navigation target, the relative pose increment of the navigation target can be obtained based on the relative pose increment of the marker components to achieve navigation of the navigation target.

[0109] In some examples, the recognition results of each marker component can be obtained in real time during navigation, and the geometric feature information of each marker component can be updated in real time to improve navigation accuracy.

[0110] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0111] Based on the same inventive concept, this application also provides a registration apparatus for implementing the registration method or surgical navigation system spatial registration method described above. The solution provided by this apparatus is similar to the implementation described in the above methods; therefore, the specific limitations in one or more registration apparatus embodiments provided below can be found in the limitations of the registration method and the surgical navigation system spatial registration method described above, and will not be repeated here.

[0112] In one embodiment, such as Figure 9 As shown, a registration device is provided, which is applied to an optical surgical navigation system; the optical surgical navigation system includes an optical navigator and one or more marker components; wherein, the optical navigator is used to capture the marker components; the marker components include a plurality of optical markers, the relative positions of the optical markers remaining unchanged; the device includes:

[0113] The image acquisition module 910 is used to acquire a target image sequence; the target image sequence includes multiple target images output by the optical navigator based on optical markers in the field of view;

[0114] The location acquisition module 920 is used to process the target image and obtain the location information of each optical marker in the target image;

[0115] The marker recognition module 930 is used to generate geometric feature information between optical markers based on the positional information in the target image sequence;

[0116] The component segmentation module 940 is used to segment each optical marker based on the geometric feature information between each optical marker to obtain the recognition result of each marker component;

[0117] The component registration module 950 is used to register the tagged components based on the recognition results.

[0118] In one embodiment, the component partitioning module 940 includes:

[0119] The segmentation result acquisition module is used to obtain the segmentation result for the optical markers based on the geometric feature information between each optical marker; the geometric feature information between each optical marker includes the relative positional relationship between each optical marker in the target image sequence; the segmentation result includes one or more groups of optical markers in the target image sequence whose relative positional relationship remains unchanged;

[0120] The recognition result acquisition module is used to obtain the recognition results of each labeled component based on the segmentation results; the recognition results include the geometric feature information of the labeled components.

[0121] In one embodiment, the identification result acquisition module is further configured to query the registration status of the marker component based on the geometric feature information of the marker component and the component registry; if the registration status of the marker component is unregistered, then add an entry information of the marker component in the component registry; the entry information includes the identification serial number of the marker component and the geometric feature information of the marker component.

[0122] In one embodiment, the identification result acquisition module is further configured to, if the registration status of the tagging component is registered, match the geometric feature information of the tagging component with the component registry to obtain the entry information of the tagging component.

[0123] In one embodiment, the entry information also includes a component configuration file; the component configuration file is generated based on the geometric feature information of the marked component; the component registration module 950 is also used to register the marked component according to the component configuration file.

[0124] In one embodiment, the device further includes a target navigation module, configured to output application object prompt information for the marker component based on the entry information if the entry information of the marker component does not contain navigation target information; the application object prompt information is used to guide the user to input navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to; if navigation target information is received, the navigation target information is added to the entry information; if the entry information of the marker component contains navigation target information, the navigation target information is displayed.

[0125] In one embodiment, the apparatus further includes an entry information update module, configured to acquire user input information if a user input instruction is received; the user input instruction includes at least one of adding information to the entry information of the marker component and overwriting information on the entry information of the marker component; the user input information includes navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to; and the entry information of the marker component is updated based on the user input instruction and the user input information.

[0126] Each module in the aforementioned registration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0127] In one embodiment, such as Figure 6 As shown, an optical surgical navigation system is provided, the optical surgical navigation system comprising:

[0128] A marking assembly comprising a plurality of optical markers; the optical markers being rigidly connected to each other; the optical markers comprising at least one of luminescent microspheres and reflective microspheres;

[0129] An optical navigator is used to capture marker components; the optical navigator includes one or more of a depth camera, an infrared camera, a visible light camera, or a near-infrared camera.

[0130] The navigation target is fixedly connected to the marking component; the navigation target includes one or more of the following: robotic arm moving carrier, robotic arm end effector, surgical instrument, registration probe, target site, and registration target;

[0131] The display unit is connected to the processing unit and is used to display scene guidance information to guide the user to input navigation target information; the display unit is also used to display one or more of the following: target images, location information, geometric feature information of marker components, and identification serial numbers of optical markers;

[0132] The processing unit is connected to the optical navigation device and includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0133] Specifically, the distribution of system components such as the optical navigator 1, display unit 2, processing unit 3, and the required support and connection structures includes, but is not limited to, the following: Figure 10 (a) shows an integrated configuration on the same movable carrier, such as Figure 10 The optical navigation device shown in (b) is a split configuration where the display unit and processing unit are integrated on one movable carrier and on another movable carrier, as well as... Figure 10 (c) shows any of the following configurations: an optical navigator, a display unit, and a processing unit, each separately distributed in a multi-body configuration. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an e-ink display screen. The input device for the processing unit can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad mounted on the processing unit's housing, or an external keyboard, touchpad, or mouse, etc.

[0134] In one embodiment, a processing unit is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the processing unit includes a processor, memory, input / output interface, communication interface, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a registration method or a spatial registration method for a surgical navigation system.

[0135] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0136] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0137] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0139] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A registration method, characterized in that, The method is applied to an optical surgical navigation system; the optical surgical navigation system includes an optical navigator and one or more marker components; wherein the optical navigator is used to capture the marker components; the marker components include a plurality of optical markers, the relative positions of the optical markers remaining unchanged; the method includes: Acquire a target image sequence; the target image sequence includes multiple target images output by the optical navigator based on the optical markers in the field of view; The target image is processed to obtain the position information of each optical marker in the target image; Based on the location information in the target image sequence, geometric feature information between the optical markers is generated. Based on the geometric feature information between each optical marker, each optical marker is divided to obtain a division result for the optical marker. Based on the division result, the recognition result of each marker component is obtained. The division result includes one or more groups of optical markers whose relative positional relationship remains unchanged in the target image sequence. Based on the identification results, the tagging component is registered.

2. The method according to claim 1, characterized in that, The step of dividing the optical markers according to the geometric feature information between them to obtain a division result for the optical markers, and obtaining the recognition result of each marker component based on the division result, includes: Based on the geometric feature information between each optical marker, a segmentation result for the optical marker is obtained; the geometric feature information between each optical marker includes the relative positional relationship between each optical marker in the target image sequence; Based on the segmentation results, the recognition results of each of the marked components are obtained; the recognition results include the geometric feature information of the marked components.

3. The method according to claim 2, characterized in that, The method of obtaining the recognition results of each of the marked components also includes: Based on the geometric feature information of the marker component and the component registry, query the registration status of the marker component; If the registration status of the tagging component is unregistered, then a new entry for the tagging component is added to the component registry; the entry information includes the identification number of the tagging component and the geometric feature information of the tagging component.

4. The method according to claim 3, characterized in that, The method of obtaining the recognition results of each of the marked components also includes: If the registration status of the marker component is "registered", then the geometric feature information of the marker component is matched with the component registry to obtain the entry information of the marker component.

5. The method according to claim 3, characterized in that, The entry information also includes a component configuration file; the component configuration file is generated based on the geometric feature information of the marked component; The step of registering the marker component based on the identification result includes: The tag component is registered according to the component configuration file.

6. The method according to claim 3 or 4, characterized in that, The method further includes: If the entry information of the marker component does not contain navigation target information, then based on the entry information, application object prompt information for the marker component is output; the application object prompt information is used to guide the user to input the navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to; If the navigation target information is received, then the navigation target information is added to the entry information; If the entry information of the marker component contains the navigation target information, then the navigation target information is displayed.

7. The method according to claim 3 or 4, characterized in that, The method further includes: If a user input instruction is received, then user input information is acquired; the user input instruction includes at least one of adding information to the entry information of the marker component and overwriting information in the entry information of the marker component; the user input information includes navigation target information; the navigation target information is used to characterize the navigation target that the marker component is used to connect to; based on the user input instruction and the user input information, the entry information of the marker component is updated.

8. The method according to claim 1, characterized in that, The target image sequence includes multiple frames of the target image acquired by the optical navigator while the marker component is in relative motion with the optical navigator.

9. A spatial registration method for a surgical navigation system comprising the registration method according to any one of claims 1-8, characterized in that, The method further includes: Retrieve entry information for registered tag components; Based on the entry information of the registered tag components, spatial registration is performed on the tag components.

10. A registration device, characterized in that, The device is used in an optical surgical navigation system; the optical surgical navigation system includes an optical navigator and one or more marker components; wherein the optical navigator is used to capture the marker components; the marker components include a plurality of optical markers, the relative positions of the optical markers remaining constant; the device includes: An image acquisition module is used to acquire a target image sequence; the target image sequence includes multiple target images output by the optical navigator based on the optical markers in the field of view; A location acquisition module is used to process the target image and obtain the location information of each optical marker in the target image; The marker recognition module is used to generate geometric feature information between each optical marker based on the position information in the target image sequence; The component segmentation module is used to segment each optical marker according to the geometric feature information between each optical marker, to obtain a segmentation result for the optical marker, and to obtain the recognition result of each marker component based on the segmentation result; the segmentation result includes one or more groups of optical markers whose relative positional relationship remains unchanged in the target image sequence; The component registration module is used to register the marked component based on the recognition result.

11. An optical surgical navigation system, characterized in that, The optical surgical navigation system includes: A marking assembly comprising a plurality of optical markers; the optical markers being rigidly connected to each other; the optical markers comprising at least one of luminescent microspheres and reflective microspheres; An optical navigator for capturing the marker component; the optical navigator includes one or more of a depth camera, an infrared camera, a visible light camera, or a near-infrared camera. A navigation target, which is fixedly connected to the marking component; the navigation target includes one or more of the following: a robotic arm moving carrier, a robotic arm end effector, a surgical instrument, a registration probe, a target site, and a registration target. The display unit is connected to the processing unit and is used to display scene guidance information to guide the user to input navigation target information; the display unit is also used to display one or more of the following: each target image, each location information, geometric feature information of the marker component, and identification serial number of each optical marker. A processing unit connected to the optical navigation device includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images