Optical-magnetic integrated marker, positioning method, system and computer device

Abstract

The application relates to an optical-magnetic integrated marker, an optical-magnetic integrated marker device and a marker positioning method. The method comprises an optical marker, an electromagnetic marker and a limiting space. The optical marker is provided with the limiting space; the electromagnetic marker is arranged in the limiting space at least in part, and the relative position of the electromagnetic marker and the optical marker is fixed. The method can effectively compensate for respective defects, deeply improve the overall performance, and effectively improve the accuracy of obtaining absolute physical coordinates and attitude information of the marker.

Description

Technical Field

[0001] This application relates to the field of medical navigation technology, and in particular to an optical-magnetic integrated marker, a positioning method, a system, a computer device, and a computer-readable storage medium. Background Technology

[0002] With the development of medical imaging technology, surgical navigation system technology has emerged. Currently, the most common surgical navigation systems are optical navigation systems (OTS) based on binocular vision and electromagnetic navigation systems (EMTS) based on time-varying magnetic field detection.

[0003] Traditional surgical navigation systems (OTS) based on binocular vision suffer from occlusion problems during surgery. Whether active or passive optical positioning, the camera must have a direct view of the ball marker; if optical occlusion exists, tracking cannot continue. Alternatively, electromagnetic navigation systems (EMTS) based on time-varying magnetic field detection are easily affected by metals and magnetic materials during surgery, leading to significant deviations. These traditional surgical navigation technologies result in low accuracy in acquiring the absolute physical coordinates and attitude information of the marker. Summary of the Invention

[0004] Based on this, it is necessary to provide an integrated optical and magnetic marker, positioning method, system, computer device, and computer-readable storage medium that can improve the accuracy of acquiring the absolute physical coordinates and attitude information of the marker, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides an integrated optical and magnetic marker. The marker includes:

[0006] An optical marker having a finite space;

[0007] An electromagnetic marker, at least a portion of which is disposed within the limiting space, and the electromagnetic marker and the optical marker are fixed in relative position.

[0008] In one embodiment, the electromagnetic marker includes:

[0009] An electromagnetic sensor is fixed in position relative to the optical marker within the confined space, and the electromagnetic sensor is used to acquire magnetic field information in a magnetic field environment.

[0010] A data transmission device is fixedly connected to the electromagnetic sensor and is used to transmit the magnetic field information to an external device.

[0011] In one embodiment, the optical marker is spherical, the limiting space passes through the center of the optical marker, and the electromagnetic sensor is limited to the position at the center of the sphere.

[0012] In one embodiment, the optical marker includes a first component and a second component that are detachably connected, and the first component and the second component are connected in a mating state to form the limiting space.

[0013] Secondly, this application provides a positioning method. The method includes:

[0014] Optical and magnetic field information is acquired by providing light and magnetic fields to multiple markers; wherein each marker is an integrated optical-magnetic marker and each marker is used to be placed on the target object;

[0015] The location information of the target object is determined based on the optical information and the magnetic field information.

[0016] In one embodiment, the optical information includes multiple optical coordinates, and the magnetic field information includes multiple magnetic field coordinates; determining the position information of the target object based on the optical information and the magnetic field information includes:

[0017] The optical information is matched with the magnetic field information to obtain a matching result;

[0018] Based on the matching results, target information is determined from the optical information and the magnetic field information;

[0019] Based on the target information, the location information of the target object is determined.

[0020] In one embodiment, matching the optical information with the magnetic field information to obtain a matching result includes:

[0021] The quantity and position of each optical coordinate and each magnetic field coordinate are matched to determine the quantity matching information and multiple sets of coordinate pairs; each coordinate pair includes one optical coordinate and one magnetic field coordinate, and the distance between the optical coordinate and the magnetic field coordinate in the world coordinate system is less than a preset threshold.

[0022] In one embodiment, determining the target information from the optical information and the magnetic field information based on the matching result includes:

[0023] If the quantity matching information is that the number of optical coordinates is greater than the number of magnetic field coordinates, then each of the magnetic field coordinates is taken as the target information;

[0024] or,

[0025] The optical coordinates in each of the coordinate pairs are used as the target information.

[0026] In one embodiment, determining the target information from the optical information and the magnetic field information based on the matching result includes:

[0027] If the quantity matching information is that the number of optical coordinates is less than the number of magnetic field coordinates, then each of the magnetic field coordinates is taken as the target information;

[0028] or,

[0029] The remaining magnetic field coordinates and the optical coordinates in each of the coordinate pairs are used as the target information; wherein, the remaining magnetic field coordinates are the magnetic field coordinates remaining after removing the magnetic field coordinates in each coordinate pair from each of the magnetic field coordinates.

[0030] Thirdly, this application also provides a positioning system, the system comprising:

[0031] Multiple optical-magnetic integrated markers, each of which is used to be set on a target object;

[0032] A magnetic field generating device is used to provide a magnetic field to each of the aforementioned optomagnetic integrated markers;

[0033] An optical tracking device is used to provide a light source to each of the aforementioned opto-magnetic integrated markers and to acquire optical information;

[0034] A controller is communicatively connected to the electromagnetic markers in each of the optical-magnetic integrated markers and the optical tracking device, and the controller is used to execute the steps of the positioning method as described above.

[0035] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0036] Optical and magnetic field information is acquired by providing light and magnetic fields to multiple markers; wherein each marker is an integrated optical-magnetic marker and each marker is used to be placed on the target object;

[0037] The location information of the target object is determined based on the optical information and the magnetic field information.

[0038] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0039] Optical and magnetic field information is acquired by providing light and magnetic fields to multiple markers; wherein each marker is an integrated optical-magnetic marker and each marker is used to be placed on the target object;

[0040] The location information of the target object is determined based on the optical information and the magnetic field information.

[0041] The aforementioned optical-magnetic integrated marker, positioning method, system, computer equipment, and computer-readable storage medium, by setting electromagnetic markers in optical markers, allow both a binocular vision optical navigation system and a time-varying magnetic field detection electromagnetic navigation system to track simultaneously, or independently. When both are tracking synchronously, they can more effectively compensate for each other's shortcomings and further improve overall performance, thereby effectively improving the accuracy of obtaining the absolute physical coordinates and attitude information of the markers. Attached Figure Description

[0042] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the structure and composition of an integrated optical and magnetic marker in one embodiment;

[0045] Figure 2 This is a schematic diagram of the optical-magnetic integrated marker interface device in one embodiment;

[0046] Figure 3 This is a schematic diagram of the structure of an integrated optical and magnetic marker device in one embodiment;

[0047] Figure 4 This is a flowchart illustrating a positioning method in one embodiment;

[0048] Figure 5 This is a flowchart illustrating the method for obtaining location information in one embodiment;

[0049] Figure 6 This is a flowchart illustrating a method for obtaining target information where the number of optical coordinates is greater than the number of magnetic field coordinates in one embodiment.

[0050] Figure 7This is a flowchart illustrating a method for providing target information in a number less than the number of magnetic field coordinates in one embodiment.

[0051] Figure 8 This is a schematic diagram of an embodiment of EMTS assisting OTS in identifying a single reflective marker;

[0052] Figure 9 This is a schematic diagram illustrating the relationship between the coordinates of various components in a navigation system in one embodiment;

[0053] Figure 10 This is a structural block diagram of a positioning device in one embodiment;

[0054] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0055] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0056] Traditional surgical navigation and positioning methods using binocular vision-based optical navigation systems (OTS) suffer from occlusion problems during surgery. Whether active or passive, optical positioning requires the camera to have a direct view of the ball marker; if optical occlusion exists, tracking cannot continue. Alternatively, electromagnetic navigation systems (EMTS) based on time-varying magnetic field detection are easily affected by metals and magnetic materials during surgery, leading to significant deviations. Traditional surgical navigation systems result in low accuracy in acquiring the absolute physical coordinates and attitude information of the marker. The optical-magnetic integrated marker 100 of this application, by setting an electromagnetic marker 102 within the optical marker 101, allows for simultaneous tracking by both the binocular vision-based optical navigation system and the time-varying magnetic field detection electromagnetic navigation system, or independent tracking by both. When both are tracking synchronously, it can more effectively compensate for their respective shortcomings, further improving overall performance and significantly increasing the accuracy of acquiring the absolute physical coordinates and attitude information of the marker.

[0057] Based on this Figure 1 Part (a) shows an overall schematic diagram of the optomagnetic integrated marker structure in the first embodiment of this application. Figure 1 Part (b) of the middle and Figure 1Part (c) shows a split schematic diagram of the optical-magnetic integrated marker in the first embodiment of this application; wherein the optical-magnetic integrated marker 100 includes: an optical marker 101, the optical marker 101 having a limiting space; and an electromagnetic marker 102, at least a portion of the electromagnetic marker 102 being disposed within the limiting space, and the position of the electromagnetic marker 102 being relatively fixed relative to the position of the optical marker 101.

[0058] Specifically, the integrated optical-magnetic marker 100 mainly consists of two parts: an optical marker 101 and an electromagnetic marker 102. The optical marker 101 can be a spherical marker with a reflective coating on its surface, made of non-metallic material, with a diameter ranging from 5 to 10 mm. The electromagnetic marker 102 can be composed of a magnetic sensor and necessary wiring connections. The magnetic sensor can be a coil sensor, or a 3-axis analog or digital sensor such as a Hall sensor, AMR sensor, or TMR sensor. For wireless applications, 3-axis AMR or TMR sensors are more suitable, and their size can be less than 1 mm × 1 mm × 1 mm. Furthermore, as... Figure 1 As shown in part (a), the optical marker 101 has a limiting space, and the electromagnetic marker 102 is fixed in the limiting space inside the optical marker 101, such as the limiting space being located at the center of the sphere; at least a portion of the electromagnetic marker 102 is located in the limiting space, and the position of the electromagnetic marker 102 is relatively fixed relative to the position of the optical marker 101.

[0059] The aforementioned optical-magnetic integrated marker, by setting an electromagnetic marker 102 in the optical marker 101, allows both a binocular vision optical navigation system and a time-varying magnetic field detection electromagnetic navigation system to track simultaneously, or to track independently. When the two are tracking synchronously, they can more effectively compensate for each other's shortcomings and further improve the overall performance, thereby effectively improving the accuracy of obtaining the absolute physical coordinates and attitude information of the marker.

[0060] In one embodiment, such as Figure 1 Part (b) and Figure 1 As shown in section (c), the electromagnetic marker 102 includes an electromagnetic sensor 1021 and a data transmission device 1022. The electromagnetic sensor 1021 is fixed relative to the position of the optical marker 101 within the confined space, and is used to acquire magnetic field information in a magnetic field environment. The data transmission device 1022 is fixedly connected to the electromagnetic sensor 1021 and is used to transmit the magnetic field information to an external device. Figure 1 Part (b) and Figure 1As shown in part (c), the upper half of the optical marker 101 includes a first component 1011 and an upper half limiting space 1012, and the lower half of the optical marker 101 includes a second component 1013, a lower half limiting space 1014, and a key 1015. The upper half limiting space 1012 and the lower half limiting space 1014 together fix the electromagnetic sensor 1021, while the key 1015 partially matches the upper half limiting space 1012 and serves to fix the data transmission component 1022.

[0061] In one embodiment, such as Figure 1 Part (b) and Figure 1 As shown in section (c), the optical marker 101 is spherical, and the limiting space passes through the center of the optical marker 101, with the electromagnetic sensor 1021 positioned at the center. The center of the optical marker 101 overlaps with one end of the limiting space, and the center of the limiting space may or may not overlap with the center of the optical marker 101.

[0062] In one embodiment, such as Figure 2 Part (a) and Figure 2 As shown in part (b), the electromagnetic marker 102 also includes an interface device 200, wherein the interface device 200 is electrically connected to the data transmission device 1022 and is used to transmit magnetic field information to an external device.

[0063] In one embodiment, such as Figure 2 As shown in part (a), a portion of the interface device 200 extends into the limiting space and is electrically connected to the data transmission element 1022 within the limiting space, while another portion of the interface device extends out of the limiting space for electrical connection with an external device. Figure 2 As shown in section (a), the optical marker 101 has a protruding interface device 200 (plug).

[0064] In one embodiment, the interface device 200 (jack) is disposed within a limiting space, and one end of the interface device 200 (jack) is connected 1022 to a data transmission device within the limiting space, while the other end of the interface device 200 (jack) is used for electrical connection with an external device. For example... Figure 2 As shown in part (b), the optical marker 101 has a small interface device 200 (jack), and external transmission can be electrically connected to the interface device 200 (jack) of the ball via pins, plugs, etc.

[0065] In one embodiment, the optical marker 101 includes a first component 1011 and a second component 1012 that are detachably connected, and the first component 1011 and the second component 1012 form a limiting space when connected. Figure 1 Part (b) and Figure 1 As shown in part (c), the upper half of the optical marker 101 includes a first component 1011 and an upper half limiting space 1012, and the lower half of the optical marker 101 includes a second component 1013, a lower half limiting space 1014, and a key 1015.

[0066] In conjunction with the above embodiments, an optical-magnetic integrated marker can be set with an electromagnetic marker in the optical marker, allowing the binocular vision optical navigation system and the time-varying magnetic field detection electromagnetic navigation system to track simultaneously or independently; and when the two are tracking synchronously, it can more effectively make up for their respective deficiencies and further improve the overall performance, thereby effectively improving the accuracy of obtaining the absolute physical coordinates and attitude information of the marker.

[0067] In one embodiment, such as Figure 3 As shown, an integrated optical-magnetic marker device 300 is provided, comprising a mounting body 301 and multiple integrated optical-magnetic markers 100, each of which is mounted on a different extension end of the mounting body 301. The multiple integrated optical-magnetic markers 100 form a marker array via the mounting body 301. Since the electromagnetic navigation system using time-varying magnetic field detection can assist the binocular vision optical navigation system in identifying individual markers, the integrated optical-magnetic marker device 300 can be in any convenient form, and multiple integrated optical-magnetic marker devices 300 can have the same shape without considering whether the binocular vision optical navigation system can distinguish them.

[0068] In one embodiment, the integrated optical-magnetic marker device 300 further includes a bracket data interface 302, which is disposed on the mounting body 301. The bracket data interface 302 is electrically connected to the electromagnetic marker 102 and is used for electrically connecting to external devices. Further, as... Figure 3 As shown in part (a), the optomagnetic integrated marker device 300 is wired. The data and power supply lines of the electromagnetic markers 102 among the multiple optomagnetic integrated markers 100 are exported through the mounting body 301 and electrically connected to external devices through the bracket data interface 302.

[0069] In one embodiment, the integrated optical-magnetic marker device 300 further includes a device control unit 303, wherein the device control unit 303 is disposed on the mounting body 301 and electrically connected to the electromagnetic marker 102, and the device control unit 303 is used to transmit magnetic field information to an external device. Further, as... Figure 3As shown in part (b), the opto-magnetic integrated marker device 300 is wireless, and the data and power supply lines of the electromagnetic markers 102 among the multiple opto-magnetic integrated markers 100 are electrically connected to the device control unit 303 on the mounting body 301 through the mounting body 301.

[0070] In one embodiment, the device control unit 303 includes a housing 3031, a microcontroller unit 3032, a wireless transmission unit 3033, and a power supply unit 3034. The housing 3031 can form an accommodating space; the microcontroller unit 3032 can be disposed within the accommodating space of the housing 3031 and is electrically connected to the electromagnetic marker 102; the wireless transmission unit 3033 can be disposed within the accommodating space of the housing 3031 and is electrically connected to the microcontroller unit 3032; the wireless transmission unit 3033 is used to transmit the magnetic field information transmitted by the microcontroller unit 3032 to an external device; the power supply unit 3034 can be disposed within the accommodating space of the housing 3031 and is electrically connected to the electromagnetic marker 102, the microcontroller unit 3032, and the wireless transmission unit 3033. Further, as... Figure 3 As shown in section (b), the housing 3031 of the device control unit 303 contains a power supply unit 3034, a microcontroller unit 3032, and a wireless transmission unit 3033. The electromagnetic marker 102, microcontroller unit 3032, and wireless transmission unit 3033 within the integrated opto-magnetic marker 100 are all powered by the internal power supply unit 3034. After data acquisition, it is transmitted to an external device via the wireless transmission unit 3033. Based on current device performance, the size of the device control unit 303 can be controlled within 30mm, allowing the electromagnetic marker 102 to operate continuously for several hours. The power supply unit 3034 can be replaced when its power is insufficient, or, for easier sterilization, can be made into a wireless charging form. The device control unit 303 has better sealing. The wireless form reduces the number of connecting wires, thereby reducing the impact on the movement of medical personnel.

[0071] In conjunction with the above embodiments, the time-varying magnetic field detection electromagnetic navigation system not only enhances the performance of the binocular vision optical navigation system, but also serves as the primary navigation system when the binocular vision optical navigation system is unavailable or obstructed. When both the binocular vision optical navigation system and the time-varying magnetic field detection electromagnetic navigation system operate simultaneously, the binocular vision optical navigation system can also serve as a standard to perform real-time corrections on the time-varying magnetic field detection electromagnetic navigation system, reducing interference from metallic and magnetic materials in the environment. This results in a more powerful and widely applicable navigation system.

[0072] In one embodiment, such as Figure 4As shown, a positioning method is provided, in which the execution subject can be a controller, and the method includes:

[0073] Step 402: Acquire optical and magnetic field information while providing light and magnetic fields to multiple markers.

[0074] The markers can be integrated optical and magnetic markers. Multiple markers are used to set on a target object, which can be a part of the target user's body.

[0075] In the case of providing a light source to multiple markers, the reflective data of optical markers among the markers can be obtained, and optical information can be determined based on the reflective data. The optical information may include optical coordinates, etc.

[0076] In the case of providing a time-varying magnetic field to multiple markers, magnetic field data of electromagnetic markers among the markers can be obtained, and magnetic field information can be determined based on the magnetic field data. The magnetic field information may include magnetic field coordinates, ID number of electromagnetic markers, etc.

[0077] Specifically, a light source and a time-varying magnetic field are provided to multiple markers. The controller sends a detection command to the optical navigation system, controlling the optical navigation system to acquire the optical information of each opto-magnetic integrated marker in the optical coordinate system. Similarly, the controller sends a detection command to the electromagnetic navigation system, controlling the electromagnetic navigation system's magnetic field generator to generate a time-varying magnetic field in the space containing multiple opto-magnetic integrated markers. Furthermore, the controller controls the electromagnetic navigation system to acquire the magnetic field information of each opto-magnetic integrated marker in the time-varying magnetic field. The opto-magnetic integrated markers are used to be placed on the target object to assist in determining the target object's position information.

[0078] Step 404: Determine the location information of the target object based on optical and magnetic field information.

[0079] The location information can be the coordinates of the target object in the world coordinate system.

[0080] Specifically, in the first scenario, if the optical navigation system is unavailable or obstructed, the target object's location can be determined solely using magnetic field information. In the second scenario, if the electromagnetic navigation system is unavailable or subject to ferromagnetic interference, the target object's location can be determined solely using optical information. In the third scenario, if both the optical and electromagnetic navigation systems are functioning normally, both optical and magnetic field information can be used together to determine the target object's location. For example, in the presence of multiple marker arrays (each array containing multiple markers), the ID numbers of the electromagnetic markers in the magnetic field information can be used to determine which markers belong to the same array. Then, based on the optical information of the optical markers within the same array, the location of that array can be determined. Thus, when multiple marker arrays are set up on a single target object, different parts of that target object can be located; when multiple target objects exist, and each target object has a marker array, different target objects can be located.

[0081] In this embodiment, the target object is located using multiple integrated optical and magnetic markers, which can be applied to various scenarios. The optical navigation system and the electromagnetic navigation system can be used individually or simultaneously, improving the applicability and flexibility of the positioning. Specifically, in scenarios where both are used simultaneously, the optical navigation system can utilize information from the electromagnetic navigation system detected by the time-varying magnetic field for single marker identification. This helps to identify and track the target object even in situations where there is no marker array, lack of prior array information, lack of distinguishability between multiple arrays, or incomplete array detection, thus improving the navigation accuracy.

[0082] In some embodiments, optical coordinates are the coordinates of an optical marker in an optical coordinate system; similarly, magnetic field coordinates are the coordinates of an electromagnetic marker in an electromagnetic coordinate system. Generally, the origins of the optical and electromagnetic coordinate systems do not coincide. Therefore, one of the coordinate systems needs to be selected as the world coordinate system. The unselected coordinate system can be converted to the world coordinate system using the transformation relationship between the optical and electromagnetic coordinate systems. Finally, based on the world coordinate system, if the errors between the optical and magnetic field coordinates are both less than a preset threshold, the optical marker corresponding to the optical coordinates and the magnetic field marker corresponding to the magnetic field coordinates can be considered to belong to the same integrated optical-magnetic marker. The position information of the target object can then be obtained based on the optical coordinates and the corresponding magnetic field coordinates.

[0083] In this case, because the distance between the centers of multiple different optical-magnetic integrated markers is not less than the marker scale (when the optical marker is a sphere, the marker size refers to the diameter of the sphere), this order of magnitude is significantly larger than the positioning error of each system. Therefore, for any calibrated optical-magnetic integrated marker, the deviation between the positioning results of EMTS and OTS in the world coordinate system is almost certainly within the marker scale. Thus, a simple method can be used to filter the positioning results and find the correspondence between the entities using a preset threshold, i.e., the marker scale; additionally, such as... Figure 8 As shown in the lower right corner of the right figure. When there are false points in the OTS results, they can also be automatically filtered out using the EMTS results. Therefore, the distance between the coordinate pairs formed by the automatically filtered optical coordinates and magnetic field coordinates in the world coordinate system is less than a preset threshold.

[0084] In one embodiment, such as Figure 5 As shown, the location information of the target object is determined based on optical and magnetic field information, including:

[0085] Step 502: Match the optical information with the magnetic field information to obtain the matching result.

[0086] The optical information includes multiple optical coordinates, and the magnetic field information includes multiple magnetic field coordinates. Matching involves matching the number of optical coordinates with the number of magnetic field coordinates, and matching the position of optical coordinates with the position of magnetic field coordinates. The matching result includes both quantity matching information and position matching information between the optical and magnetic field coordinates.

[0087] The quantity matching information includes the number of optical coordinates being equal to the number of magnetic field coordinates, the number of optical coordinates being greater than the number of magnetic field coordinates, and the number of optical coordinates being less than the number of magnetic field coordinates. The position matching information may include whether each optical coordinate in the world coordinate system has a magnetic field coordinate at a distance less than or equal to a preset threshold, and whether each magnetic field coordinate in the world coordinate system has an optical coordinate at a distance less than or equal to a preset threshold. Furthermore, the position matching information may also include multiple sets of coordinate pairs, each coordinate pair including one optical coordinate and one magnetic field coordinate, and the distance between the optical coordinate and the magnetic field coordinate in the world coordinate system is less than or equal to a preset threshold.

[0088] Step 504: Determine the target information from the optical and magnetic field information based on the matching results.

[0089] Specifically, the methods for determining target information differ depending on the circumstances.

[0090] If the number of optical coordinates is equal to the number of magnetic field coordinates in the matching result, and each optical coordinate has a magnetic field coordinate pair, and each magnetic field coordinate has an optical coordinate pair, then only the optical coordinates can be selected as the target information, or both the optical coordinates and their corresponding magnetic field coordinates can be selected as the target information.

[0091] In some embodiments, where multiple marker arrays exist in the environment and cannot be distinguished by optical coordinates, magnetic field coordinates can be used to determine which optical coordinates belong to the first array, which optical coordinates belong to the second array, and so on. Then, the pose information of the first array is calculated using the optical coordinates of the first array, the pose information of the second array is calculated using the optical coordinates of the second array, and so on.

[0092] In some embodiments, since the optical navigation system can only obtain the coordinate information of optical markers, at least three optical markers are needed to obtain 6DoF information in order to achieve high-precision navigation. However, by using integrated optical-magnetic markers, the coordinate information obtained by the optical navigation system and the attitude information obtained by the electromagnetic navigation system can be used to form 6DoF information. Alternatively, the coordinate information obtained by the optical navigation system can be fed into the solver of the electromagnetic navigation system to resolve the attitude information, thereby obtaining 6DoF information. In this way, only one or two integrated optical-magnetic markers are needed to obtain the 6DoF information of the array. Therefore, when the number of optical coordinates used as target information is insufficient to solve the attitude information, magnetic field coordinates can be introduced as target information to complete the array attitude information.

[0093] If the number of optical coordinates in the matching result is greater than the number of magnetic coordinates, this situation may be due to false optical coordinates. To avoid errors caused by this, select magnetic coordinates or optical coordinates in the coordinate pair as target information. If the number of optical coordinates in the matching result is less than the number of magnetic coordinates, this situation may be due to occlusion of optical navigation, making it impossible to obtain optical coordinates. To avoid the inability to locate the target object due to the lack of optical coordinates, use magnetic coordinates as target information, or use the optical coordinates in the coordinate pair and the magnetic coordinates outward from the coordinate pair as target information.

[0094] Step 506: Determine the location information of the target object based on the target information.

[0095] Specifically, when the number of optical coordinates is greater than the number of magnetic field coordinates, one approach is to determine the position information of the target object based on the magnetic field coordinates; another approach is to select the optical coordinates in the coordinate pair to determine the position information of the target object. Furthermore, the magnetic field coordinates can also be used as auxiliary information for the optical coordinates in the coordinate pair to help determine the array information to which the optical coordinates belong, or to provide 6DoF information acquisition when there are fewer optical coordinates. For specific details, please refer to the relevant descriptions in the above embodiments, which will not be repeated in this embodiment.

[0096] When the number of optical coordinates is less than the number of magnetic field coordinates, one method is to determine the position information of the target object based on the magnetic field coordinates; another method is to select the optical coordinates in the coordinate pair and the magnetic field coordinates of the coordinate pair to determine the position information of the target object. Furthermore, the magnetic field coordinates in the coordinate pair can also be used as auxiliary information of the optical coordinates to help determine the array information to which the optical coordinates belong, or to provide 6DoF information acquisition when there are few optical coordinates. For the specific process, please refer to the relevant description in the above embodiments, which will not be repeated in this embodiment.

[0097] In this embodiment, by matching the optical information and magnetic field information, target information for determining the location information of the target object is selected, which can eliminate abnormal data and avoid introducing abnormal data that would cause a large deviation in the positioning and navigation results.

[0098] In one embodiment, the optical information includes multiple optical coordinates, and the magnetic field information includes multiple magnetic field coordinates; matching the optical information and the magnetic field information to obtain a matching result includes:

[0099] The quantity and position of each optical coordinate are matched with those of each magnetic field coordinate to determine the quantity matching information and multiple sets of coordinate pairs.

[0100] Optical coordinates can be information used to represent coordinates in optical information.

[0101] Among them, magnetic field coordinates can be the information used to represent coordinates in magnetic field information.

[0102] Specifically, the number of optical coordinates in the optical information is matched with the number of magnetic field coordinates to obtain quantity matching information; and each optical coordinate in the optical information is matched with each magnetic field coordinate to obtain coordinate pairs.

[0103] The quantity matching information indicates whether the number of optical coordinates and the number of magnetic field coordinates are equal, including whether the number of optical coordinates equals the number of magnetic field coordinates, the number of optical coordinates is greater than the number of magnetic field coordinates, and the number of optical coordinates is less than the number of magnetic field coordinates. If the number of optical coordinates equals the number of magnetic field coordinates, the coordinate pair is obtained with reference to either the number of optical coordinates or the number of magnetic field coordinates; if the number of optical coordinates is less than the number of magnetic field coordinates, the coordinate pair is obtained with reference to the number of optical coordinates; if the number of optical coordinates is greater than the number of magnetic field coordinates, the coordinate pair is obtained with reference to the number of magnetic field coordinates.

[0104] In this embodiment, by determining the relationship between the number of optical coordinates and the number of magnetic field coordinates, and by constructing coordinate pairs of optical coordinates and magnetic field coordinates, the method of using optical coordinates and magnetic field coordinates in the coordinate pairs to determine target information can be adjusted in a timely manner based on the quantity matching information, thereby improving the flexibility of positioning while ensuring the accuracy of target information acquisition.

[0105] In one embodiment, such as Figure 6 As shown, based on the matching results, target information is determined from optical and magnetic field information, including:

[0106] Step 602: If the number of optical coordinates is greater than the number of magnetic field coordinates in the quantity matching information, then each magnetic field coordinate is taken as the target information.

[0107] Specifically, when the quantity matching information indicates that the number of optical coordinates is greater than the number of magnetic field coordinates, the same number of coordinate pairs are generated accordingly, using the number of magnetic field coordinates as a reference. Furthermore, each magnetic field coordinate is extracted as target information for locating the position of the target object.

[0108] or,

[0109] Step 604: Use the optical coordinates in each coordinate pair as target information.

[0110] Specifically, the optical coordinates in each coordinate pair are extracted as target information, which is used to locate the position information of the target object.

[0111] In this embodiment, when the number of optical coordinates is greater than the number of magnetic field coordinates, the positioning can be performed by selecting each magnetic field coordinate or the optical coordinates in each coordinate pair, depending on the specific situation. This is beneficial for accurately locating the target object even when interference occurs in one of the coordinates, thus improving the efficiency of target object positioning.

[0112] In one embodiment, such as Figure 7 As shown, based on the matching results, target information is determined from optical and magnetic field information, including:

[0113] Step 702: If the number of optical coordinates in the quantity matching information is less than the number of magnetic field coordinates, then each magnetic field coordinate is taken as the target information.

[0114] Specifically, when the number of optical coordinates is less than the number of magnetic field coordinates in the quantity matching information, the same number of coordinate pairs are generated accordingly, using the number of optical coordinates as a reference. Furthermore, each magnetic field coordinate is extracted as target information for locating the position of the target object.

[0115] or,

[0116] Step 704: Use the remaining magnetic field coordinates and the optical coordinates in each coordinate pair as target information.

[0117] Specifically, magnetic field coordinates that are the same as those in each coordinate pair are removed, and the remaining magnetic field coordinates are taken as the remaining magnetic field coordinates; optical coordinates are extracted from each coordinate pair, and the union of each optical coordinate and each remaining magnetic field coordinate is taken as the target information.

[0118] In this embodiment, when the number of optical coordinates is less than the number of magnetic coordinates, magnetic coordinates are used as a reference. Positioning can be performed using magnetic coordinates alone or by combining magnetic coordinates and optical coordinates. This allows for comprehensive coverage using magnetic coordinates when optical coordinates cannot provide high-precision navigation. This improves navigation efficiency while ensuring navigation accuracy.

[0119] In one embodiment, for any optical-magnetic integrated marker, the marker needs to be inspected before navigating and locating the target object.

[0120] Specifically, by Figure 9 As shown, this integrated optical-magnetic navigation system comprises three coordinate systems: an optical coordinate system {T} for binocular vision, an electromagnetic coordinate system {E} for time-varying magnetic field detection, and the coordinate system {m} of the electromagnetic marker 102 itself. Other parameters in the figure are explained below:

[0121] The homogeneous transformation matrix of {m} relative to {E} can be obtained through positioning by an electromagnetic navigation system that detects time-varying magnetic fields.

[0122] Denotes the homogeneous transformation matrix of {E} relative to {T};

[0123] m P s The homogeneous coordinates of the center of sphere 101 in {m} can be measured by methods such as DR and CT.

[0124] T Ps The homogeneous coordinates of the center of sphere 101 in {T} can be determined by a binocular vision optical navigation system.

[0125] Among them, the two homogeneous transformation matrices and the two homogeneous coordinates mentioned above must satisfy the following conditions:

[0126] Therefore, if measured in advance by DR, CT, etc. m P s Then it can be calibrated through experiments. This unifies {T} and {E}. Of course, they can also be pre-defined. Next, measure the bias within each integrated optical and magnetic marker. m P s Each opto-magnetic integrated marker undergoes... m P s After calibration, the parameter can be stored in the system and uniquely associated with the entity. Alternatively, each opto-magnetic integrated marker can undergo factory inspection to obtain... m P s .

[0127] when m P s The offset is considered acceptable when the value is less than τ0, where τ0 is the offset inspection threshold, such as 0.02 to 0.10 mm. Since τ0 is significantly less than the accuracy requirements of navigation applications, it can be considered acceptable in subsequent use. If the centers of the electromagnetic marker and the optical marker coincide, then the optical-magnetic integrated marker that meets the accuracy requirements can be regarded as the tested optical-magnetic integrated marker.

[0128] In this embodiment, by performing quality inspection on each optical-magnetic integrated marker of the optical-magnetic integrated marker device before using the device, unqualified optical-magnetic integrated markers can be eliminated before constructing the device, thus avoiding serious data deviations in subsequent use and improving the reliability of the device.

[0129] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0130] Based on the same inventive concept, this application also provides a marker positioning system for implementing the marker positioning method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of the one or more marker positioning system embodiments provided below can be found in the limitations of the marker positioning method described above, and will not be repeated here.

[0131] In one embodiment, a marker positioning system is provided, comprising: the system includes:

[0132] Multiple integrated optical and magnetic markers, each used to be applied to the target object;

[0133] A magnetic field generating device is used to provide a magnetic field to each opto-magnetic integrated marker;

[0134] An optical tracking device is used to provide a light source to each opto-magnetic integrated marker and to acquire optical information;

[0135] The controller is communicatively connected to the electromagnetic markers and optical tracking devices in each of the opto-magnetic integrated markers, and is used to perform steps such as a positioning method.

[0136] The various devices in the aforementioned marker positioning system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0137] In one embodiment, such as Figure 10 As shown, a positioning device is provided, including: a sample data acquisition module 1002 and an image feature fusion module 1004, wherein:

[0138] The data information acquisition module 1002 is used to acquire optical information and magnetic field information when providing light source and magnetic field to multiple markers; wherein each marker is an optical-magnetic integrated marker, and each marker is used to be set on the target object;

[0139] The location information determination module 1004 is used to determine the location information of the target object based on optical information and magnetic field information.

[0140] In one embodiment, the location information determination module 1004 is further configured to match optical information with magnetic field information to obtain a matching result; determine target information from the optical information and magnetic field information based on the matching result; and determine the location information of the target object based on the target information.

[0141] In one embodiment, the location information determination module 1004 is further configured to match the quantity and position of each optical coordinate with each magnetic field coordinate to determine quantity matching information and multiple sets of coordinate pairs; each coordinate pair includes an optical coordinate and a magnetic field coordinate, and the distance between the optical coordinate and the magnetic field coordinate in the world coordinate system is less than a preset threshold.

[0142] In one embodiment, the location information determination module 1004 is further configured to, if the number of optical coordinates in the quantity matching information is greater than the number of magnetic field coordinates, use the magnetic field coordinates in each coordinate pair as the target information; or, use the optical coordinates in each coordinate pair as the target information.

[0143] In one embodiment, the location information determination module 1004 is further configured to, if the number of optical coordinates in the quantity matching information is less than the number of magnetic field coordinates, use the magnetic field coordinates in each coordinate pair as the target information; or, use the remaining magnetic field coordinates and the optical coordinates in each coordinate pair as the target information; wherein, the remaining magnetic field coordinates are the magnetic field coordinates remaining after removing the magnetic field coordinates in each coordinate pair from each magnetic field coordinate.

[0144] The modules in the aforementioned positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0145] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores controller data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a marker positioning method.

[0146] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0148] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0149] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

[0154] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0155] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0156] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0157] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0158] It should be noted that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A photomagnetic integrated marker, characterized in that, The integrated optical and magnetic marker (100) includes: An optical marker (101) has a finite space. An electromagnetic marker (102) is provided at least in the limiting space, and the position of the electromagnetic marker (102) is fixed relative to the position of the optical marker (101). The electromagnetic marker (102) includes: An electromagnetic sensor (1021) is positioned relative to the position of the optical marker (101) within the confined space, and the electromagnetic sensor (1021) is used to acquire magnetic field information in a magnetic field environment. A data transmission device (1022) is fixedly connected to the electromagnetic sensor (1021) and is used to transmit the magnetic field information to an external device.

2. The optomagnetic integrated marker according to claim 1, characterized in that, The optical marker (101) is spherical, the limiting space passes through the center of the optical marker (101), and the electromagnetic sensor (1021) is limited to the position of the center of the sphere.

3. The optomagnetic integrated marker according to claim 1, characterized in that, The optical marker (101) includes a first component (1011) and a second component (1012) that are detachably connected, and the first component (1011) and the second component (1012) form the limiting space when connected.

4. A positioning method, characterized in that, The method includes: Optical and magnetic field information is acquired by providing light and magnetic fields to multiple markers; wherein each marker is an optical-magnetic integrated marker as described in any one of claims 1-3, and each marker is used to be set on a target object; Based on the optical information and the magnetic field information, the position information of the target object is determined; wherein, the optical information and the magnetic field information are matched to obtain a matching result; based on the matching result, target information is determined from the optical information and the magnetic field information; and based on the target information, the position information of the target object is determined.

5. The positioning method according to claim 4, characterized in that, The optical information includes multiple optical coordinates, and the magnetic field information includes multiple magnetic field coordinates; the matching of the optical information and the magnetic field information to obtain a matching result includes: The quantity and position of each optical coordinate and each magnetic field coordinate are matched to determine the quantity matching information and multiple sets of coordinate pairs; each coordinate pair includes one optical coordinate and one magnetic field coordinate, and the distance between the optical coordinate and the magnetic field coordinate in the world coordinate system is less than a preset threshold.

6. The positioning method according to claim 5, characterized in that, The step of determining the target information from the optical information and the magnetic field information based on the matching result includes: If the quantity matching information is that the number of optical coordinates is greater than the number of magnetic field coordinates, then each of the magnetic field coordinates is taken as the target information; or, The optical coordinates in each of the coordinate pairs are used as the target information.

7. The positioning method according to claim 5, characterized in that, The step of determining the target information from the optical information and the magnetic field information based on the matching result includes: If the quantity matching information is that the number of optical coordinates is less than the number of magnetic field coordinates, then each of the magnetic field coordinates is taken as the target information; or, The remaining magnetic field coordinates and the optical coordinates in each of the coordinate pairs are used as the target information; wherein, the remaining magnetic field coordinates are the magnetic field coordinates remaining after removing the magnetic field coordinates in each coordinate pair from each of the magnetic field coordinates.

8. A positioning system, characterized in that, The system includes: The optomagnetic integrated marker according to any one of claims 1-3, each of the optomagnetic integrated markers being used to be disposed on a target object; A magnetic field generating device is used to provide a magnetic field to each of the aforementioned optomagnetic integrated markers; An optical tracking device is used to provide a light source to each of the aforementioned opto-magnetic integrated markers and to acquire optical information; A controller is communicatively connected to the electromagnetic markers in each of the optical-magnetic integrated markers and the optical tracking device, and the controller is used to perform the steps of the positioning method as described in any one of claims 4-7.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the positioning method according to any one of claims 4-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the positioning method according to any one of claims 4-7.

Images