Image registration methods, electronic devices, storage media and products

Abstract

This invention discloses an image registration method, electronic device, storage medium, and product. A first transformation relationship is determined based on the reference coordinate data of the optical target in a first coordinate system and the optical coordinate data in a second coordinate system. A second transformation relationship is determined based on the preoperative image coordinate data and the reference coordinate data. A third transformation relationship is determined based on the preoperative target coordinate data and the optical coordinate data. A fourth transformation relationship is determined based on the intraoperative target coordinate data and the preoperative target coordinate data. When the fourth transformation relationship does not meet preset conditions, a target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first, second, third, and fourth transformation relationships. This allows for automatic registration of preoperative and intraoperative images based on the target transformation relationship, avoiding the need for re-acquiring and re-registering images due to movement of the optical locator, thus improving the efficiency and accuracy of image registration.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image registration method, electronic device, storage medium, and product. Background Technology

[0002] During the operation of the surgical arm on the surgical area, an optical locator is typically used to position the surgical area, and the image acquisition device in the C-arm is used to acquire and process images of the surgical area. However, during the operation of the surgical arm, the optical locator is easily affected by external interference, such as being blocked or the position of the surgical area being moved, which affects the operation of the surgical arm.

[0003] Currently, the existing technology mainly addresses the above situation by adjusting the position of the optical locator and re-scanning and processing the surgical area, then re-registering the images based on the re-acquired images. However, this method of re-registering the images not only increases the radiation dose received by the surgical area but also affects the efficiency of image registration processing, thereby impacting the operational efficiency and accuracy of the surgical arm. Summary of the Invention

[0004] This invention provides an image registration method, electronic device, storage medium, and product that reduces the radiation dose received by the surgical area and improves the efficiency and accuracy of image registration.

[0005] According to one aspect of the present invention, an image registration method is provided, the method comprising:

[0006] Based on the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, the first transformation relationship of the optical target is determined. The first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical positioning instrument as the origin.

[0007] Based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, the second transformation relationship is determined. The preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin.

[0008] Based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, the third transformation relationship is determined. The preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin.

[0009] The fourth transformation relationship is determined based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system. The intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin.

[0010] When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to register the preoperative image and intraoperative image acquired when performing the target task based on the target transformation relationship; wherein, the intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

[0011] According to another aspect of the present invention, an image registration apparatus is provided, the apparatus comprising:

[0012] The first transformation relationship determination module is used to determine the first transformation relationship of the optical target based on the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system. The first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical positioning instrument as the origin.

[0013] The second transformation relationship determination module is used to determine the second transformation relationship based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system. The preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin.

[0014] The third transformation relationship determination module is used to determine the third transformation relationship based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system. The preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin.

[0015] The fourth transformation relationship determination module is used to determine the fourth transformation relationship based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system. The intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin.

[0016] The image registration module is used to determine the target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the first, second, third and fourth transformation relationships when the fourth transformation relationship does not meet the preset conditions. The target transformation relationship is used to register the preoperative image and the intraoperative image acquired when performing the target task. The intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory that is communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the image registration method of any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the image registration method of any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, characterized in that the computer program, when executed by a processor, implements an image registration method as described in any embodiment of the present invention.

[0023] The technical solution of this invention determines a first transformation relationship for the optical target based on the reference coordinate data in the first coordinate system and the optical coordinate data in the second coordinate system, facilitating the mutual conversion between the reference coordinate data and the optical coordinate data. A second transformation relationship is determined based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, facilitating the mutual conversion between the preoperative image coordinate data and the reference coordinate data. A third transformation relationship is determined based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, facilitating the mutual conversion between the preoperative target coordinate data and the optical coordinate data. A fourth transformation relationship is determined based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, facilitating the mutual conversion between the intraoperative target coordinate data and the preoperative target coordinate data. When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first, second, third, and fourth transformation relationships. The preoperative and intraoperative images acquired during the execution of the target task are then registered based on the target transformation relationship. This solves the problems of excessive radiation dose received in the surgical area and low image registration efficiency caused by repeated acquisition and registration of surgical area images in the prior art. It realizes real-time automatic registration of preoperative and intraoperative images, reduces the radiation dose received in the surgical area, and improves the efficiency and accuracy of image registration, thereby improving the overall operating efficiency and precision of the surgical arm.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of an image registration method provided in an embodiment of the present invention;

[0027] Figure 2 This is an example diagram of the surgical robot system provided in an embodiment of the present invention;

[0028] Figure 3This is a flowchart of a first transformation relationship determination method provided in an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the structure of an image registration device provided in an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the image registration method of this invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Before implementing the technical solution provided by this invention, the application scenario can be described first. The image registration method provided in this embodiment of the invention is applied to a surgical robot system. The surgical robot system includes a surgical operating arm module, a C-arm module, and an optical positioning module. The surgical operating arm module and the C-arm module are rigidly connected. The surgical operating arm module is used to operate on the surgical area, the C-arm module is used to scan the surgical area image, and the optical positioning module includes an optical locator, which is used to locate the position of an optical target.

[0034] Example 1

[0035] Figure 1This is a flowchart of an image registration method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where the surgical area moves or the optical target is obscured. It achieves automatic registration of intraoperative and preoperative images through target transformation relationships. This method can be executed by an image registration device, which can be implemented in hardware and / or software. The image registration device can be configured in electronic devices such as mobile phones, computers, or servers. Figure 1 As shown, the method includes:

[0036] S110. Based on the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, determine the first transformation relationship of the optical target.

[0037] The first coordinate system is a three-dimensional coordinate system with the center of the base of the surgical arm module as its origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical locator as its origin. The optical target is a device used for positioning and identification by the optical locator. The optical target may contain one or more reflective spheres. For example, an optical target may contain four reflective spheres. The optical target can be mounted on any object requiring real-time positioning and detection during surgery, such as the surgical arm, surgical area, or C-arm. The distribution distance and number of reflective spheres on different detection objects vary to facilitate the identification of different detection objects. Reference coordinate data can be used to characterize the three-dimensional coordinate data corresponding to the reflective spheres of the optical target in the first coordinate system. Optical coordinate data is used to characterize the three-dimensional coordinate data corresponding to the reflective spheres of the optical target in the second coordinate system. The center of the base of the surgical arm module can be the center point of the base of the surgical arm. The optical center of the optical locator can be the midpoint between the two optical centers of the binocular camera's optical locator. A first transformation relationship is used to characterize the transformation relationship between the coordinate data in the first and second coordinate systems. Optionally, the first transformation relation can be characterized by a homogeneous transformation matrix.

[0038] Specifically, in the surgical robot system, a first coordinate system is established with the base center of the surgical arm module as the origin, and the reference coordinate data corresponding to the reflective spheres among multiple optical targets in the first coordinate system are determined. A second coordinate system is established with the optical center of the optical positioner in the optical positioning module as the origin, and the optical coordinate data corresponding to the reflective spheres among multiple optical targets in the second coordinate system are determined. Based on the reference coordinate data and the optical coordinate data, the rotation matrix and translation vector between the reference coordinate data and the optical coordinate data are determined, and the first transformation relationship between the first coordinate system and the second coordinate system is determined based on the rotation matrix and the translation vector.

[0039] It should be noted that, in the embodiments of the present invention, the determination and updating of the first transformation relationship, the second transformation relationship, the third transformation relationship, and the fourth transformation relationship can all be determined by using the least squares method and the singular value decomposition algorithm to determine the rotation matrix and the translation vector, so as to determine the corresponding transformation relationship based on the rotation matrix and the translation vector.

[0040] For example, see Figure 2 , Figure 2 This is an example diagram of a surgical robot system. Figure 2 In this diagram, 1 represents the C-arm module, 2 the surgical operating arm module, and 3 the optical positioning module. A first coordinate system R is established with the center of the surgical operating arm module's base as the origin. Based on the reflective spheres of the optical targets installed at various positions within the first coordinate system, the reference coordinate data corresponding to the reflective spheres of each optical target are determined within the first coordinate system R. The set of all reference coordinate data is denoted as M. A second coordinate system C is established with the optical center of the binocular camera in the optical positioning module as the origin. The optical coordinate data corresponding to each optical target is determined within the second coordinate system, and the set of all optical coordinate data is denoted as H. The homogeneous transformation matrix between the first and second coordinate systems is determined based on the sets M and H. And the homogeneous transformation matrix This serves as the first transformation relationship corresponding to the optical target.

[0041] Optionally, the method further includes: when the movement of the optical positioning device is detected, determining the optical coordinate data of each optical target in the second coordinate system after the movement, and determining the correction matrix based on the optical coordinate data before and after the movement; updating the first transformation relationship based on the correction matrix to obtain the updated first transformation relationship.

[0042] The optical coordinate data after movement refers to the three-dimensional coordinates of each optical target in a three-dimensional coordinate system with the optical center of the moved optical positioning instrument as the origin. Optionally, since the base of the surgical arm module will not be moved, the optical coordinate data after movement can be the coordinate data of the optical targets deployed on the base of the surgical arm module. The correction matrix can be used to characterize the positional changes in the optical coordinate data after movement.

[0043] Specifically, in practical applications, optical targets may be obstructed. In such cases, the position of the optical locator will be moved to adjust the observation range. A second coordinate system is determined with the optical center of the moved optical locator as the origin. The optical coordinate data of each optical target corresponding to the moved target in this second coordinate system is then determined. A correction matrix is ​​established based on the optical coordinate data after the move and the optical coordinate data before the move. The first transformation relationship is adjusted and updated according to the correction matrix to obtain the updated first transformation relationship. This updated first transformation relationship is then used to determine subsequent target transformation relationships and perform automatic image registration. It should be noted that, to improve processing efficiency, the first transformation relationship can be updated every preset acquisition time. The preset acquisition time can be a duration set according to actual needs. For example, the preset acquisition time could be 10 seconds.

[0044] For example, in conjunction with the above example, during the operation of the surgical arm module on the surgical area, if the optical positioning module detects that any optical target is obstructed during the operation, the position of the optical positioning device can be moved to adjust the observation range. Since the base of the surgical arm module is not moved during the operation, it can be ensured that the optical target on the base is within the observation position of the optical positioning device when adjusting the observation range. During the adjustment process, the optical positioning device can collect the coordinate data of the optical target deployed at the base of the surgical arm in real time, and update the first transformation relationship based on the coordinate data.

[0045] This can be understood as follows: at a certain moment during the movement of the optical positioning device, a second coordinate system C' is established based on the optical center of the current optical positioning device as the origin, and the optical coordinate data of the surgical arm module base position under the second coordinate system C' after the movement is determined. The set corresponding to all the optical coordinate data after the movement is denoted as H'. The correction matrix is ​​determined based on set H' and set H. Based on the correction matrix Update the first transformation relationship Obtain the updated first transformation relation

[0046] S120. Determine the second transformation relationship based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system.

[0047] The preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as its origin. The C-arm module typically includes a detector, X-ray source, and related mechanical and electronic components, which work together to acquire preoperative and intraoperative images. When the C-arm module acquires images, the scanning center usually refers to the center point where the X-ray beam passes through the scanned object (e.g., the surgical area). Preoperative image coordinate data can be understood as the three-dimensional coordinate data corresponding to the reflective spheres of each optical target in the preoperative image coordinate system. The second transformation relationship is used to characterize the transformation relationship between the coordinate data in the preoperative image coordinate system and the first coordinate system. Optionally, the second transformation relationship can be characterized using a homogeneous transformation matrix.

[0048] Specifically, a preoperative image coordinate system is established with the scanning center corresponding to the preoperative C-arm module as the origin. The preoperative image coordinate data of each optical target under this coordinate system are then determined. Since there is a rigid connection between the surgical arm module and the C-arm, the preoperative image coordinate data in the preoperative image coordinate system and the reference coordinate data in the first coordinate system can be processed through measurement and calibration to determine the second transformation relationship between the preoperative image coordinate system and the first coordinate system.

[0049] For example, in conjunction with the above example, a preoperative image coordinate system I is established with the scanning center corresponding to the preoperative C-arm module as the origin. Since there is a rigid connection between the surgical operating arm module and the C-arm module, the second transformation relationship between the preoperative image coordinate system I and the first coordinate system R can be determined through measurement and calibration.

[0050] S130. Determine the third transformation relationship based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system.

[0051] The preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as its origin. Preoperative target coordinate data can be the three-dimensional coordinate data corresponding to all optical targets in the preoperative optical target coordinate system. The third transformation relationship is used to characterize the transformation relationship between the preoperative optical target coordinate system and the second coordinate system.

[0052] Specifically, before operating on the surgical area, the scanning center of the C-arm module can be aligned with the center of the surgical area to detect the position and status of the surgical area in real time. Optical targets are then deployed at the surgical area location. A preoperative optical target coordinate system is established with the center of each reflective sphere on all preoperative optical targets as the origin. The preoperative target coordinate data corresponding to each optical target in the preoperative optical target coordinate system is determined. Based on the preoperative target coordinate data and the optical coordinate data, the rotation matrix and translation vector between the preoperative target coordinate data and the optical coordinate data are determined, and the third transformation relationship between the preoperative optical target coordinate system and the second coordinate system is determined based on the rotation matrix and translation vector.

[0053] For example, in conjunction with the above example, before operating on the surgical area, the scanning center of the C-arm module can be aligned with the center of the surgical area to detect the position and status of the surgical area in real time. A preoperative optical target coordinate system P is established with the center of each reflective sphere on all preoperative optical targets as the origin. Based on the H set of the second coordinate system C and the preoperative target coordinate data of the preoperative optical target coordinate system P, a third transformation relationship between the second coordinate system and the preoperative optical target coordinate system is determined.

[0054] S140. Determine the fourth transformation relationship based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system.

[0055] The intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as its origin. The intraoperative target coordinate data consists of the three-dimensional coordinate data corresponding to each optical target under the intraoperative optical target coordinate system.

[0056] Specifically, the C-arm module is rotated to acquire intraoperative images, and the optical locator is used to detect the position information of each optical target in the current surgical area in real time. An intraoperative optical target coordinate system is established with the center of the reflective sphere on all intraoperative optical targets as the origin. Based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, the rotation matrix and translation vector between the intraoperative and preoperative target coordinate data are determined. Based on the rotation matrix and translation vector, the fourth transformation relationship between the intraoperative and preoperative optical target coordinate systems is determined.

[0057] For example, in conjunction with the above example, the C-arm module is controlled to rotate and acquire intraoperative images, while the optical locator determines the position information of the optical target in the current surgical area in real time. After the intraoperative images are acquired, an intraoperative optical target coordinate system P' is established with the center of the reflective sphere on all intraoperative optical targets as the origin. Based on the preoperative target coordinate data of the preoperative optical target coordinate system P and the intraoperative target coordinate data of the intraoperative optical target coordinate system P', a fourth transformation relationship between the preoperative optical target coordinate system P and the intraoperative optical target coordinate system P' is determined.

[0058] In this embodiment of the invention, before determining the target transformation relationship, it can be further determined whether the fourth transformation relationship meets the preset conditions based on the pose change information. Specifically, the determination method can be: based on the fourth transformation relationship, determine the pose change information of the preoperative optical target and the intraoperative optical target, wherein the pose change information includes displacement change information and attitude angle change information, so that if the pose change information exceeds the preset pose change threshold, it is determined that the fourth transformation relationship does not meet the preset conditions.

[0059] The pose change information includes displacement change information and attitude angle change information. Displacement change information characterizes the movement between preoperative and intraoperative target coordinate data. Attitude angle change information characterizes the angular change between preoperative and intraoperative target coordinate data. Preset pose change thresholds include displacement change thresholds and attitude angle change thresholds. The displacement change threshold can be understood as the standard value of displacement change between the preoperative and intraoperative optical targets. The attitude angle change threshold can be understood as the standard value of angular change between the preoperative and intraoperative optical targets. Preset conditions can be pre-defined conditions. If the fourth transformation relationship does not meet the preset conditions, it indicates that image registration processing is required.

[0060] Specifically, based on the fourth transformation relationship between the preoperative and intraoperative optical target coordinate systems, the displacement and attitude angle changes between the preoperative and intraoperative optical targets are determined. The displacement change information is then compared to a preset pose change threshold to determine if it exceeds that threshold. Similarly, the attitude angle change information is compared to a preset attitude angle change threshold to determine if it exceeds that threshold. If both the displacement and attitude angle changes exceed their respective thresholds, the fourth transformation relationship is considered not to meet the preset conditions, and image registration is required.

[0061] S150. When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to register the preoperative image and intraoperative image acquired when performing the target task based on the target transformation relationship.

[0062] The intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as its origin. The target transformation relationship characterizes the transformation relationship between the intraoperative image coordinate system and the first coordinate system. Through the target transformation relationship, the relative positional relationship between the intraoperative image and the surgical arm module can be determined, enabling image registration processing through this relative positional relationship. The target task can be the task of image registration.

[0063] Specifically, when the pose change information exceeds a preset pose change threshold, the fourth transformation relationship is determined to not meet the preset conditions. When the fourth transformation relationship does not meet the preset conditions, the transformation relationship between the preoperative image coordinate system and the intraoperative optical target coordinate system is determined based on the first, second, third, and fourth transformation relationships. According to the second transformation relationship between the preoperative image coordinate system and the first coordinate system, the transformation relationship between the intraoperative optical target coordinate system and the first coordinate system is determined. Based on the preset mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined, enabling image registration processing of the preoperative and intraoperative images acquired during the execution of the target task according to the target transformation relationship.

[0064] For example, in conjunction with the above example, when the pose change information exceeds a preset pose change threshold, the first transformation relationship between the first coordinate system R and the second coordinate system C is used. The second transformation relationship between the preoperative image coordinate system I and the first coordinate system R The third transformation relationship between the second coordinate system C and the preoperative optical target coordinate system P And the fourth transformation relationship between the preoperative optical target coordinate system P and the intraoperative optical target coordinate system P'. Determine the transformation relationship between the preoperative image coordinate system I and the intraoperative optical target coordinate system P'. Based on the pre-defined mapping relationship between the intraoperative optical target coordinate system P' and the intraoperative image coordinate system I', the transformation relationship between the preoperative image coordinate system I and the intraoperative image coordinate system I' is determined as follows: According to the second transformation relationship as well as Determine the target transformation relationship between the intraoperative image coordinate system I' and the first coordinate system R. Image registration between intraoperative and preoperative images is achieved based on the target transformation relationship.

[0065] In this embodiment of the invention, the target transformation relationship can be determined as follows: based on the first transformation relationship, the second transformation relationship, the third transformation relationship, and the fourth transformation relationship, a fifth transformation relationship between the intraoperative optical target coordinate system and the first coordinate system is determined; based on the mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth transformation relationship, a target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined.

[0066] The fifth transformation relationship is used to characterize the transformation relationship between the intraoperative optical target coordinate system and the coordinate data in the first coordinate system. The mapping relationship can be a pre-set correspondence between the intraoperative optical target coordinate system and the intraoperative image coordinate system.

[0067] Specifically, based on the first, second, third, and fourth transformation relationships, the transformation relationship between the preoperative image coordinate system and the intraoperative optical target coordinate system is determined. Based on the second transformation relationship between the preoperative image coordinate system and the first coordinate system, the fifth transformation relationship between the intraoperative optical target coordinate system and the first coordinate system is determined. Based on the mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth transformation relationship between the intraoperative optical target coordinate system and the first coordinate system, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined.

[0068] The technical solution of this embodiment determines a first transformation relationship for the optical target based on the reference coordinate data in the first coordinate system and the optical coordinate data in the second coordinate system, facilitating the mutual conversion between the reference coordinate data and the optical coordinate data. A second transformation relationship is determined based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, facilitating the mutual conversion between the preoperative image coordinate data and the reference coordinate data. A third transformation relationship is determined based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, facilitating the mutual conversion between the preoperative target coordinate data and the optical coordinate data. A fourth transformation relationship is determined based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, facilitating the mutual conversion between the intraoperative target coordinate data and the preoperative target coordinate data. When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first, second, third, and fourth transformation relationships. The preoperative and intraoperative images acquired during the execution of the target task are then registered based on the target transformation relationship. This solves the problems of excessive radiation dose received in the surgical area and low image registration efficiency caused by repeated acquisition and registration of surgical area images in the prior art. It realizes real-time automatic registration of preoperative and intraoperative images, reduces the radiation dose received in the surgical area, and improves the efficiency and accuracy of image registration, thereby improving the overall operating efficiency and precision of the surgical arm.

[0069] Example 2

[0070] Figure 3 This is a flowchart of a method for determining a first transformation relationship according to Embodiment 2 of the present invention. This embodiment is a refinement of the step "determining the first transformation relationship of the optical target based on the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system" based on the above embodiment. For specific implementation details, please refer to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiment will not be repeated here. Figure 3 As shown, the method includes:

[0071] S210. Determine the reference centroid coordinate data based on multiple reference coordinate data, and determine the optical centroid coordinate data based on multiple optical coordinate data.

[0072] The reference centroid coordinate data can be understood as the coordinates of the center point corresponding to all reference coordinate data. The optical centroid coordinate data can be understood as the coordinates of the center point corresponding to all optical coordinate data.

[0073] Specifically, the reference coordinate data in the first coordinate system are averaged to determine the reference centroid coordinate data in the first coordinate system. Correspondingly, the reference centroid coordinate data in the second coordinate system are averaged to obtain the optical centroid coordinate data in the second coordinate system.

[0074] It should be noted that, in this embodiment of the invention, before determining the reference centroid coordinate data and the optical centroid coordinate data, the method further includes: determining the coordinate distance value between each reference coordinate data and each optical coordinate data; and sorting the coordinate distance values ​​so that the reference coordinate data corresponds to the optical coordinate data.

[0075] The coordinate distance value is used to characterize the coordinate distance between the reference coordinate data and the optical coordinate data.

[0076] Specifically, based on the three-dimensional coordinate distance determination formula, namely the Euclidean distance formula, the coordinate distance value between each reference coordinate data and each optical coordinate data is determined. The coordinate distance values ​​are then sorted from largest to smallest or smallest to largest to ensure that the reference coordinate data and optical coordinate data of the same optical target correspond.

[0077] For example, referring to the above example, the coordinate distance between each coordinate data in the set M corresponding to the reference coordinate data and the set H corresponding to the optical coordinate data is calculated. The order of the coordinate data in sets M and H is adjusted by the least squares method so that the reference coordinate data and the optical coordinate data corresponding to the same optical target correspond one-to-one.

[0078] S220. Based on the reference centroid coordinate data, each reference coordinate data is decentralized to obtain the reference coordinate data to be processed. Based on the optical centroid coordinate data, each optical coordinate data is decentralized to obtain the optical coordinate data to be processed.

[0079] Decentralization can be achieved by subtracting the reference centroid coordinates from each reference coordinate data point. Similarly, it can be achieved by subtracting the optical centroid coordinates from the optical coordinate data. The reference coordinate data to be processed is the coordinate data obtained by subtracting the reference centroid coordinates from the reference coordinate data. Likewise, the optical coordinate data to be processed is the coordinate data obtained by subtracting the optical centroid coordinates from the optical coordinate data.

[0080] Specifically, each reference coordinate data in the first coordinate system is decentered, that is, the reference centroid coordinate data is subtracted from each reference coordinate data to obtain the reference coordinate data to be processed. Similarly, each optical coordinate data in the second coordinate system is decentered, that is, the optical centroid coordinate data is subtracted from each optical coordinate data to obtain the optical coordinate data to be processed.

[0081] S230. Based on the reference coordinate data and optical coordinate data to be processed, construct the matrix to be processed.

[0082] The matrix to be processed can be a matrix constructed from the reference coordinate data and the optical coordinate data to be processed. The matrix to be processed can be represented as follows:

[0083]

[0084] Where D represents the matrix to be processed, m i h represents the reference coordinate data to be processed. i This represents the optical coordinate data to be processed, where n represents the number of optical targets at preset positions. The preset positions can be fixed positions in the base coordinate system or positions that will not move. Optionally, the preset positions can be the base of the surgical arm module.

[0085] Specifically, the processing matrix shown above is constructed based on the reference coordinate data to be processed at the preset position and the optical coordinate data to be processed at the preset position.

[0086] S240. Perform singular value decomposition on the matrix to be processed to determine the rotation matrix and translation vector between the first coordinate system and the second coordinate system, and determine the first transformation relationship between the first coordinate system and the second coordinate system based on the rotation matrix and translation vector.

[0087] Singular value decomposition (SVD) can be understood as decomposing the matrix to be processed into a product of three matrices. The SVD process for the matrix to be processed can be illustrated by the following equation.

[0088] D=P∑Q T

[0089] Where D represents the matrix to be processed, P and Q are orthogonal matrices, and ∑ is a diagonal matrix composed of singular values. The rotation matrix can be determined by the orthogonal matrix and is used to characterize the degree of rotation between the first and second coordinate systems. The translation vector can be a vector used to characterize the degree of translation between the first and second coordinate systems.

[0090] Specifically, singular value decomposition is performed on the matrix to be processed to obtain the corresponding orthogonal and diagonal matrices. The rotation matrix corresponding to the matrix to be processed is determined based on the orthogonal matrix. Based on the rotation matrix, the reference coordinates to be processed at a preset position, and the optical coordinates to be processed at the preset position, the translation vector between the first and second coordinate systems is determined. Based on the rotation matrix and the translation vector, a homogeneous transformation matrix between the first and second coordinate systems is constructed, and this homogeneous transformation matrix is ​​used as the first transformation relationship.

[0091] In this embodiment of the invention, the method for determining the rotation matrix and translation vector may be as follows: performing singular value decomposition on the matrix to be processed, so as to determine the rotation matrix corresponding to the matrix to be processed when the matrix to be processed is a full-rank matrix; constructing a residual function based on the reference coordinate data to be processed, the optical coordinate data to be processed, and the rotation matrix, so as to obtain the translation vector between the first coordinate system and the second coordinate system when the residual function satisfies the preset residual condition.

[0092] Here, a full-rank matrix is ​​a matrix whose rank is equal to the number of rows or columns of the matrix to be processed. For a square matrix, it is a matrix whose rank is equal to its order. The residual function can be a function used to solve for a specific translation vector. The preset residual conditions can be pre-set conditions that the calculation results of the residual function must satisfy. Optionally, the preset residual conditions can be that the sum of squares of the calculation results of the residual function reaches a minimum value.

[0093] Specifically, singular value decomposition is performed on the matrix to be processed, resulting in two orthogonal matrices and a diagonal matrix. When the matrix to be processed is a full-rank matrix, the two orthogonal matrices are processed to obtain a rotation matrix. A residual function is constructed based on the reference coordinate data to be processed, the optical coordinate data to be processed, and the rotation matrix. When the calculation result of the residual function satisfies the preset residual condition, the translation vector between the first and second coordinate systems is obtained.

[0094] For example, in conjunction with the above example, singular value decomposition is performed on the matrix to be processed, D = P∑Q T We obtain orthogonal matrices P and Q, and a diagonal matrix ∑. When the matrix D to be processed is a full-rank matrix, the rotation matrix R is calculated based on the orthogonal matrices P and Q. That is, R = PQ T If the determinant of the calculated rotation matrix is ​​negative, then the negative determinant is taken as the optimal rotation matrix, and subsequent processing is performed based on the optimal rotation matrix. A residual function Y is constructed based on the rotation matrix R, the optical coordinate data to be processed, and the reference coordinate data to be processed.

[0095]

[0096] Where R is the rotation matrix, m i h represents the reference coordinate data to be processed. i This represents the optical coordinate data to be processed, where n represents the number of optical targets at the preset position, and t is the translation vector.

[0097] The residual function is solved to minimize the sum of squares of the calculated residual function result Y, thus obtaining the translation vector t. Based on this, a homogeneous transformation matrix is ​​constructed using the rotation matrix R and the translation vector t. Its composition is as follows:

[0098]

[0099] Based on this, the above homogeneous transformation matrix is ​​used as the first transformation relation, and subsequent target transformation relations and image registration processing are performed based on the first transformation relation.

[0100] The technical solution of this embodiment determines reference centroid coordinate data based on multiple reference coordinate data and optical centroid coordinate data based on multiple optical coordinate data. Each reference coordinate data is decentered based on the reference centroid coordinate data to obtain the reference coordinate data to be processed, and each optical coordinate data is decentered based on the optical centroid coordinate data to obtain the optical coordinate data to be processed. This increases the comparability between the coordinate data of the first and second coordinate systems. A matrix to be processed is constructed based on the reference coordinate data to be processed and the optical coordinate data to be processed. Singular value decomposition is performed on the matrix to be processed to determine the rotation matrix and translation vector between the first and second coordinate systems. Based on the rotation matrix and translation vector, a first transformation relationship between the first and second coordinate systems is determined. This facilitates subsequent determination of the target transformation relationship based on the accurately determined first transformation relationship, enabling subsequent image registration processing. Therefore, the radiation dose received by the surgical area is reduced, and the efficiency and accuracy of image registration are improved.

[0101] Example 3

[0102] Figure 4 This is a schematic diagram of the structure of an image registration device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a first conversion relationship determination module 310, a second conversion relationship determination module 320, a third conversion relationship determination module 330, a fourth conversion relationship determination module 340, and an image registration module 350.

[0103] The first transformation relationship determination module 310 is used to determine the first transformation relationship of the optical target based on the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system. The first coordinate system is a three-dimensional coordinate system with the base center of the surgical arm module as its origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical positioning instrument as its origin. The second transformation relationship determination module 320 is used to determine the second transformation relationship based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system. The preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as its origin. The third transformation relationship determination module 330 is used to determine the third transformation relationship based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system. The preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin. The fourth transformation relationship determination module 340 is used to determine the fourth transformation relationship based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system. The intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin. The image registration module 350 is used to determine the target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship, and the fourth transformation relationship when the fourth transformation relationship does not meet the preset conditions. The target transformation relationship is used to register the preoperative image and the intraoperative image acquired when performing the target task. The intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

[0104] The technical solution of this embodiment determines a first transformation relationship for the optical target based on the reference coordinate data in the first coordinate system and the optical coordinate data in the second coordinate system, facilitating the mutual conversion between the reference coordinate data and the optical coordinate data. A second transformation relationship is determined based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, facilitating the mutual conversion between the preoperative image coordinate data and the reference coordinate data. A third transformation relationship is determined based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, facilitating the mutual conversion between the preoperative target coordinate data and the optical coordinate data. A fourth transformation relationship is determined based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, facilitating the mutual conversion between the intraoperative target coordinate data and the preoperative target coordinate data. When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first, second, third, and fourth transformation relationships. The preoperative and intraoperative images acquired during the execution of the target task are then registered based on the target transformation relationship. This solves the problems of excessive radiation dose received in the surgical area and low image registration efficiency caused by repeated acquisition and registration of surgical area images in the prior art. It realizes real-time automatic registration of preoperative and intraoperative images, reduces the radiation dose received in the surgical area, and improves the efficiency and accuracy of image registration, thereby improving the overall operating efficiency and precision of the surgical arm.

[0105] Based on the above embodiments, optionally, the first transformation relationship determination module includes: a centroid coordinate data determination unit, used to determine reference centroid coordinate data based on multiple reference coordinate data, and to determine optical centroid coordinate data based on multiple optical coordinate data; a coordinate data decentralization unit, used to decentralize each reference coordinate data based on the reference centroid coordinate data to obtain reference coordinate data to be processed, and to decentralize each optical coordinate data based on the optical centroid coordinate data to obtain optical coordinate data to be processed; a matrix to be processed determination unit, used to construct a matrix to be processed based on the reference coordinate data to be processed and the optical coordinate data to be processed; and a first transformation relationship determination unit, used to perform singular value decomposition on the matrix to be processed to determine the rotation matrix and translation vector between the first coordinate system and the second coordinate system, and to determine the first transformation relationship between the first coordinate system and the second coordinate system based on the rotation matrix and the translation vector.

[0106] Optionally, the first transformation relationship determination unit includes: a rotation matrix determination subunit, used to perform singular value decomposition on the matrix to be processed, so as to determine the rotation matrix corresponding to the matrix to be processed when the matrix to be processed is a full-rank matrix; and a translation vector determination subunit, used to construct a residual function based on the reference coordinate data to be processed, the optical coordinate data to be processed, and the rotation matrix, so as to obtain the translation vector between the first coordinate system and the second coordinate system when the residual function satisfies the preset residual condition.

[0107] Optionally, the device further includes: a coordinate data correspondence module, used to determine the coordinate distance value between each reference coordinate data and each optical coordinate data; and to sort the coordinate distance values ​​so that the reference coordinate data corresponds to the optical coordinate data.

[0108] Optionally, the image registration module includes: a fifth transformation relationship determination unit, used to determine the fifth transformation relationship between the intraoperative optical target coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship; and a target transformation relationship determination unit, used to determine the target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth transformation relationship.

[0109] Optionally, the device further includes: a fourth conversion relationship judgment module, used to determine the pose change information of the preoperative optical target and the intraoperative optical target based on the fourth conversion relationship, wherein the pose change information includes displacement change information and attitude angle change information, so as to determine that the fourth conversion relationship does not meet the preset conditions when the pose change information exceeds the preset pose change threshold.

[0110] Optionally, the device further includes: a first transformation relationship update module, used to determine the optical coordinate data of each optical target in the second coordinate system after the movement is detected when the optical positioning device moves, and to determine the correction matrix based on the optical coordinate data before and after the movement; and to update the first transformation relationship according to the correction matrix to obtain the updated first transformation relationship.

[0111] The image registration apparatus provided in this embodiment of the invention can execute the image registration method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0112] Example 4

[0113] Figure 5This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0114] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0115] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0116] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image registration methods.

[0117] In some embodiments, the image registration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image registration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image registration method by any other suitable means (e.g., by means of firmware).

[0118] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] Computer programs for implementing the image registration method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0120] Example 5

[0121] Embodiment 5 of the present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute an image registration method, the method comprising:

[0122] Based on the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, a first transformation relationship for the optical target is determined. The first coordinate system is a three-dimensional coordinate system with the center of the surgical arm module's base as its origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical positioning device as its origin. Based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, a second transformation relationship is determined. The preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as its origin. Based on the preoperative target coordinate data in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, a third transformation relationship is determined. The preoperative optical target coordinate system is... A three-dimensional coordinate system with the center of all preoperative optical targets as the origin is established. A fourth transformation relationship is determined based on the intraoperative target coordinate data in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system. The intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin. If the fourth transformation relationship does not meet preset conditions, a target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first, second, third, and fourth transformation relationships. This target transformation relationship is used to register the preoperative and intraoperative images acquired during the target task. The intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

[0123] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0124] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0125] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0126] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0127] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0128] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image registration method characterized by, The application is applied to a surgical robot system, the surgical robot system comprises a surgical operating arm module, a C-shaped arm module and an optical positioning module, the surgical operating arm module is rigidly connected with the C-shaped arm module, the surgical operating arm module is used for operating a surgical area, the C-shaped arm module is used for scanning a surgical area image, and the optical positioning module comprises an optical positioner, the optical positioner is used for positioning a surgical area position, and the method comprises the following steps: According to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, the first conversion relationship of the optical target is determined, wherein the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical positioner as the origin; According to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, the second conversion relationship is determined, wherein the preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-shaped arm module as the origin; According to the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, the third conversion relationship is determined, wherein the preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin; According to the intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, the fourth conversion relationship is determined, wherein the intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin; When the fourth conversion relationship does not meet the preset condition, the target conversion relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first conversion relationship, the second conversion relationship, the third conversion relationship and the fourth conversion relationship, so that the preoperative image and the intraoperative image collected when the target task is performed are registered based on the target conversion relationship; Wherein the intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-shaped arm module as the origin.

2. The method of claim 1, wherein, According to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, the first conversion relationship of the optical target is determined, wherein the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical positioner as the origin; The reference centroid coordinate data is determined based on a plurality of reference coordinate data, and the optical centroid coordinate data is determined based on a plurality of optical coordinate data; Each reference coordinate data is processed by decentralization based on the reference centroid coordinate data, and each optical coordinate data is processed by decentralization based on the optical centroid coordinate data, to obtain the to-be-processed reference coordinate data and the to-be-processed optical coordinate data; The to-be-processed matrix is constructed based on the to-be-processed reference coordinate data and the to-be-processed optical coordinate data. perform singular value decomposition processing on the to-be-processed matrix to determine a rotation matrix and a translation vector between the first coordinate system and the second coordinate system, and determine a first conversion relationship between the first coordinate system and the second coordinate system based on the rotation matrix and the translation vector.

3. The method of claim 2, wherein, The singular value decomposition processing on the to-be-processed matrix to determine the rotation matrix and the translation vector between the first coordinate system and the second coordinate system comprises: perform singular value decomposition processing on the to-be-processed matrix to determine a rotation matrix corresponding to the to-be-processed matrix when the to-be-processed matrix is a full rank matrix; construct a residual function based on the to-be-processed reference coordinate data, the to-be-processed optical coordinate data, and the rotation matrix to obtain a translation vector between the first coordinate system and the second coordinate system when the residual function satisfies a preset residual condition.

4. The method of claim 2, wherein, Before determining the reference centroid coordinate data and the optical centroid coordinate data, the method further comprises: determine a coordinate distance value between each of the reference coordinate data and each of the optical coordinate data; sort the coordinate distance values so that the reference coordinate data correspond to the optical coordinate data.

5. The method of claim 1, wherein, The determination of the target conversion relationship between the intraoperative image coordinate system and the first coordinate system based on the first conversion relationship, the second conversion relationship, the third conversion relationship, and the fourth conversion relationship comprises: determine a fifth conversion relationship between the intraoperative optical target coordinate system and the first coordinate system based on the first conversion relationship, the second conversion relationship, the third conversion relationship, and the fourth conversion relationship; determine the target conversion relationship between the intraoperative image coordinate system and the first coordinate system based on a mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth conversion relationship.

6. The method of claim 1, wherein, Further comprising: determine pose change information of the preoperative optical target and the intraoperative optical target based on the fourth conversion relationship, wherein the pose change information comprises displacement change information and attitude angle change information, so as to determine that the fourth conversion relationship does not satisfy a preset condition when the pose change information exceeds a preset pose change threshold.

7. The method of claim 1, wherein, Further comprising: when it is detected that the optical positioner moves, determine post-movement optical coordinate data corresponding to each optical target in a second coordinate system after movement, and determine a correction matrix based on the pre-movement optical coordinate data and the post-movement optical coordinate data; update the first conversion relationship according to the correction matrix to obtain an updated first conversion relationship.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image registration method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the image registration method of any one of claims 1-7 when executed.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the image registration method of any one of claims 1-7.

Images