Image registration method, electronic equipment, storage medium and product

Abstract

The invention discloses an image registration method, electronic equipment, a storage medium and a product. Determining a first conversion relation according to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data of the optical target in the second coordinate system; determining a second conversion relation according to the preoperative image coordinate data and the reference coordinate data; determining a third conversion relation according to the preoperative target coordinate data and the optical coordinate data; determining a fourth conversion relation according to the intra-operative target coordinate data and the pre-operative target coordinate data; and when the fourth conversion relation does not meet a preset condition, determining a target conversion relation between the intra-operative image coordinate system and the first coordinate system based on the first conversion relation, the second conversion relation, the third conversion relation and the fourth conversion relation, so as to perform automatic registration on the pre-operative image and the intra-operative image based on the target conversion relation. The situation that images need to be collected again and registration needs to be conducted again due to movement of the optical locator is avoided, and the efficiency and accuracy of image registration are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image registration method, electronic equipment, storage medium and product. Background Art

[0002] When the surgical arm is operating the surgical area, an optical locator is usually used to locate the surgical area and the image acquisition device in the C-arm is used to acquire and process the image of the surgical area. However, during the operation of the surgical arm, the optical locator is easily interfered by the outside world, and the optical locator may be blocked or the surgical area may be moved, thus affecting the operation of the surgical arm.

[0003] At present, in view of the above situation, the existing technology mainly adjusts the position of the optical locator and re-scans and photographs the surgical area to re-register the surgical area image after the position adjustment. However, the above-mentioned re-registration of the image will not only increase the radiation dose received by the surgical area, but also affect the efficiency of the image registration process, thereby affecting the operating efficiency and accuracy of the surgical operating arm. Summary of the invention

[0004] The present invention provides an image registration method, electronic equipment, storage medium and product, which reduce the radiation dose received by the surgical area and improve the efficiency and accuracy of image registration.

[0005] According to one aspect of the present invention, there is provided an image registration method, the method comprising:

[0006] Determine a first conversion relationship of the optical target according to reference coordinate data of the optical target in a first coordinate system and optical coordinate data of the optical target in a second coordinate system, wherein the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical locator as the origin;

[0007] Determine a second conversion relationship according to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, wherein the preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin;

[0008] Determining a third conversion relationship according to preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and optical coordinate data in the second coordinate system, wherein the preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin;

[0009] Determining a fourth conversion relationship according to intraoperative target coordinate data of the optical target in an intraoperative optical target coordinate system and preoperative target coordinate data in a preoperative optical target coordinate system, wherein the intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin;

[0010] When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to align the preoperative image and the intraoperative image collected when performing the target task based on the target transformation relationship; wherein the intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

[0011] According to another aspect of the present invention, there is provided an image registration device, the device comprising:

[0012] A first conversion relationship determination module, used to determine a first conversion relationship of the optical target according to reference coordinate data of the optical target in a first coordinate system and optical coordinate data of the optical target in a second coordinate system, wherein the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical locator as the origin;

[0013] A second conversion relationship determination module is used to determine a second conversion relationship based on preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, wherein the preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin;

[0014] A third conversion relationship determination module, used to determine a third conversion relationship according to preoperative target coordinate data of the optical target in a preoperative optical target coordinate system and optical coordinate data in a second coordinate system, wherein the preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin;

[0015] a fourth conversion relationship determination module, configured to determine a fourth conversion relationship according to intraoperative target coordinate data of the optical target in an intraoperative optical target coordinate system and preoperative target coordinate data in a preoperative optical target coordinate system, wherein the intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin;

[0016] An image registration module is used to determine a target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship when the fourth transformation relationship does not meet the preset conditions, so as to align the preoperative image and the intraoperative image collected when performing the target task based on the target transformation relationship; wherein the intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the image registration method of any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image registration method of any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein the computer program implements the image registration method according to any embodiment of the present invention when executed by a processor.

[0023] The technical solution of the embodiment of the present invention determines the first conversion relationship of the optical target according to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, so as to facilitate the mutual conversion between the reference coordinate data and the optical coordinate data according to the first conversion relationship. Determine the second conversion relationship according to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, so as to facilitate the mutual conversion between the preoperative image coordinate data and the reference coordinate data according to the second conversion relationship. Determine the third conversion relationship according to the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, so as to facilitate the mutual conversion between the preoperative target coordinate data and the optical coordinate data according to the third conversion relationship. Determine the fourth conversion relationship according to the intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, so as to facilitate the mutual conversion between the intraoperative target coordinate data and the preoperative target coordinate data according to the fourth conversion relationship. When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to align the preoperative image and the intraoperative image acquired when performing the target task based on the target transformation relationship. This solves the problems of excessive radiation dose received by the surgical area and low image registration efficiency caused by repeated acquisition of surgical area images and repeated registration in the prior art, realizes real-time automatic registration of preoperative images and intraoperative images, reduces the radiation dose received by the surgical area, improves the efficiency and accuracy of image registration, and thus improves the overall operating efficiency and accuracy of the surgical operating arm.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of an image registration method provided by an embodiment of the present invention;

[0027] Figure 2 is an example diagram of a surgical robot system provided by an embodiment of the present invention;

[0028] Figure 3is a flow chart of a first conversion relationship determination method provided by an embodiment of the present invention;

[0029] Figure 4 is a structural schematic diagram of an image registration device provided by an embodiment of the present invention;

[0030] Figure 5 It is a schematic diagram of the structure of an electronic device for implementing the image registration method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Before executing the technical solution provided by the present invention, the application scenario may be described first. The image registration method provided in an embodiment of the present invention is applied to a surgical robot system. The surgical robot system includes a surgical operating arm module, a C-arm module, and an optical positioning module. The surgical operating arm module is rigidly connected to the C-arm module, the surgical operating arm module is used to operate the surgical area, the C-arm module is used to scan the surgical area image, and the optical positioning module includes an optical locator, which is used to locate the position of the optical target.

[0034] Embodiment 1

[0035] Figure 1This is a flow chart of an image registration method provided by the first embodiment of the present invention. This embodiment can be applied to the case where the surgical area moves or the optical target is blocked, and the intraoperative image and the preoperative image are automatically registered through the target conversion relationship. The method can be executed by an image registration device, which can be implemented in the form of hardware and / or software, and can be configured in electronic devices such as mobile phones, computers or servers. Figure 1 As shown, the method includes:

[0036] S110, determining a first conversion relationship of the optical target according to reference coordinate data of the optical target in the first coordinate system and optical coordinate data of the optical target in the second coordinate system.

[0037] Among them, the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical locator as the origin. The optical target is a device used to be located and identified by the optical locator. The optical target may include one or more reflective balls. For example, an optical target may include 4 reflective balls. The optical target can be installed on any object that needs to be positioned and detected in real time during surgery, such as a surgical operating arm, a surgical area, or a C-arm. The distribution distance and number of reflective balls of the optical target on different detection objects are different, so that different detection objects can be identified. The reference coordinate data can be used to characterize the three-dimensional coordinate data corresponding to the reflective ball of the optical target in the first coordinate system. The optical coordinate data is used to characterize the three-dimensional coordinate data corresponding to the reflective ball of the optical target in the second coordinate system. The base center of the surgical operating arm module can be the center point position of the base of the surgical operating arm. The optical center of the optical locator can be the midpoint position of the two optical centers in the optical locator of the binocular camera. The first conversion relationship is used to characterize the conversion relationship between the coordinate data in the first coordinate system and the second coordinate system. Optionally, the first conversion relationship can be represented by a homogeneous transformation matrix.

[0038] Specifically, in the surgical robot system, a first coordinate system is established with the base center of the surgical operating arm module as the origin, and the reference coordinate data corresponding to the reflective balls in the multiple optical targets in the first coordinate system are determined. A second coordinate system is established with the optical center of the optical locator in the optical positioning module as the origin, and the optical coordinate data corresponding to the reflective balls in the multiple optical targets in the second coordinate system are determined. Based on the reference coordinate data and the optical coordinate data, the rotation matrix and the translation vector between the reference coordinate data and the optical coordinate data are determined, so as to determine the first conversion relationship between the first coordinate system and the second coordinate system based on the rotation matrix and the translation vector.

[0039] It should be noted that in an embodiment of the present invention, the determination and update of the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship can all be determined by using the least squares method and the singular value decomposition algorithm to determine the rotation matrix and the translation vector, so as to determine the corresponding transformation relationship based on the rotation matrix and the translation vector.

[0040] For example, see Figure 2 , Figure 2 This is an example diagram of a surgical robot system. Figure 2 In it, 1 is the C-arm module, 2 is the surgical operating arm module, and 3 is the optical positioning module. A first coordinate system R is established with the center of the base of the surgical operating arm module as the origin. According to the reflective balls of the optical targets installed at various positions in the first coordinate system, the reference coordinate data corresponding to the reflective balls of each optical target in the first coordinate system R is determined. And the set corresponding to all the reference coordinate data is denoted as M. A second coordinate system C is established with the midpoint of the optical center of the binocular camera of the optical locator in the optical positioning module as the origin. Determine the optical coordinate data corresponding to each optical target in the second coordinate system, and denoted as H the set corresponding to all the optical coordinate data. Determine the homogeneous transformation matrix of the first coordinate system and the second coordinate system based on the M set and the H set And the homogeneous transformation matrix As the first conversion relationship corresponding to the optical target.

[0041] Optionally, the method also includes: when movement of the optical locator is detected, determining the optical coordinate data after movement corresponding to each optical target in the second coordinate system after movement, and determining a correction matrix based on the optical coordinate data before movement and the optical coordinate data after movement; updating the first transformation relationship based on the correction matrix to obtain an updated first transformation relationship.

[0042] The optical coordinate data after movement is the three-dimensional coordinate data corresponding to each optical target in the three-dimensional coordinate system with the optical center of the optical locator after movement as the origin. Optionally, since the base portion of the surgical operating arm module will not be moved, the optical coordinate data after movement may be the coordinate data of the optical target deployed by the base portion of the surgical operating arm module. The correction matrix may be the position change information used to characterize the optical coordinate data after movement.

[0043] Specifically, in the actual application process, the optical target may be blocked. At this time, the position of the optical locator will be moved to adjust the observation range. The second coordinate system after the move is determined with the optical center of the moved optical locator as the origin. And determine the optical coordinate data after the move corresponding to each optical target in the second coordinate system after the move. According to the optical coordinate data after the move and the optical coordinate data before the move, a correction matrix is ​​established. According to the correction matrix, the first transformation relationship is adjusted and updated to obtain an updated first transformation relationship, so as to determine the subsequent target transformation relationship and automatically align the image based on the updated first transformation relationship. It should be noted that in order to improve processing efficiency, the first transformation relationship can be updated once every preset acquisition time. Among them, the preset acquisition time can be a time set according to actual needs. For example, the preset acquisition time can be 10 seconds.

[0044] Exemplarily, in combination with the above example, during the operation of the surgical operating arm module on the surgical area, if the optical locator module detects that any optical target during the operation is blocked, the position of the optical locator can be moved to adjust the observation range. Since the base portion of the surgical operating arm module will not be moved during the operation, it is possible to ensure that the optical target of the base portion is within the observation position of the optical locator when adjusting the observation range. During the adjustment process, the optical locator can collect the coordinate data of the optical target deployed at the base of the surgical operating arm in real time to update the first conversion relationship based on the coordinate data.

[0045] It can be understood that at a certain moment in the movement of the optical locator, a second coordinate system C' is established based on the optical center of the current optical locator as the origin, and the optical coordinate data of the surgical operating arm module base position after the movement in the second coordinate system C' is determined. The set corresponding to all the optical coordinate data after the movement is recorded as H'. Determine the correction matrix based on the set H' and the set H Based on the correction matrix Update the first conversion relationship Get the updated first conversion relationship

[0046] S120 , determining a second conversion relationship according to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system.

[0047] Among them, the preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin. The C-arm module usually includes a detector, an X-ray source, and related mechanical and electronic components, etc. These components work together to obtain preoperative images and intraoperative images. When the C-arm module acquires images, the scanning center usually refers to the center point when the X-ray beam passes through the scanned object (such as the surgical area). The preoperative image coordinate data can be understood as the three-dimensional coordinate data corresponding to the reflective balls of each optical target in the preoperative image coordinate system. The second transformation relationship is used to characterize the transformation relationship between the coordinate data in the preoperative image coordinate system and the first coordinate system. Optionally, the second transformation relationship can be represented by a homogeneous transformation matrix.

[0048] Specifically, a preoperative image coordinate system is established with the scanning center corresponding to the preoperative C-arm module as the origin. The preoperative image coordinate data of each optical target in the preoperative image coordinate system is determined. Since there is a rigid connection between the surgical operating arm module and the C-arm, the preoperative image coordinate data in the preoperative image coordinate system and the reference coordinate data in the first coordinate system can be processed by measurement and calibration to determine the second conversion relationship between the preoperative image coordinate system and the first coordinate system.

[0049] Exemplarily, in combination with the above example, the scanning center corresponding to the preoperative C-arm module is used as the origin to establish the preoperative image coordinate system I. Since there is a rigid connection between the surgical operating arm module and the C-arm module, the second transformation relationship between the preoperative image coordinate system I and the first coordinate system R can be determined by measurement and calibration.

[0050] S130 , determining a third conversion relationship according to the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system.

[0051] The preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin. The preoperative target coordinate data may be the three-dimensional coordinate data corresponding to all optical targets in the preoperative optical target coordinate system. The third transformation relationship is used to characterize the transformation relationship between the preoperative optical target coordinate system and the second coordinate system.

[0052] Specifically, before operating the surgical area, the scanning center of the C-arm module can be aligned with the center of the surgical area to detect the position and state of the surgical area in real time. And deploy an optical target at the surgical area. Establish a preoperative optical target coordinate system with the center of each reflective ball on all preoperative optical targets as the origin. Determine the preoperative target coordinate data corresponding to each optical target in the preoperative optical target coordinate system. According to the preoperative target coordinate data and the optical coordinate data, determine the rotation matrix and translation vector between the preoperative target coordinate data and the optical coordinate data, so as to determine the third transformation relationship between the preoperative optical target coordinate system and the second coordinate system according to the rotation matrix and the translation vector.

[0053] Exemplarily, in combination with the above example, before operating the surgical area, the scanning center of the C-arm module can be aligned with the center of the surgical area to detect the position and state of the surgical area in real time. The preoperative optical target coordinate system P is established with the center of each reflective ball on all preoperative optical targets as the origin. According to the H set of the second coordinate system C and the preoperative target coordinate data of the preoperative optical target coordinate system P, the third conversion relationship between the second coordinate system and the preoperative optical target coordinate system is determined.

[0054] S140: Determine a fourth conversion relationship according to intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and preoperative target coordinate data in the preoperative optical target coordinate system.

[0055] The intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all optical targets during the operation as the origin. The intraoperative target coordinate data is the three-dimensional coordinate data corresponding to each optical target in the intraoperative optical target coordinate system.

[0056] Specifically, the C-arm module is controlled to rotate to collect intraoperative images, and the position information of each optical target in the current surgical area is detected in real time using an optical locator. An intraoperative optical target coordinate system is established with the center of the reflective ball on all intraoperative optical targets as the origin. According to the intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, the rotation matrix and translation vector between the intraoperative target coordinate data and the preoperative target coordinate data are determined, so as to determine the fourth transformation relationship between the intraoperative optical target coordinate system and the preoperative optical target coordinate system according to the rotation matrix and translation vector.

[0057] Exemplarily, in combination with the above example, the C-arm module is controlled to rotate to collect intraoperative images, and at the same time, the optical locator determines the position information of the optical target in the current surgical area in real time. After the intraoperative image acquisition is completed, the center of the reflective ball on all the intraoperative optical targets is used as the origin to establish the intraoperative optical target coordinate system P'. According to the preoperative target coordinate data of the preoperative optical target coordinate system P and the intraoperative target coordinate data of the intraoperative optical target coordinate system P', the fourth transformation relationship between the preoperative optical target coordinate system P and the intraoperative optical target coordinate system P' is determined.

[0058] In an embodiment of the present invention, before determining the target conversion relationship, it is also possible to determine whether the fourth conversion relationship meets the preset conditions according to the posture change information. The specific determination method may be: based on the fourth conversion relationship, determine the posture change information of the preoperative optical target and the intraoperative optical target, wherein the posture change information includes displacement change information and attitude angle change information, so as to determine that the fourth conversion relationship does not meet the preset conditions when the posture change information exceeds the preset posture change threshold.

[0059] Among them, the posture change information includes displacement change information and posture angle change information. The displacement change information can be used to characterize the movement between the preoperative target coordinate data and the intraoperative target coordinate data. The posture angle change information can be used to characterize the angle change between the preoperative target coordinate data and the intraoperative target coordinate data. The preset posture change threshold includes a displacement change threshold and a posture angle change threshold. The displacement change threshold can be understood as the standard value of the displacement change between the preoperative optical target and the intraoperative optical target. The posture angle change threshold can be understood as the standard value of the angle change between the preoperative optical target and the intraoperative optical target. The preset condition can be a pre-set condition. When the fourth conversion relationship does not meet the preset condition, it means that image registration processing is required at this time.

[0060] Specifically, according to the fourth transformation relationship between the preoperative optical target coordinate system and the intraoperative optical target coordinate system, the displacement change information and the attitude angle change information between the preoperative optical target and the intraoperative optical target are determined. According to the displacement change information and the displacement change threshold in the preset posture change threshold, it is determined whether the displacement change information exceeds the displacement change threshold. According to the attitude angle change information and the attitude angle change threshold in the preset posture change threshold, it is determined whether the attitude angle change information exceeds the attitude angle change threshold. In the case where the displacement change information exceeds the displacement change threshold and the attitude angle change information exceeds the attitude angle change threshold, it is considered that the fourth transformation relationship does not meet the preset conditions at this time, that is, image registration processing is required at this time.

[0061] S150. When the fourth transformation relationship does not meet the preset conditions, determine the target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to align the preoperative image and the intraoperative image collected when performing the target task based on the target transformation relationship.

[0062] The intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin. The target transformation relationship is used to characterize the transformation relationship between the intraoperative image coordinate system and the first coordinate system. The relative position relationship between the intraoperative image and the surgical operating arm module can be determined by the target transformation relationship, so as to realize the registration processing of the image by the relative position relationship. The target task can be a task of image registration.

[0063] Specifically, when the posture change information exceeds the preset posture change threshold, it is determined that the fourth transformation relationship does not meet the preset conditions. When the fourth transformation relationship does not meet the preset conditions, the transformation relationship between the preoperative image coordinate system and the intraoperative optical target coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship. According to the second transformation relationship between the preoperative image coordinate system and the first coordinate system, the transformation relationship between the intraoperative optical target coordinate system and the first coordinate system is determined. According to the preset mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined, so that the preoperative image and the intraoperative image collected when performing the target task are image registration processed according to the target transformation relationship.

[0064] Exemplarily, in combination with the above example, when the posture change information exceeds the preset posture change threshold, according to the first conversion relationship between the first coordinate system R and the second coordinate system C The second transformation relationship between the preoperative image coordinate system I and the first coordinate system R The third transformation relationship between the second coordinate system C and the preoperative optical target coordinate system P And the fourth transformation relationship between the preoperative optical target coordinate system P and the intraoperative optical target coordinate system P' Determine the transformation relationship between the preoperative image coordinate system I and the intraoperative optical target coordinate system P' According to the preset mapping relationship between the intraoperative optical target coordinate system P' and the intraoperative image coordinate system I', the conversion relationship between the preoperative image coordinate system I and the intraoperative image coordinate system I' is determined as follows: According to the second conversion relationship as well as Determine the target transformation relationship between the intraoperative image coordinate system I' and the first coordinate system R Image registration of intraoperative images and preoperative images is achieved based on the target transformation relationship.

[0065] In an embodiment of the present invention, the target transformation relationship may be determined by: determining the fifth transformation relationship between the intraoperative optical target coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship, and the fourth transformation relationship; determining the target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth transformation relationship.

[0066] The fifth conversion relationship is used to characterize the conversion relationship between the intraoperative optical target coordinate system and the coordinate data in the first coordinate system. The mapping relationship can be a pre-set corresponding relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system.

[0067] Specifically, the conversion relationship between the preoperative image coordinate system and the intraoperative optical target coordinate system is determined according to the first conversion relationship, the second conversion relationship, the third conversion relationship, and the fourth conversion relationship. According to the second conversion relationship between the preoperative image coordinate system and the first coordinate system, the fifth conversion relationship between the intraoperative optical target coordinate system and the first coordinate system is determined. According to the mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth conversion relationship between the intraoperative optical target coordinate system and the first coordinate system, the target conversion relationship between the intraoperative image coordinate system and the first coordinate system is determined.

[0068] The technical solution of this embodiment determines the first conversion relationship of the optical target according to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, so as to facilitate the mutual conversion between the reference coordinate data and the optical coordinate data according to the first conversion relationship. Determine the second conversion relationship according to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, so as to facilitate the mutual conversion between the preoperative image coordinate data and the reference coordinate data according to the second conversion relationship. Determine the third conversion relationship according to the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, so as to facilitate the mutual conversion between the preoperative target coordinate data and the optical coordinate data according to the third conversion relationship. Determine the fourth conversion relationship according to the intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, so as to facilitate the mutual conversion between the intraoperative target coordinate data and the preoperative target coordinate data according to the fourth conversion relationship. When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to align the preoperative image and the intraoperative image acquired when performing the target task based on the target transformation relationship. This solves the problems of excessive radiation dose received by the surgical area and low image registration efficiency caused by repeated acquisition of surgical area images and repeated registration in the prior art, realizes real-time automatic registration of preoperative images and intraoperative images, reduces the radiation dose received by the surgical area, improves the efficiency and accuracy of image registration, and thus improves the overall operating efficiency and accuracy of the surgical operating arm.

[0069] Embodiment 2

[0070] Figure 3 This is a flow chart of a method for determining a first conversion relationship provided by the second embodiment of the present invention. This embodiment is based on the above embodiment and refines the step of "determining the first conversion relationship of the optical target according to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system". The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above embodiment are not repeated here. Figure 3 As shown, the method includes:

[0071] S210, determining reference centroid coordinate data based on a plurality of reference coordinate data, and determining optical centroid coordinate data based on a plurality of optical coordinate data.

[0072] The reference centroid coordinate data can be understood as the center point coordinate data corresponding to all reference coordinate data, and the optical centroid coordinate data can be understood as the center point coordinate data corresponding to all optical coordinate data.

[0073] Specifically, a plurality of reference coordinate data in the first coordinate system are averaged to determine the reference centroid coordinate data in the first coordinate system. Accordingly, a plurality of optical coordinate data in the second coordinate system are averaged to obtain the optical centroid coordinate data in the second coordinate system.

[0074] It should be noted that, in an embodiment of the present invention, before determining the reference center of mass coordinate data and the optical center of mass coordinate data, the method also includes: determining the coordinate distance value between each reference coordinate data and each optical coordinate data; and sorting the coordinate distance values ​​so that the reference coordinate data corresponds to the optical coordinate data.

[0075] The coordinate distance value is used to represent the coordinate distance between the reference coordinate data and the optical coordinate data.

[0076] Specifically, the coordinate distance value between each reference coordinate data and each optical coordinate data is determined according to the three-dimensional coordinate distance determination formula, that is, the Euclidean distance formula. The coordinate distance values ​​are sorted from large to small or from small to large so that the reference coordinate data and the optical coordinate data of the same optical target correspond to each other.

[0077] Exemplarily, in combination with the above example, the coordinate distance values ​​between the coordinate data in the set M corresponding to the reference coordinate data and the set H corresponding to the optical coordinate data are calculated. The order of the coordinate data in the set M and the set H is adjusted by the least squares method so that the reference coordinate data corresponding to the same optical target corresponds to the optical coordinate data one by one.

[0078] S220, based on the reference centroid coordinate data, performing decentralized processing on each reference coordinate data to obtain reference coordinate data to be processed, and based on the optical centroid coordinate data, performing decentralized processing on each optical coordinate data to obtain optical coordinate data to be processed.

[0079] The decentralization process may be a process of subtracting the reference centroid coordinate data from each reference coordinate data. Correspondingly, the decentralization process may be a process of subtracting the optical centroid coordinate data from the optical coordinate data. The reference coordinate data to be processed is the coordinate data obtained by subtracting the reference centroid coordinate data from the reference coordinate data. Correspondingly, the optical coordinate data to be processed is the coordinate data obtained by subtracting the optical centroid coordinate data from the optical coordinate data.

[0080] Specifically, each reference coordinate data in the first coordinate system is decentralized, that is, each reference coordinate data is subtracted from the reference centroid coordinate data to obtain the reference coordinate data to be processed. Each optical coordinate data in the second coordinate system is decentralized, that is, each optical coordinate data is subtracted from the optical centroid coordinate data to obtain the optical coordinate data to be processed.

[0081] S230: construct a matrix to be processed based on the reference coordinate data to be processed and the optical coordinate data to be processed.

[0082] The matrix to be processed may be a matrix constructed by the reference coordinate data to be processed and the optical coordinate data to be processed. The matrix to be processed may be expressed as follows:

[0083]

[0084] Among them, D represents the matrix to be processed, m i Indicates the reference coordinate data to be processed, h i represents the optical coordinate data to be processed, and n represents the number of optical targets at the preset position. The preset position may be a fixed position in the base coordinate system or a position that will not move. Optionally, the preset position may be the base of the surgical operating arm module.

[0085] Specifically, the matrix to be processed shown above is constructed according to the reference coordinate data to be processed at the preset position and the optical coordinate data to be processed at the preset position.

[0086] S240, performing singular value decomposition processing on the matrix to be processed, determining a rotation matrix and a translation vector between the first coordinate system and the second coordinate system, and determining a first conversion relationship between the first coordinate system and the second coordinate system based on the rotation matrix and the translation vector.

[0087] The singular value decomposition process can be understood as a process of decomposing the matrix to be processed into the product form of three matrices. The singular value decomposition process of the matrix to be processed can be shown as follows.

[0088] D=P∑Q T

[0089] Wherein, D represents the matrix to be processed, P and Q are orthogonal matrices, and ∑ is a diagonal matrix composed of singular values. The rotation matrix can be a matrix determined by an orthogonal matrix and used to characterize the degree of rotation between the first coordinate system and the second coordinate system. The translation vector can be a vector used to characterize the degree of translation between the first coordinate system and the second coordinate system.

[0090] Specifically, singular value decomposition is performed on the matrix to be processed to obtain an orthogonal matrix and a diagonal matrix corresponding to the matrix to be processed. A rotation matrix corresponding to the matrix to be processed is determined according to the orthogonal matrix. A translation vector between the first coordinate system and the second coordinate system is determined according to the rotation matrix, the reference coordinate data to be processed at a preset position, and the optical coordinate data to be processed at a preset position. A homogeneous transformation matrix between the first coordinate system and the second coordinate system is constructed according to the rotation matrix and the translation vector, and the homogeneous transformation matrix is ​​used as the first conversion relationship.

[0091] In an embodiment of the present invention, a method for determining a rotation matrix and a translation vector may be: performing singular value decomposition processing on a processing matrix to determine a rotation matrix corresponding to the processing matrix when the processing matrix is ​​a full-rank matrix; constructing a residual function based on the processing reference coordinate data, the processing optical coordinate data and the rotation matrix to obtain a translation vector between the first coordinate system and the second coordinate system when the residual function satisfies a preset residual condition.

[0092] Among them, the full rank matrix is ​​a matrix whose rank is equal to the number of rows or columns of the matrix to be processed. For a square matrix, it is a matrix whose rank is equal to the order. The residual function can be a function used to solve and determine the translation vector. The preset residual condition can be a pre-set condition that the calculation result of the residual function needs to meet. Optionally, the preset residual condition can be that the sum of the squares of the calculation results of the residual function reaches a minimum value.

[0093] Specifically, singular value decomposition is performed on the matrix to be processed to obtain two orthogonal matrices and a diagonal matrix. When the matrix to be processed is a full-rank matrix, the two orthogonal matrices are processed to obtain a rotation matrix. A residual function is constructed according to the reference coordinate data to be processed, the optical coordinate data to be processed, and the rotation matrix, so that when the calculation result of the residual function meets the preset residual condition, a translation vector between the first coordinate system and the second coordinate system is obtained.

[0094] Exemplarily, in combination with the above example, singular value decomposition is performed on the matrix to be processed, D = P∑Q T , we get orthogonal matrices P and Q, and diagonal matrix ∑. When the matrix D to be processed is a full rank matrix, the rotation matrix R is calculated based on the orthogonal matrices P and Q. That is, R = PQ T If the determinant of the calculated rotation matrix is ​​a negative number, the negative number of the determinant is taken as the optimal rotation matrix, and subsequent processing is performed based on the optimal rotation matrix. The residual function Y is constructed according to the rotation matrix R, the optical coordinate data to be processed, and the reference coordinate data to be processed.

[0095]

[0096] Among them, R is the rotation matrix, m i Indicates the reference coordinate data to be processed, h i represents the optical coordinate data to be processed, n represents the number of optical targets at the preset position, and t is the translation vector.

[0097] The residual function is solved so that the sum of the squares of the residual function calculation result Y reaches a minimum value, and the translation vector t is obtained. Based on this, the homogeneous transformation matrix is ​​constructed according to the rotation matrix R and the translation vector t Its composition is:

[0098]

[0099] Based on this, the above homogeneous transformation matrix is ​​used as the first transformation relationship, so as to perform subsequent target transformation relationship determination and image registration processing based on the first transformation relationship.

[0100] The technical solution of this embodiment is to determine the reference centroid coordinate data based on multiple reference coordinate data, and to determine the optical centroid coordinate data based on multiple optical coordinate data. Decentralize each reference coordinate data based on the reference centroid coordinate data to obtain the reference coordinate data to be processed, and decentralizedize each optical coordinate data based on the optical centroid coordinate data to obtain the optical coordinate data to be processed. Based on this, the comparability between the coordinate data of the first coordinate system and the second coordinate system can be increased. Based on the reference coordinate data to be processed and the optical coordinate data to be processed, a matrix to be processed is constructed. Singular value decomposition is performed on the matrix to be processed to determine the rotation matrix and translation vector between the first coordinate system and the second coordinate system, and based on the rotation matrix and the translation vector, the first conversion relationship between the first coordinate system and the second coordinate system is determined. Based on this, it is convenient to determine the target conversion relationship based on the accurately determined first conversion relationship in the future to realize the registration processing of subsequent images. Thereby, the radiation dose received by the surgical area is reduced, and the efficiency and accuracy of image registration are improved.

[0101] Embodiment 3

[0102] Figure 4 Schematic diagram of the structure of an image registration device provided by Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a first conversion relationship determination module 310 , a second conversion relationship determination module 320 , a third conversion relationship determination module 330 , a fourth conversion relationship determination module 340 and an image registration module 350 .

[0103] The first transformation relationship determination module 310 is used to determine the first transformation relationship of the optical target according to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, wherein the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical locator as the origin; the second transformation relationship determination module 320 is used to determine the second transformation relationship according to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, wherein the preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin; the third transformation relationship determination module 330 is used to determine the third transformation relationship according to the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, wherein, The preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as its origin; the fourth transformation relationship determination module 340 is used to determine the fourth transformation relationship based on the intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, wherein the intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as its origin; the image registration module 350 is used to determine the target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship when the fourth transformation relationship does not meet the preset conditions, so as to register the preoperative image and the intraoperative image collected when performing the target task based on the target transformation relationship; wherein the intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as its origin.

[0104] The technical solution of this embodiment determines the first conversion relationship of the optical target according to the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, so as to facilitate the mutual conversion between the reference coordinate data and the optical coordinate data according to the first conversion relationship. Determine the second conversion relationship according to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, so as to facilitate the mutual conversion between the preoperative image coordinate data and the reference coordinate data according to the second conversion relationship. Determine the third conversion relationship according to the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, so as to facilitate the mutual conversion between the preoperative target coordinate data and the optical coordinate data according to the third conversion relationship. Determine the fourth conversion relationship according to the intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, so as to facilitate the mutual conversion between the intraoperative target coordinate data and the preoperative target coordinate data according to the fourth conversion relationship. When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to align the preoperative image and the intraoperative image acquired when performing the target task based on the target transformation relationship. This solves the problems of excessive radiation dose received by the surgical area and low image registration efficiency caused by repeated acquisition of surgical area images and repeated registration in the prior art, realizes real-time automatic registration of preoperative images and intraoperative images, reduces the radiation dose received by the surgical area, improves the efficiency and accuracy of image registration, and thus improves the overall operating efficiency and accuracy of the surgical operating arm.

[0105] On the basis of the above embodiments, optionally, the first transformation relationship determination module includes: a centroid coordinate data determination unit, which is used to determine the reference centroid coordinate data based on multiple reference coordinate data, and to determine the optical centroid coordinate data based on multiple optical coordinate data; a coordinate data decentralization unit, which is used to decentralize each reference coordinate data based on the reference centroid coordinate data to obtain the reference coordinate data to be processed, and to decentralize each optical coordinate data based on the optical centroid coordinate data to obtain the optical coordinate data to be processed; a unit for determining a matrix to be processed, which is used to construct a matrix to be processed based on the reference coordinate data to be processed and the optical coordinate data to be processed; a first transformation relationship determination unit, which is used to perform singular value decomposition on the matrix to be processed, determine the rotation matrix and translation vector between the first coordinate system and the second coordinate system, and determine the first transformation relationship between the first coordinate system and the second coordinate system based on the rotation matrix and the translation vector.

[0106] Optionally, the first transformation relationship determination unit includes: a rotation matrix determination subunit, used to perform singular value decomposition on the matrix to be processed, so as to determine the rotation matrix corresponding to the matrix to be processed when the matrix to be processed is a full rank matrix; a translation vector determination subunit, used to construct a residual function based on the reference coordinate data to be processed, the optical coordinate data to be processed and the rotation matrix, so as to obtain the translation vector between the first coordinate system and the second coordinate system when the residual function satisfies a preset residual condition.

[0107] Optionally, the device further includes: a coordinate data corresponding module, used to determine the coordinate distance value between each reference coordinate data and each optical coordinate data; and sort the coordinate distance values ​​so that the reference coordinate data corresponds to the optical coordinate data.

[0108] Optionally, the image registration module includes: a fifth transformation relationship determination unit, used to determine the fifth transformation relationship between the intraoperative optical target coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship; a target transformation relationship determination unit, used to determine the target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth transformation relationship.

[0109] Optionally, the device also includes: a fourth transformation relationship judgment module, which is used to determine the posture change information of the preoperative optical target and the intraoperative optical target based on the fourth transformation relationship, wherein the posture change information includes displacement change information and attitude angle change information, so as to determine that the fourth transformation relationship does not meet the preset conditions when the posture change information exceeds a preset posture change threshold.

[0110] Optionally, the device also includes: a first transformation relationship updating module, which is used to determine the post-movement optical coordinate data corresponding to each optical target in the second coordinate system after the movement when the movement of the optical locator is detected, and determine the correction matrix based on the optical coordinate data before the movement and the optical coordinate data after the movement; and update the first transformation relationship according to the correction matrix to obtain an updated first transformation relationship.

[0111] The image registration device provided in the embodiment of the present invention can execute the image registration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0112] Embodiment 4

[0113] Figure 51 is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0114] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0115] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0116] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as an image registration method.

[0117] In some embodiments, the image registration method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image registration method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image registration method in any other appropriate manner (e.g., by means of firmware).

[0118] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] The computer program for implementing the image registration method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0120] Embodiment 5

[0121] Embodiment 5 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute an image registration method, the method comprising:

[0122] A first conversion relationship of the optical target is determined based on the reference coordinate data of the optical target in the first coordinate system and the optical coordinate data in the second coordinate system, wherein the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical locator as the origin; a second conversion relationship is determined based on the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, wherein the preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin; a third conversion relationship is determined based on the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, wherein the preoperative optical target coordinate system is A three-dimensional coordinate system with the center of all preoperative optical targets as the origin; determining a fourth transformation relationship based on intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and the preoperative target coordinate data in the preoperative optical target coordinate system, wherein the intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin; when the fourth transformation relationship does not meet the preset conditions, determining a target transformation relationship between the intraoperative image coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to align the preoperative image and the intraoperative image collected when performing the target task based on the target transformation relationship; wherein the intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

[0123] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image registration method, characterized in that: The method is applied to a surgical robot system, the surgical robot system comprising: a surgical operating arm module, a C-arm module and an optical positioning module, the surgical operating arm module and the C-arm module are rigidly connected, the surgical operating arm module is used to operate the surgical area, the C-arm module is used to scan the surgical area image, the optical positioning module comprises an optical positioning instrument, and the optical positioning instrument is used to locate the surgical area. The method comprises: Determine a first conversion relationship of the optical target according to reference coordinate data of the optical target in a first coordinate system and optical coordinate data of the optical target in a second coordinate system, wherein the first coordinate system is a three-dimensional coordinate system with the base center of the surgical operating arm module as the origin, and the second coordinate system is a three-dimensional coordinate system with the optical center of the optical locator as the origin; Determine a second conversion relationship according to the preoperative image coordinate data of the optical target in the preoperative image coordinate system and the reference coordinate data in the first coordinate system, wherein the preoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the preoperative C-arm module as the origin; determining a third conversion relationship according to the preoperative target coordinate data of the optical target in the preoperative optical target coordinate system and the optical coordinate data in the second coordinate system, wherein the preoperative optical target coordinate system is a three-dimensional coordinate system with the center of all preoperative optical targets as the origin; Determining a fourth conversion relationship according to intraoperative target coordinate data of the optical target in the intraoperative optical target coordinate system and preoperative target coordinate data in the preoperative optical target coordinate system, wherein the intraoperative optical target coordinate system is a three-dimensional coordinate system with the center of all intraoperative optical targets as the origin; When the fourth transformation relationship does not meet the preset conditions, the target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined based on the first transformation relationship, the second transformation relationship, the third transformation relationship and the fourth transformation relationship, so as to align the preoperative image and the intraoperative image acquired when performing the target task based on the target transformation relationship; wherein, the intraoperative image coordinate system is a three-dimensional coordinate system with the scanning center corresponding to the intraoperative C-arm module as the origin.

2. The method according to claim 1, characterized in that Determining a first conversion relationship of the optical target according to reference coordinate data of the optical target in the first coordinate system and optical coordinate data in the second coordinate system includes: determining reference centroid coordinate data based on the plurality of reference coordinate data, and determining optical centroid coordinate data based on the plurality of optical coordinate data; Based on the reference centroid coordinate data, each of the reference coordinate data is decentralized to obtain reference coordinate data to be processed, and based on the optical centroid coordinate data, each of the optical coordinate data is decentralized to obtain optical coordinate data to be processed; Constructing a matrix to be processed based on the reference coordinate data to be processed and the optical coordinate data to be processed; Performing singular value decomposition processing on the matrix to be processed, determining a rotation matrix and a translation vector between the first coordinate system and the second coordinate system, and determining a first conversion relationship between the first coordinate system and the second coordinate system based on the rotation matrix and the translation vector.

3. The method according to claim 2, characterized in that The performing singular value decomposition processing on the matrix to be processed to determine the rotation matrix and the translation vector between the first coordinate system and the second coordinate system includes: Performing singular value decomposition processing on the matrix to be processed, so as to determine a rotation matrix corresponding to the matrix to be processed when the matrix to be processed is a full rank matrix; A residual function is constructed based on the reference coordinate data to be processed, the optical coordinate data to be processed, and the rotation matrix, so as to obtain a translation vector between the first coordinate system and the second coordinate system when the residual function satisfies a preset residual condition.

4. The method according to claim 2, characterized in that: Before determining the reference center of mass coordinate data and the optical center of mass coordinate data, the method further includes: Determine a coordinate distance value between each of the reference coordinate data and each of the optical coordinate data; The coordinate distance values ​​are sorted so that the reference coordinate data corresponds to the optical coordinate data.

5. The method according to claim 1, characterized in that The determining, based on the first conversion relationship, the second conversion relationship, the third conversion relationship, and the fourth conversion relationship, of a target conversion relationship between an intraoperative image coordinate system and the first coordinate system comprises: Determine a fifth transformation relationship between the intraoperative optical target coordinate system and the first coordinate system based on the first transformation relationship, the second transformation relationship, the third transformation relationship, and the fourth transformation relationship; Based on the mapping relationship between the intraoperative optical target coordinate system and the intraoperative image coordinate system, and the fifth transformation relationship, a target transformation relationship between the intraoperative image coordinate system and the first coordinate system is determined.

6. The method according to claim 1, characterized in that Also includes: Based on the fourth conversion relationship, the posture change information of the preoperative optical target and the intraoperative optical target is determined, wherein the posture change information includes displacement change information and attitude angle change information, so as to determine that the fourth conversion relationship does not meet the preset condition when the posture change information exceeds a preset posture change threshold.

7. The method according to claim 1, characterized in that Also includes: When the movement of the optical locator is detected, determining the optical coordinate data corresponding to each optical target in the second coordinate system after the movement, and determining the correction matrix according to the optical coordinate data before the movement and the optical coordinate data after the movement; The first conversion relationship is updated according to the correction matrix to obtain an updated first conversion relationship.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the image registration method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image registration method according to any one of claims 1 to 7 when executed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the image registration method according to any one of claims 1 to 7.

Images