Image registration method, apparatus, system, and storage medium

Abstract

The application relates to the technical field of image processing, and provides an image registration method, device, system and storage medium, which can provide anatomical structure information of a wider imaging field of view range during surgery. The application acquires a non-real-time preoperative image of a wide imaging field of view range and a real-time intraoperative image of a narrow imaging field of view range; when the real-time intraoperative image belongs to the first respiratory cycle, the value of an optimizable weight in a deformation field statistical shape model based on respiratory motion is adjusted to obtain a target deformation field for registering the non-real-time preoperative image and the real-time intraoperative image; according to the target value of the optimizable weight used to obtain the target deformation field, registration-related information is obtained, and a correlation relationship between the registration-related information and a respiratory state corresponding to the real-time intraoperative image is obtained; when the real-time intraoperative image belongs to a non-first respiratory cycle, based on the correlation relationship, registration-related information associated with the respiratory state corresponding to the real-time intraoperative image is obtained, and the non-real-time preoperative image and the real-time intraoperative image are registered according to the registration-related information and the deformation field statistical shape model.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image registration method, apparatus, intraoperative real-time guidance system, storage medium, and computer program product. Background Technology

[0002] In image-guided robotic puncture surgery, using real-time CT imaging (CTF, Computed Tomography Fluoroscopy), doctors can observe an extremely narrow area (about 10 mm) around the target area. Through master-slave operation of the robotic arm, the robotic arm can be controlled outside the radiation room, thereby achieving faster and more accurate puncture under the guidance of real-time intraoperative images.

[0003] However, due to the extremely narrow field of view in real-time CT imaging, doctors are almost unable to observe areas outside the very narrow zone surrounding the target area. This means that some lesions located at the base of the lung or the top of the liver, which are greatly affected by respiration, may be out of the imaging field of view during the puncture process due to the fluctuations in respiratory movements. Furthermore, critical tissues and organs outside the imaging field of view may not be visible during the procedure, posing a risk of incomplete puncture and damage to normal tissues and organs, thus limiting the precision and safety of the surgery. Summary of the Invention

[0004] Therefore, it is necessary to provide an image registration method, apparatus, intraoperative real-time guidance system, storage medium, and computer program product to address the aforementioned technical problems.

[0005] This application provides an image registration method, the method comprising:

[0006] Acquire non-real-time preoperative images and real-time intraoperative images of the target object; the imaging field of the non-real-time preoperative image is wider than that of the real-time intraoperative image.

[0007] When the real-time intraoperative image belongs to the first respiratory cycle, the value of the optimizable weight in the deformation field statistical shape model based on respiratory motion is adjusted to obtain the target deformation field for registration of the non-real-time preoperative image and the real-time intraoperative image. Based on the target value of the optimizable weight used to obtain the target deformation field, the registration-related information is obtained, and the correlation between the registration-related information and the respiratory state corresponding to the real-time intraoperative image is obtained.

[0008] When the real-time intraoperative image is not in the first respiratory cycle, the registration-related information of the respiratory state corresponding to the real-time intraoperative image is obtained based on the correlation. According to the target value of the optimizable weight in the registration-related information and the deformation field statistical shape model, the non-real-time preoperative image and the real-time intraoperative image are registered.

[0009] In one embodiment, the registration-related information also includes a deformation image, which is obtained by applying the target deformation field to a non-real-time preoperative image;

[0010] Based on the target values ​​of optimizable weights and the statistical shape model of the deformation field in the registration-related information, the non-real-time preoperative images and real-time intraoperative images are registered, including:

[0011] Obtain the deformation image from the registration-related information;

[0012] Acquire the portion of the deformed image that shares the same imaging field of view as the real-time intraoperative image;

[0013] When the similarity between the image portion and the real-time intraoperative image does not reach the threshold, the target value of the optimizable weight in the registration-related information is used as the initial value for adjustment. The value of the optimizable weight in the deformation field statistical shape model is then adjusted to obtain the target deformation field for registration between the non-real-time preoperative image and the real-time intraoperative image.

[0014] In one embodiment, the method further includes:

[0015] When the similarity between the image portion and the real-time intraoperative image reaches a threshold, the deformed image is used as the registration result between the non-real-time preoperative image and the real-time intraoperative image.

[0016] In one embodiment, based on the correlation, registration-related information concerning the respiratory status associated with the real-time intraoperative image is obtained, including:

[0017] Obtain several relationships;

[0018] Among the respiratory states corresponding to several relationships, determine whether there is a respiratory state that is consistent with the respiratory state corresponding to the real-time intraoperative image.

[0019] If it exists, the respiratory state that is consistent with the respiratory state corresponding to the real-time intraoperative image will be taken as the target respiratory state.

[0020] If it does not exist, the respiratory state adjacent to the respiratory state corresponding to the real-time intraoperative image will be taken as the target respiratory state.

[0021] The correlation of the corresponding target respiratory state is used as the target correlation; the registration-related information corresponding to the target correlation is used as the registration-related information of the respiratory state correlation corresponding to the real-time intraoperative image.

[0022] In one embodiment, the deformation field statistical shape model based on respiratory motion includes several sub-deformation field statistical shape models. Each sub-deformation field statistical shape model is associated with a reference respiratory state and a moving respiratory state. The moving respiratory states associated with each sub-deformation field statistical shape model are the same, while the reference respiratory states associated with each sub-deformation field statistical shape model are different. The non-real-time preoperative image is collected when the target object is in a moving respiratory state.

[0023] When the real-time intraoperative image is within the first respiratory cycle, adjust the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion, including:

[0024] When the real-time intraoperative image belongs to the first respiratory cycle, the target sub-deformation field statistical shape model is determined among several sub-deformation field statistical shape models; the reference respiratory state associated with the target sub-deformation field statistical shape model is matched with the respiratory state corresponding to the real-time intraoperative image.

[0025] Adjust the values ​​of the optimizable weights in the statistical shape model of the target sub-deformation field.

[0026] In one embodiment, the deformation field statistical shape model based on respiratory motion includes several sub-deformation field statistical shape models. Each sub-deformation field statistical shape model is associated with a reference respiratory state and a moving respiratory state. The several sub-deformation field statistical shape models are arranged in sequence, and the reference respiratory state of the previous sub-deformation field statistical shape model is the moving respiratory state of the next sub-deformation field statistical shape model.

[0027] When the real-time intraoperative image is within the first respiratory cycle, adjust the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion, including:

[0028] The target's moving respiratory state is obtained based on the respiratory state corresponding to the non-real-time preoperative image;

[0029] The target reference respiratory status is obtained based on the respiratory status corresponding to the real-time intraoperative image;

[0030] Among several sub-deformation field statistical shape models, the target sub-deformation field statistical shape model is determined; the mobile breathing state associated with the target sub-deformation field statistical shape model is the target mobile breathing state, and the reference breathing state associated with the target sub-deformation field statistical shape model is the target reference breathing state.

[0031] Adjust the values ​​of the optimizable weights in the statistical shape model of the target sub-deformation field.

[0032] In one embodiment, among several sub-deformation field statistical shape models, determining the target sub-deformation field statistical shape model includes:

[0033] Based on the statistical shape models of several sub-deformation fields arranged in sequence, a model sequence is obtained;

[0034] In the model sequence, a portion of the model sequence is extracted; the mobile breathing state associated with the first sub-deformation field statistical shape model in the portion of the model sequence is the target mobile breathing state, and the reference breathing state associated with the last sub-deformation field statistical shape model in the portion of the model sequence is the target reference breathing state.

[0035] Based on a partial model sequence, the statistical shape model of the target sub-deformation field is obtained.

[0036] This application provides an image registration apparatus, the apparatus comprising:

[0037] The image acquisition module is used to acquire non-real-time preoperative images and real-time intraoperative images of the target object; the imaging field of the non-real-time preoperative image is wider than that of the real-time intraoperative image.

[0038] The registration processing module is used to adjust the value of the optimizable weight in the statistical shape model of the deformation field based on respiratory motion when the real-time intraoperative image belongs to the first respiratory cycle, to obtain the target deformation field for registration of the non-real-time preoperative image and the real-time intraoperative image. Based on the target value of the optimizable weight used to obtain the target deformation field, the registration-related information is obtained, and the correlation between the registration-related information and the respiratory state corresponding to the real-time intraoperative image is obtained.

[0039] The registration processing module is also used to obtain the registration-related information of the respiratory state associated with the real-time intraoperative image based on the correlation relationship when the real-time intraoperative image belongs to a non-first respiratory cycle. Based on the target value of the optimizable weight and the deformation field statistical shape model in the registration-related information, the non-real-time preoperative image and the real-time intraoperative image are registered.

[0040] This application provides an intraoperative real-time guidance system, which includes a processor for performing the following steps:

[0041] During the first respiratory cycle, the non-real-time preoperative image of the target subject and the real-time intraoperative image corresponding to each respiratory state in the first respiratory cycle are registered according to the above method, so as to provide real-time intraoperative guidance during the first respiratory cycle.

[0042] During non-first respiratory cycles, the non-real-time preoperative images of the target subject and the real-time intraoperative images corresponding to each respiratory state during non-first respiratory cycles are registered according to the above method in order to provide real-time intraoperative guidance during non-first respiratory cycles.

[0043] In one embodiment, the processor is also configured to perform the following steps:

[0044] The respiratory status of the target object is obtained through a respiratory gating device;

[0045] If the deformation field statistical shape model is constructed under a fixed mobile breathing state, then when the breathing state of the target object matches the fixed mobile breathing state, the target object is scanned to obtain a non-real-time preoperative image of the target object; the non-real-time preoperative image corresponds to the fixed mobile breathing state.

[0046] If the deformation field statistical shape model is constructed under non-fixed mobile respiratory state, then the target object is scanned when the target object is in any respiratory state, and a non-real-time preoperative image of the target object is obtained.

[0047] This application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor using the methods described above.

[0048] This application provides a computer program product having a computer program stored thereon, the computer program being executed by a processor using the methods described above.

[0049] The aforementioned image registration methods, devices, intraoperative real-time guidance systems, storage media, and computer program products, because the imaging field of view of the target object is wider than that of the real-time intraoperative image, can provide anatomical information with a wider imaging field of view when registering non-real-time preoperative images and real-time intraoperative images. Surgeons can observe other parts outside a very narrow area around the target region, improving the accuracy and safety of the surgery. Furthermore, when the real-time intraoperative image is in the first respiratory cycle, the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion are adjusted to obtain the target deformation field for registration between the non-real-time preoperative image and the real-time intraoperative image. The optimizable weights used to obtain the target deformation field... The target value is used to obtain registration-related information, and the correlation between the registration-related information and the respiratory state corresponding to the real-time intraoperative image is obtained. When the real-time intraoperative image is not in the first respiratory cycle, the registration-related information of the respiratory state corresponding to the real-time intraoperative image is obtained based on the correlation. According to the target value of the optimizable weight in the registration-related information and the deformation field statistical shape model, the non-real-time preoperative image and the real-time intraoperative image are registered. Fine-tuning is performed with the registration-related information obtained in the first respiratory cycle, so that the non-real-time preoperative image changes continuously with the different respiratory movements in the real-time intraoperative image. This improves the registration accuracy while ensuring the registration speed, which can meet the time performance requirements of several hundred milliseconds in the real-time CT imaging mode. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating an image registration method in one embodiment;

[0052] Figure 2 This is a comparison diagram of the imaging field of view between a non-real-time preoperative image and a real-time intraoperative image in one embodiment;

[0053] Figure 3 This is a flowchart illustrating the process of adjusting optimizable weights in one embodiment;

[0054] Figure 4 This is a schematic diagram of the respiratory curve in one embodiment;

[0055] Figure 5 This is a flowchart illustrating the process of constructing a statistical shape model of a deformation field in one embodiment;

[0056] Figure 6 A schematic diagram of the overall process for modeling respiratory motion in one embodiment;

[0057] Figure 7 This is a schematic diagram of the image registration process in an individual space in one embodiment;

[0058] Figure 8 This is a schematic diagram of the image registration process in the atlas space and the individual space in one embodiment;

[0059] Figure 9 This is a schematic diagram of a navigation interface in one embodiment;

[0060] Figure 10 This is a structural block diagram of an image registration device in one embodiment;

[0061] Figure 11 This is an internal structure diagram of an intraoperative real-time guidance system in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] The image registration method provided in this application may include... Figure 1 The steps shown are as follows:

[0064] Step S101: Obtain the non-real-time preoperative image and the real-time intraoperative image of the target object.

[0065] The tissue or organ being operated on can be called the target object, such as the lungs or liver. Using CT imaging, preoperative and intraoperative images of the target object are obtained. Compared to intraoperative images, preoperative images are non-real-time images, acquired before the start of surgery; these can be called non-real-time preoperative images. In image-guided robotic puncture surgery, images of the target object can be acquired in real time, resulting in intraoperative images; these can be called real-time intraoperative images.

[0066] The preoperative imaging field of view on at least one imaging plane is wider than the intraoperative imaging field of view on that imaging plane. Therefore, the imaging field of view of the non-real-time preoperative image is wider than that of the real-time intraoperative image.

[0067] exist Figure 2 In the example shown, the imaging plane is the coronal plane. The preoperative imaging field of view in the coronal plane is wider than the intraoperative imaging field of view in the coronal plane. The imaging field of view of the non-real-time preoperative image in the coronal plane is a wide imaging field of view, while the imaging field of view of the real-time intraoperative image in the coronal plane is a narrow imaging field of view, which is approximately a 2D image.

[0068] Step S102: When the real-time intraoperative image belongs to the first respiratory cycle, adjust the value of the optimizable weight in the deformation field statistical shape model based on respiratory motion to obtain the target deformation field for registration of the non-real-time preoperative image and the real-time intraoperative image. Based on the target value of the optimizable weight used to obtain the target deformation field, obtain the registration-related information and obtain the correlation between the registration-related information and the respiratory state corresponding to the real-time intraoperative image.

[0069] The respiratory state can include respiratory phase and / or respiratory amplitude. The deformation field statistical shape model based on respiratory motion is a general model that can be used on different individuals. The deformation field statistical shape model mainly describes the transformation rules between images of different respiratory states. Cross-population respiratory motion modeling methods based on statistical shape models can construct deformation field statistical shape models.

[0070] The statistical shape model of the deformation field can be specifically described as follows: Obtain the product of the optimizable weights and the principal component feature matrix; and sum the product with the average deformation field to obtain the new deformation field. Correspondingly, this can be expressed mathematically as: .

[0071] in, The mean deformation field determined by the cross-population respiratory motion modeling method based on statistical shape models. This is the principal component feature matrix determined by a cross-population respiratory motion modeling method based on statistical shape models. The principal component feature matrix includes the eigenvectors corresponding to the principal components. To optimize weights, For new deformation fields. and Given quantities For optimizable weights.

[0072] After the robotic-guided surgery begins, a specific respiratory cycle can be designated as the first respiratory cycle based on actual needs. Respiratory cycles following this first respiratory cycle are referred to as non-first respiratory cycles.

[0073] When the real-time intraoperative image is within the first respiratory cycle, the optimizable weights can be adjusted multiple times. The value. (Refer to...) Figure 3 Each adjustment can optimize the weights. The value of can be used to obtain a new deformation field; apply this new deformation field to the non-real-time preoperative image to obtain a deformation image; obtain the image portion of the deformation image within the same imaging field of view as the real-time intraoperative image; obtain the content similarity between this image portion and the real-time intraoperative image, which can be obtained through normalized cross-correlation, mutual information, gradient correlation, or mean squared error; if the content similarity does not reach the threshold, it indicates that the new deformation field does not meet the requirements, and the optimizable weights can be further adjusted. The value; if the content similarity reaches the threshold, it means that the new deformation field meets the requirements and can be used as the target deformation field; apply the new deformation field to the non-real-time preoperative image, transform the non-real-time preoperative image to obtain a deformation image, process the deformation image and the real-time intraoperative image, and the processing result can be used for real-time intraoperative guidance.

[0074] Furthermore, it can convert the preoperative tissue segmentation map of the target object into a mesh map; it can also invert the target deformation field and apply it to the mesh map to obtain a deformed mesh map; apply the deformed mesh map and the target deformation field to the non-real-time preoperative map to obtain a deformed image, and process the deformed image and the real-time intraoperative map.

[0075] Additionally, when the content similarity reaches a threshold, the optimizable weights obtained from this adjustment can be used. The value is used as the target value; based on the optimizable weights... The target value is obtained, and registration-related information is acquired; the correlation between registration-related information and the respiratory status corresponding to the real-time intraoperative image is constructed.

[0076] Understandably, with the continuous acquisition of real-time intraoperative images, the correlation between different respiratory states and registration-related information can be established in the first respiratory cycle.

[0077] Step S103: When the real-time intraoperative image is not the first respiratory cycle, based on the correlation, the registration-related information of the respiratory state corresponding to the real-time intraoperative image is obtained. According to the target value of the optimizable weight and the deformation field statistical shape model in the registration-related information, the non-real-time preoperative image and the real-time intraoperative image are registered.

[0078] The association between different respiratory states and registration-related information can be established in the first respiratory cycle. This association can be used in every respiratory cycle after the first respiratory cycle, such as the second respiratory cycle, the third respiratory cycle, and so on. Every respiratory cycle after the first respiratory cycle can be called a non-first respiratory cycle.

[0079] For example, the second respiratory cycle will be used as an example:

[0080] When the acquired real-time intraoperative image belongs to the second respiratory cycle, the corresponding respiratory state can be determined. This respiratory state is then matched with each respiratory state of the first respiratory cycle. Matching methods include, but are not limited to, based on respiratory amplitude, phase, and gradient information. Through matching, registration-related information associated with the respiratory state corresponding to the real-time intraoperative image can be determined in the correlation relationship. Based on the optimizable weights in the registration-related information... The target value and the statistical shape model of the deformation field are used to register the non-real-time preoperative image with the real-time intraoperative image.

[0081] In the aforementioned image registration method, since the imaging field of view of the target object is wider than that of the real-time intraoperative image, registering the non-real-time preoperative image and the real-time intraoperative image can provide anatomical information with a wider imaging field. Surgeons can observe other parts outside a very narrow area around the target region, improving the accuracy and safety of the surgery. Furthermore, when the real-time intraoperative image is in the first respiratory cycle, the values ​​of the optimizable weights in the statistical shape model of the deformation field based on respiratory motion are adjusted to obtain the target deformation field for registration between the non-real-time preoperative image and the real-time intraoperative image. Based on the target value of the optimizable weights used to obtain the target deformation field, registration-related information is obtained. The system obtains the correlation between registration-related information and the respiratory state corresponding to the real-time intraoperative image. When the real-time intraoperative image is not in the first respiratory cycle, based on the correlation, the registration-related information of the respiratory state corresponding to the real-time intraoperative image is obtained. According to the target value of the optimizable weight in the registration-related information and the deformation field statistical shape model, the non-real-time preoperative image and the real-time intraoperative image are registered. Fine-tuning is performed with the registration-related information obtained from the first respiratory cycle, so that the non-real-time preoperative image changes continuously with the different respiratory movements in the real-time intraoperative image. This improves the registration accuracy while ensuring the registration speed, which can meet the time performance requirements of several hundred milliseconds in the real-time CT imaging mode.

[0082] In one embodiment, the registration-related information further includes a deformation image, which is obtained by applying the target deformation field to a non-real-time preoperative image. Registration of the non-real-time preoperative image and the real-time intraoperative image is performed based on the target value of the optimizable weights in the registration-related information and the statistical shape model of the deformation field. This includes: acquiring the deformation image from the registration-related information; acquiring the image portion of the deformation image that shares the same imaging field of view as the real-time intraoperative image; and when the content similarity between the image portion and the real-time intraoperative image does not reach a threshold, adjusting the value of the optimizable weights in the statistical shape model of the deformation field using the target value of the optimizable weights in the registration-related information as the initial adjustment value to obtain the target deformation field for registration of the non-real-time preoperative image and the real-time intraoperative image.

[0083] When the real-time intraoperative image is within the first respiratory cycle, the optimizable weights can be adjusted multiple times. The value. Each adjustment optimizes the weights. The value of can be used to obtain a new deformation field; apply this new deformation field to the non-real-time preoperative image to obtain a deformation image; obtain the image portion of the deformation image within the same imaging field of view as the real-time intraoperative image; obtain the content similarity between this image portion and the real-time intraoperative image, which can be obtained through normalized cross-correlation, mutual information, gradient correlation, or mean squared error; if the content similarity does not reach the threshold, it indicates that the new deformation field does not meet the requirements, and the optimizable weights can be further adjusted. The value; if the content similarity reaches the threshold, it means that the new deformation field meets the requirements and can be used as the target deformation field; apply the new deformation field to the non-real-time preoperative image, transform the non-real-time preoperative image to obtain a deformation image, and the result of processing the deformation image and the real-time intraoperative image can be used for real-time intraoperative guidance, providing anatomical structural information with a wider imaging field of view during the operation.

[0084] Additionally, when the content similarity reaches a threshold, the optimizable weights obtained from this adjustment can be used. The value is used as the target value; based on the optimizable weights... The target value and the new deformation field are applied to the deformation image obtained from the non-real-time preoperative image to form registration-related information; at this time, the registration-related information includes The target value and deformation image were obtained; the correlation between registration-related information and the respiratory status corresponding to the real-time intraoperative image was constructed.

[0085] Additionally, the deformed mesh image can be used as one of the registration-related information items. In this case, the registration-related information includes... The target value, the deformed image, and the deformed mesh image.

[0086] With continuous acquisition of real-time intraoperative images, the correlation between different respiratory states and registration-related information can be established during the first respiratory cycle.

[0087] Let's take the second respiratory cycle as an example, which is not the first respiratory cycle:

[0088] Because respiratory curves differ across different respiratory cycles, such as respiratory amplitude and respiratory rate, for example... Figure 4 In the example shown, for Figure 4 The curve shown is segmented to obtain respiratory curves for multiple respiratory cycles, each with its own differences. Therefore, the deformation image or deformed mesh image obtained in the first respiratory cycle is not used directly. Before use, respiratory state matching is performed. That is, among the multiple respiratory states in the first respiratory cycle, the respiratory state matching the real-time intraoperative image is determined; the deformation image associated with the matched respiratory state is obtained; the similarity between the image portion in the deformation image within the same imaging field of view as the real-time intraoperative image and the content of the real-time intraoperative image is determined; based on whether the content similarity reaches a threshold, it is determined whether to directly use the deformation image or deformed mesh image obtained in the first respiratory cycle.

[0089] When registration-related information includes optimizable weights When calculating the target value and deformed image, if the acquired real-time intraoperative image belongs to the second respiratory cycle, the respiratory state corresponding to the real-time intraoperative image can be determined. Registration-related information associated with this respiratory state is identified in the correlation relationship. The deformed image in the registration-related information is obtained, and the image portion within the deformed image that shares the same imaging field of view as the real-time intraoperative image is determined. The content similarity between this image portion and the real-time intraoperative image is then obtained. If the content similarity reaches a threshold, the deformed image and the real-time intraoperative image can be directly processed, and the processing result can be used for real-time intraoperative guidance without needing to transform the non-real-time preoperative image using a deformation field. If the content similarity does not reach a threshold, the optimizable weights can be adjusted. Fine-tuning is then performed; at this point, the optimizable weights from the relevant registration information are used. The objective value is to adjust the initial value for the optimizable weights. The search is performed on the value; each time an optimizable weight is found... If the value is found, a new deformation field can be obtained by combining it with the deformation field statistical shape model. This new deformation field is then applied to the non-real-time preoperative image to obtain a deformation image. The portion of this deformation image that shares the same imaging field of view as the real-time intraoperative image is then obtained, and the content similarity between this portion and the real-time intraoperative image is calculated. If the content similarity does not reach the threshold, it indicates that the new deformation field does not meet the requirements, and the search for optimizable weights can continue. The value; if the content similarity reaches the threshold, it means that the new deformation field meets the requirements. The new deformation field is used as the target deformation field. The target deformation field is applied to the non-real-time preoperative image to obtain the deformation image. The result of processing the deformation image and the real-time intraoperative image can be used for real-time intraoperative guidance, providing anatomical structural information with a wider imaging field of view during the operation.

[0090] When registration-related information includes optimizable weights When acquiring the target value, deformed image, and deformed mesh image, if the acquired real-time intraoperative image belongs to the second respiratory cycle, the respiratory state corresponding to the real-time intraoperative image can be determined, and the registration-related information associated with the respiratory state can be identified in the correlation. The deformed image in the registration-related information is then obtained, and the image portion within the deformed image that shares the same imaging field of view as the real-time intraoperative image is identified. The content similarity between this image portion and the real-time intraoperative image is then obtained. If the content similarity reaches a threshold, the deformed image, the deformed mesh image in the registration-related information, and the real-time intraoperative image can be directly processed. The processing result can be used for real-time intraoperative guidance, eliminating the need to transform the non-real-time preoperative image using a deformation field. If the content similarity does not reach a threshold, the optimizable weights can be adjusted. Fine-tuning is then performed; at this point, the optimizable weights from the relevant registration information are used. The objective value is to adjust the initial value for the optimizable weights. The search is performed on the value, and each time an optimizable weight is found... If the value is found, a new deformation field can be obtained by combining it with the deformation field statistical shape model. This new deformation field is then applied to the non-real-time preoperative image to obtain a deformation image. The portion of this deformation image that shares the same imaging field of view as the real-time intraoperative image is then obtained, and the content similarity between this portion and the real-time intraoperative image is calculated. If the content similarity does not reach the threshold, it indicates that the new deformation field does not meet the requirements, and the search for optimizable weights can continue. The value of the content similarity reaches the threshold, indicating that the new deformation field meets the requirements and is used as the target deformation field. The target deformation field is applied to the non-real-time preoperative image to obtain the deformation image, and the target deformation field is applied to the mesh image to obtain the deformed mesh image. The deformation image, the deformed mesh image and the real-time intraoperative image are processed, and the processing results can be used for real-time intraoperative guidance.

[0091] The search algorithms used above may include, but are not limited to, Powell's algorithm, genetic algorithm, particle swarm optimization, etc.

[0092] In the above embodiments, when establishing the association in the first respiratory cycle, the deformation image obtained by applying the target deformation field to the non-real-time preoperative image is used as one of the registration-related information. In subsequent respiratory cycles, for the acquired real-time intraoperative image, the registration-related information associated with the respiratory state corresponding to the real-time intraoperative image is obtained. The image portion of the deformation image in the registration-related information that is in the same imaging field of view as the real-time intraoperative image is obtained. Based on the content similarity between the image portion and the real-time intraoperative image, it is determined whether the deformation image is usable. If the content similarity reaches the threshold, the deformation image is usable, and the deformation image and the real-time intraoperative image can be directly processed. The processing result is used for real-time intraoperative guidance, and the registration accuracy will not be significantly lost. Moreover, there is no need to search for the value of the optimizable weight, thus ensuring the registration speed. If the content similarity reaches the threshold, the deformation image is unusable. In this case, the target value of the optimizable weight in the registration-related information can be used as the initial value for adjustment. At this time, the search space is small, and it can converge quickly, thus ensuring the registration speed.

[0093] In one embodiment, the method provided by this application further includes: when the similarity between the image portion and the content of the real-time intraoperative image reaches a threshold, using the deformed image as the registration result between the non-real-time preoperative image and the real-time intraoperative image.

[0094] After establishing the correlation between different respiratory states and registration-related information in the first respiratory cycle, the second respiratory cycle (not the first) will be used as an example for further explanation:

[0095] When the acquired real-time intraoperative image belongs to the second respiratory cycle, the respiratory state corresponding to the real-time intraoperative image can be determined, and the registration-related information associated with the respiratory state can be determined in the correlation. The deformation image in the registration-related information is obtained, and the image part in the deformation image that is in the same imaging field of view as the real-time intraoperative image is determined. The content similarity between the image part and the real-time intraoperative image is obtained. If the content similarity reaches a threshold, the deformation image and the real-time intraoperative image can be directly processed. The processing result can be used for real-time intraoperative guidance without the need to use the deformation field to transform the non-real-time preoperative image.

[0096] In the above embodiments, when establishing the association in the first respiratory cycle, the deformation image obtained by applying the target deformation field to the non-real-time preoperative image is used as one of the registration-related information. In subsequent respiratory cycles, for the acquired real-time intraoperative image, the registration-related information associated with the respiratory state corresponding to the real-time intraoperative image is obtained. The image portion of the deformation image in the registration-related information that is in the same imaging field of view as the real-time intraoperative image is obtained. Based on the content similarity between the image portion and the real-time intraoperative image, it is determined whether the deformation image is usable. If the content similarity reaches a threshold, the deformation image is usable, and the deformation image and the real-time intraoperative image can be directly processed. The processing result is used for real-time intraoperative guidance, and the registration accuracy will not be significantly lost. Moreover, there is no need to search for values ​​of optimizable weights, ensuring the registration speed.

[0097] In one embodiment, based on the association relationships, the registration-related information of the respiratory state corresponding to the real-time intraoperative image is obtained, including: acquiring several association relationships; among the respiratory states corresponding to each of the several association relationships, determining whether there is a respiratory state consistent with the respiratory state corresponding to the real-time intraoperative image; if there is, taking the respiratory state consistent with the respiratory state corresponding to the real-time intraoperative image as the target respiratory state; if there is no, taking the respiratory state adjacent to the respiratory state corresponding to the real-time intraoperative image as the target respiratory state; taking the association relationship corresponding to the target respiratory state as the target association relationship; and taking the registration-related information corresponding to the target association relationship as the registration-related information of the respiratory state corresponding to the real-time intraoperative image.

[0098] The following is an example of a respiratory cycle that includes respiratory states 1 to 5, and is not the first respiratory cycle but the second respiratory cycle.

[0099] During the first respiratory cycle, the following relationships can be established: the relationship between respiratory state 1 and registration-related information (which can be called the first relationship), the relationship between respiratory state 2 and registration-related information (which can be called the second relationship), the relationship between respiratory state 3 and registration-related information (which can be called the third relationship), the relationship between respiratory state 4 and registration-related information (which can be called the fourth relationship), and the relationship between respiratory state 5 and registration-related information (which can be called the fifth relationship).

[0100] When the acquired real-time intraoperative image belongs to the second respiratory cycle, the respiratory state corresponding to the real-time intraoperative image can be obtained, and it can be determined whether there is a respiratory state that is consistent with the respiratory state corresponding to the real-time intraoperative image among the respiratory states corresponding to the aforementioned five correlations.

[0101] For example, if the respiratory state corresponding to the real-time intraoperative image is respiratory state 1, then it can be determined that: among the respiratory states corresponding to the aforementioned five correlations, there exists a respiratory state that is consistent with the respiratory state corresponding to the real-time intraoperative image; that is, the respiratory state corresponding to the first correlation is consistent with the respiratory state corresponding to the real-time intraoperative image; and the respiratory state 1 corresponding to the first correlation can be taken as the target respiratory state.

[0102] For example, if the respiratory state corresponding to the real-time intraoperative image is respiratory state 1.5, then it can be determined that: among the respiratory states corresponding to the aforementioned five correlations, there is no respiratory state that is consistent with the respiratory state corresponding to the real-time intraoperative image; if the respiratory state 1.5 corresponding to the real-time intraoperative image is located between respiratory states 1 and 2, then respiratory state 1 or 2 can be taken as the target respiratory state.

[0103] After determining the target breathing state, the corresponding relationship between the target breathing state can be used as the target relationship; for example, if the target breathing state is breathing state 1, then the first relationship can be used as the target relationship.

[0104] The registration-related information corresponding to the target association is used as the registration-related information associated with the respiratory state corresponding to the real-time intraoperative image; for example, if the target association is the first association, then the registration-related information corresponding to the first association (that is, the registration-related information associated with respiratory state 1) is used as the registration-related information associated with the respiratory state corresponding to the real-time intraoperative image.

[0105] In the above embodiments, when the real-time intraoperative image is not the first respiratory cycle, if the respiratory state corresponding to the real-time intraoperative image was not collected and calculated in the first respiratory cycle; due to the registration-related information of adjacent respiratory states... Therefore, in cases where the respiratory status is lacking, the respiratory status adjacent to the respiratory status corresponding to the real-time intraoperative image can be used as the target respiratory status. This allows for fine-tuning of the corresponding registration-related information, ensuring the normal progress of real-time intraoperative guidance while also considering registration efficiency.

[0106] In one embodiment, considering the difficulty in acquiring 4D CT images for each individual during interventional surgery, and the challenge in establishing personalized respiratory motion models for each individual, this application employs a cross-population respiratory motion modeling method based on statistical shape models. This method studies respiratory motion patterns in different populations, analyzes general respiratory motion laws, and obtains a deformation field statistical shape model. This model is applicable to different individuals and can better adapt to the respiratory conditions of different individuals in actual interventional scenarios. Constructing the deformation field statistical shape model mainly includes three steps: atlas creation, deformation field acquisition, and modeling, such as... Figure 5 As shown.

[0107] The process for constructing a statistical shape model of a deformation field is as follows:

[0108] 1. Atlas creation.

[0109] The purpose of creating the atlas is to transfer all individuals to the same space, so that the same point corresponds to the same anatomical structure, in order to find the movement pattern of the same anatomical structure in different individuals with respiratory movements.

[0110] The process for creating the atlas is as follows:

[0111] (1) Select the initial template image.

[0112] Based on the 4D CT images of each individual during non-surgical periods, images of each individual under different respiratory states can be obtained. Images of different individuals under the same respiratory state are selected, and one image is chosen from these images as the initial template image.

[0113] For example, suppose there are 100 individuals, and 10 images of respiratory states are collected within a single respiratory cycle for each individual. These 10 respiratory states are numbered {00, 10, ..., 90}, and the images of the 10 respiratory states of the i-th individual are denoted as {image_p i _00、image_p i _10、…、image_p i _90}, the 10 images of the i-th individual's respiratory states can be called the image sequence of the i-th individual. Images of the same respiratory state are selected from the image sequence of 100 individuals, such as image_p1_00, image_p2_00, ..., image_p 100 _00, thus obtaining 100 images with a breathing state of 00, and selecting one image from the 100 images with a breathing state of 00 as the initial template image.

[0114] Then repeat steps (2) to (5) until the stopping condition is met, and use the new template image obtained when the stopping condition is met as the target template image.

[0115] (2) Registration.

[0116] Determine the initial template image for this round. In the first cycle, after determining the initial template image from images of different individuals with the same respiratory state, set the initial template image as the initial template image for this round. In subsequent cycles, use the new template image from the previous round as the initial template image for this round.

[0117] The initial template image can be registered with other images of different individuals in the same respiratory state, excluding the initial template image, to obtain the registered image.

[0118] For example, in the first loop, at image_p1_00, image_p2_00, ..., image_p 100 Of these 100 images, image_p1_00 is selected as the initial template image. Therefore, image_p1_00 is the initial template image for this round. Image_p2_00 is registered with image_p1_00 to obtain the registered image image'_p2_00. Similarly, image_p3_00 is registered with image_p1_00 to obtain the registered image image'_p3_00. This process can be repeated to obtain image'_p4_00, ..., image'_p... 100 _00.

[0119] (3) Obtain the average template image.

[0120] The average template image is obtained by averaging the initial template image and the registered image.

[0121] For example, in the first loop, the initial template image is image_p1_00, and the registered images include image'_p2_00, image'_p3_00, image'_p4_00, ..., image'_p 100 _00, for image_p1_00, image'_p2_00, image'_p3_00, image'_p4_00,..., image'_p 100 The average of _00 is calculated to obtain the average template image.

[0122] (4) Obtain the average registration relationship.

[0123] In (2), by registering other images with the initial template image of this round, the deformation field between other images and the initial template image of this round can be obtained. The average deformation field is obtained by averaging multiple deformation fields.

[0124] For example, in the first loop, at image_p1_00, image_p2_00, ..., image_p 100 Of the 100 images, image_p1_00 is selected as the initial template image. Since image_p1_00 is the initial template image for this round, the deformation field def_p between image_p2_00 and image_p1_00 can be obtained. 2-1 The deformation field def_p between image_p3_00 and image_p1_00 3-1 _00, ..., image_p 100The deformation field def_p between _00 and image_p1_00 100-1 _00, average these 99 deformation fields to obtain the average deformation field.

[0125] (5) Obtain a new template image.

[0126] The inverse of the average deformation field is applied to the average template image to obtain the new template image for this round. If the new template image and the initial template image satisfy the similarity test, the next round of iteration can be stopped, and the new template image is used as the target template image. If the new template image and the initial template image do not satisfy the similarity test, the next round of iteration begins.

[0127] 2. Deformation field acquisition.

[0128] An individual's 4D CT images include 3D CT images of that individual at different respiratory states throughout a complete respiratory cycle, such as 3D CT images of different respiratory states from end-inspiratory to end-expiratory to end-inspiratory. These images are obtained at different respiratory states, and the deformation field characterization obtained by registering these images represents how the respiratory state changes from one respiratory state to another; therefore, the deformation field can be used for respiratory motion modeling.

[0129] As previously assumed, images of 10 respiratory states are collected within one respiratory cycle, and these 10 respiratory states are denoted as 00, 10, 20, ..., 90.

[0130] Based on subtle differences in the deformation field, the statistical shape model of the deformation field can be divided into the statistical shape model of the deformation field under independent breathing conditions, the statistical shape model of the deformation field under full breathing conditions, and the statistical shape model of the deformation field under continuous breathing conditions.

[0131] Statistical shape model of deformation field for independent breathing states: Modeling is performed under the condition of fixed mobile breathing states; when one breathing state is used as the fixed mobile breathing state, the other breathing states are used as reference breathing states.

[0132] In subsequent application phases, the respiratory status of the non-real-time preoperative image remains consistent with the fixed mobile respiratory status.

[0133] When respiratory state 00 is taken as a fixed moving respiratory state, respiratory states 10 to 90 are taken as reference respiratory states. In this case, the possible combinations of [moving respiratory state, reference respiratory state] include: [00, 10], [00, 20], ..., [00, 90]. The image corresponding to the reference respiratory state is the reference image, and the image of the moving respiratory state is the moving image.

[0134] After obtaining the target template image, the deformation fields of all individuals from the mobile respiratory state to the reference respiratory state are unified into the spectral space where the target template image is located.

[0135] As in the previous example, images of different individuals at respiratory state 00 are selected to construct the target template image. Taking the i-th individual as an example, the images of the i-th individual at 10 respiratory states are {image_p i _00、image_p i _10、…、image_p i _90}.

[0136] Method 1, based on {image_p i _00、image_p i _10、…、image_p i _90} allows us to obtain the deformation fields of the i-th individual in all possible combinations of [movement breathing state, reference breathing state]. These deformation fields belong to the non-spectral space. Based on image_p i _00 is registered to the deformation field of the target template image, transforming the deformation field in the non-spectral space to the spectral space.

[0137] Method two can be based on image_p i _00 is registered to the deformation field of the target template image, and {image_p i _00、image_p i _10、…、image_p i _90} is transformed into the atlas space, resulting in {image''_p i _00、image''_p i _10、…、image''_p i _90}, thus obtaining the deformation fields of the i-th individual in various possible combinations of [moving breathing state, reference breathing state], these deformation fields belong to the deformation fields in the atlas space.

[0138] Statistical shape model of deformation field in all breathing states: Modeling is performed under non-fixed, mobile breathing states; each breathing state is considered a mobile breathing state; when a certain breathing state is considered a mobile breathing state, other breathing states are considered reference breathing states. [Mobile breathing state, reference breathing state] can have various possible combinations, such as [10,00], [00,30], [20,30], etc.

[0139] After obtaining the target template image, the deformation fields of all individuals from the mobile respiratory state to the reference respiratory state are unified into the spectral space where the target template image is located.

[0140] As in the previous example, images of different individuals at respiratory state 00 are selected to construct the target template image. Taking the i-th individual as an example, the images of the i-th individual at 10 respiratory states are {image_p i _00、image_p i _10、…、image_p i _90}.

[0141] Method 1, based on {image_p i _00、image_p i _10、…、image_p i _90} can be used to obtain the deformation fields of the i-th individual in all possible combinations of [movement breathing state, reference breathing state]. These deformation fields belong to the non-spectral space. Based on image_p i _00 is registered to the deformation field of the target template image, transforming the deformation field from non-spectral space to spectral space.

[0142] Method two can be based on image_p i _00 is registered to the deformation field of the target template image, and {image_p i _00、image_p i _10、…、image_p i _90} is transformed into the atlas space, resulting in {image''_p i _00、image''_p i _10、…、image''_p i _90}, thus obtaining the deformation fields of the i-th individual in various possible combinations of [moving breathing state, reference breathing state], these deformation fields belong to the deformation fields in the atlas space.

[0143] Statistical shape model of deformation field in continuous breathing state: Modeling is performed under the condition of non-fixed moving breathing phase; each breathing phase is used as a moving breathing phase, and the breathing phase used as the reference breathing phase is adjacent to the breathing phase used as the moving breathing phase. Taking the aforementioned 10 breathing states as an example, the possible combinations of [moving breathing state, reference breathing state] are [00,10], [10,20], ..., [80,90].

[0144] After obtaining the target template image, the deformation fields of all individuals from the mobile respiratory state to the reference respiratory state are unified into the spectral space where the target template image is located.

[0145] As in the previous example, images of different individuals at respiratory state 00 are selected to construct the target template image. Taking the i-th individual as an example, the images of the i-th individual at 10 respiratory states are {image_pi _00、image_p i _10、…、image_p i _90}.

[0146] Method 1, based on {image_p i _00、image_p i _10、…、image_p i _90} can be used to obtain the deformation fields of the i-th individual in all possible combinations of [movement breathing state, reference breathing state]. These deformation fields belong to the non-spectral space. Based on image_p i _00 is registered to the deformation field of the target template image, transforming the deformation field from non-spectral space to spectral space.

[0147] Method two can be based on image_p i _00 is registered to the deformation field of the target template image, and {image_p i _00、image_p i _10、…、image_p i _90} is transformed into the atlas space, resulting in {image''_p i _00、image''_p i _10、…、image''_p i _90}, thus obtaining the deformation fields of the i-th individual in various possible combinations of [moving breathing state, reference breathing state], these deformation fields belong to the deformation fields in the atlas space.

[0148] Optionally, the registration method is not limited to non-deep learning registration methods such as Demons and SyN; deep learning-based registration methods such as VoxelMorph and SymNet can also be used.

[0149] 3. Respiratory motion modeling.

[0150] The overall process of respiratory motion modeling is as follows: Figure 6 As shown, the statistical shape model is used to find the law of deformation field. The statistical shape model is such as the PCA model: .

[0151] Reference Figure 6 , Figure 6 In this context, SSModel represents the statistical shape model, the curve represents the breathing curve, and there are points on the breathing curve with corresponding grid plots. The points represent the breathing state, and the grid plots represent the deformation field. The average deformation field is then determined. and principal component feature matrix Then, adjust the optimizable weights. The value of can be used to obtain the corresponding new deformation field. ; Figure 6 The term "different shape" refers to six grid diagrams, each corresponding to a new deformation field. .

[0152] After the second step, we can obtain the deformation field of each individual in the atlas space from the moving breathing state to the reference breathing state, and use these deformation fields to construct the corresponding deformation field statistical shape model.

[0153] If we construct a statistical shape model of the deformation field for independent breathing states, in the deformation field of any individual from the moving breathing state to the reference breathing state, the moving breathing state is the same, but the reference breathing state is different.

[0154] Taking the i-th individual with a movement breathing state of 00 as an example, the deformation field of the i-th individual in the atlas space from the movement breathing state to the reference breathing state can be denoted as: {def model _i_00-10、def model _i_00-20、def model _i_00-30、...、def model _i_00-90}.

[0155] In the graph space, for each individual from a moving breathing state to a reference breathing state, the deformation fields of the same reference breathing state are selected, and a feature matrix corresponding to that reference breathing state is constructed. Each column of the feature matrix represents a deformation field, the number of columns in the feature matrix is ​​the number of deformation fields, and the number of rows in the feature matrix is ​​the size of the deformation fields. The number of deformation fields constituting the feature matrix can be determined according to requirements. A smaller number of deformation fields results in a statistical shape model of the deformation field containing more global features, while a larger number of deformation fields results in a statistical shape model of the deformation field containing more local features, and thus more parameters to be solved subsequently.

[0156] For example, selecting the deformation fields of 100 individuals with a mobile respiratory state of 00 and a reference respiratory state of 10, and constructing the feature matrices corresponding to the mobile respiratory state 00 and the reference respiratory state 10, can be represented as [def model _1_00-10、def model _2_00-10、def model _3_00-10、...、def model _100_00-10].

[0157] For example, selecting the deformation fields of 100 individuals with a mobile respiratory state of 00 and a reference respiratory state of 20, and constructing the feature matrix corresponding to the mobile respiratory state 00 and the reference respiratory state 20, can be represented as [def model_1_00-20、def model _2_00-20、def model _3_00-20、...、def model _100_00-20].

[0158] For the feature matrix corresponding to each reference breathing state, the average deformation field can be modeled. Principal component feature matrix and optimizable weights This yields the statistical shape model of the sub-deformation field corresponding to each reference breathing state. For example, the statistical shape models of the sub-deformation field corresponding to moving breathing state 00 and reference breathing state 10. For example, the statistical shape model of the sub-deformation field corresponding to the mobile breathing state 00 and the reference breathing state 20. .

[0159] In each sub-deformation field statistical shape model corresponding to the reference breathing state, the moving breathing state is the same. Taking the previous example, the moving breathing state of these sub-deformation field statistical shape models is 00. Each sub-deformation field statistical shape model is associated with a moving breathing state and a reference breathing state.

[0160] Based on the feature matrices corresponding to the mobile breathing state 00 and the reference breathing state 10, a corresponding sub-deformation field statistical shape model is obtained. The mobile breathing state associated with this sub-deformation field statistical shape model is 00, and the reference breathing state is 10. This sub-deformation field statistical shape model can be denoted as M_00-10.

[0161] Similarly, the statistical shape models for other sub-deformation fields can be denoted as M_00-20, M_00-30, ..., M_00-90.

[0162] These sub-deformation field statistical shape models are associated with the same mobile breathing state, but each sub-deformation field statistical shape model is associated with a different reference breathing state. These sub-deformation field statistical shape models constitute the deformation field statistical shape model. That is, the deformation field statistical shape model includes several sub-deformation field statistical shape models, such as M_00-10, M_00-20, M_00-30, ..., M_00-90. In these sub-deformation field statistical shape models, the mobile breathing state is 00, but the reference breathing state is different. Therefore, this deformation field statistical shape model is called the deformation field statistical shape model of independent breathing state.

[0163] If an independent respiratory state deformation field statistical shape model is constructed, the non-real-time preoperative image must be collected when the target subject is in a moving respiratory state. In this case, when the real-time intraoperative image belongs to the first respiratory cycle, the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion are adjusted, including: when the real-time intraoperative image belongs to the first respiratory cycle, determining the target sub-deformation field statistical shape model among several sub-deformation field statistical shape models; matching the reference respiratory state associated with the target sub-deformation field statistical shape model with the respiratory state corresponding to the real-time intraoperative image; and adjusting the values ​​of the optimizable weights in the target sub-deformation field statistical shape model.

[0164] As in the previous example, the sub-deformation field statistical shape model includes the following sub-deformation field statistical shape models: M_00-10, M_00-20, M_00-30, ..., M_00-90. When the real-time intraoperative image belongs to the first respiratory cycle, the respiratory state corresponding to the real-time intraoperative image is determined. If the respiratory state corresponding to the real-time intraoperative image is 10, then among the several sub-deformation field statistical shape models, the associated reference respiratory state 10 can be determined, namely M_00-10. M_00-10 is then used as the target sub-deformation field statistical shape model, and the optimizable weights in M_00-10 are adjusted multiple times. The value of .

[0165] Each adjustment The value of can be used to obtain a new deformation field. This new deformation field is applied to the non-real-time preoperative image to obtain a deformed image. The portion of this deformed image that shares the same imaging field of view as the real-time intraoperative image is then obtained, and the content similarity between this portion and the real-time intraoperative image is calculated. If the content similarity does not reach the threshold, it indicates that the new deformation field does not meet the requirements and can be further adjusted. The value; if the content similarity reaches the threshold, it means that the new deformation field meets the requirements, and the new deformation field can be used as the target deformation field. The target deformation field is applied to the non-real-time preoperative image to obtain the deformation image. The deformation image and the real-time intraoperative image are processed, and the processing result is used for real-time intraoperative guidance; in addition, when the content similarity reaches the threshold, the optimizable weights obtained in this adjustment can be... The value is used as the target value, based on the optimizable weights. The target value is obtained, registration-related information is acquired, and the correlation between the registration-related information and the respiratory status corresponding to the real-time intraoperative image is constructed.

[0166] If we construct a statistical shape model of the deformation field of the entire breathing state, we can model it under the condition of non-fixed moving breathing state; each breathing state is regarded as a moving breathing state; when a certain breathing state is regarded as a moving breathing state, other breathing states are regarded as reference breathing states.

[0167] After obtaining the deformation fields of each individual in the atlas space from the moving respiratory state and the reference respiratory state, each deformation field is treated as a feature vector, and these deformation fields are incorporated into the same feature matrix. The average deformation field is then obtained through modeling. Principal component feature matrix and optimizable weights This leads to the statistical shape model of the deformation field, which can be expressed mathematically as follows: .

[0168] If a statistical shape model of the deformation field under all respiratory states is constructed, non-real-time preoperative images can be acquired when the target object is in any respiratory state. When the real-time intraoperative image is in the first respiratory cycle, regardless of the respiratory state of the real-time intraoperative image, the target deformation field can be obtained by adjusting the optimizable weight values ​​in the statistical shape model of the deformation field to register the non-real-time preoperative image with the real-time intraoperative image.

[0169] If we construct a deformation field statistical shape model for continuous breathing states, taking the aforementioned 10 breathing states as an example, the sub-deformation field statistical shape models included in this model can be denoted as M_00-10, M_10-20, M_20-30, ..., M_80-90, respectively. These sub-deformation field statistical shape models are all associated with a moving breathing state and a reference breathing state. Several sub-deformation field statistical shape models are arranged sequentially, with the reference breathing state of the preceding sub-deformation field statistical shape model serving as the moving breathing state of the following sub-deformation field statistical shape model.

[0170] Taking the acquisition of the statistical shape model M_00-10 of the sub-deformation field as an example, the following will be used as an example:

[0171] We can select deformation fields from 100 individuals whose mobile breathing state is 00 and whose reference breathing state is 10 to construct the feature matrix corresponding to the mobile breathing state 00 and the reference breathing state 10, which can be represented as [def model _1_00-10、def model _2_00-10、def model _3_00-10、...、def model [_100_00-10]. Based on this characteristic matrix, the average deformation field is obtained through modeling. Principal component feature matrix and optimizable weights The statistical shape model M_00-10 of the sub-deformation field is obtained for the mobile breathing state 00 and the reference breathing state 10. .

[0172] If a statistical shape model of the deformation field in a continuous breathing state is constructed, non-real-time preoperative images can be acquired when the target object is in any breathing state.

[0173] When the real-time intraoperative image belongs to the first respiratory cycle, adjust the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion, including: obtaining the target moving respiratory state based on the respiratory state corresponding to the non-real-time preoperative image; obtaining the target reference respiratory state based on the respiratory state corresponding to the real-time intraoperative image; determining the target sub-deformation field statistical shape model among several sub-deformation field statistical shape models; the moving respiratory state associated with the target sub-deformation field statistical shape model is the target moving respiratory state, and the reference respiratory state associated with the target sub-deformation field statistical shape model is the target reference respiratory state; and adjust the values ​​of the optimizable weights in the target sub-deformation field statistical shape model.

[0174] As in the previous example, the sub-deformation field statistical shape models included in the deformation field statistical shape model are: M_00-10, M_10-20, M_20-30, ..., M_80-90. When the real-time intraoperative image belongs to the first respiratory cycle, the respiratory state corresponding to the real-time intraoperative image is determined, and the respiratory state corresponding to the real-time intraoperative image is used as the target reference respiratory state, while the respiratory state corresponding to the non-real-time preoperative image is used as the target moving respiratory state.

[0175] Among several sub-deformation field statistical shape models, determine whether there exists a sub-deformation field statistical shape model whose associated mobile breathing state and reference breathing state are the target mobile breathing state and the target reference breathing state, respectively.

[0176] If a statistical shape model for the sub-deformation field exists, then that statistical shape model for the sub-deformation field is taken as the target statistical shape model for the sub-deformation field.

[0177] For example, if the respiratory state corresponding to the non-real-time preoperative image is 00 and the respiratory state corresponding to the real-time intraoperative image is 10, then the target moving respiratory state is 00 and the target reference respiratory state is 10. It can be determined that the moving respiratory state and reference respiratory state associated with M_00-10 are 00 and 10 respectively. Therefore, M_00-10 can be used as the statistical shape model of the target sub-deformation field, and the optimizable weights in M_00-10 can be adjusted multiple times. The value of .

[0178] If the statistical shape model of the sub-deformation field does not exist, then among several statistical shape models of the sub-deformation field, the target statistical shape model of the sub-deformation field is determined, including: obtaining a model sequence based on several statistical shape models of the sub-deformation field arranged in sequence; extracting a portion of the model sequence; the mobile breathing state associated with the first statistical shape model of the sub-deformation field in the portion of the model sequence is the target mobile breathing state, and the reference breathing state associated with the last statistical shape model of the sub-deformation field in the portion of the model sequence is the target reference breathing state; and obtaining the target statistical shape model of the sub-deformation field based on the portion of the model sequence.

[0179] For example, if the respiratory state corresponding to the non-real-time preoperative image is 00 and the respiratory state corresponding to the real-time intraoperative image is 40, then the target moving respiratory state is 00 and the target reference respiratory state is 40. In M_00-10, M_10-20, M_20-30, ..., M_80-90, there is no sub-deformation field statistical shape model with a moving respiratory state of 00 and a reference respiratory state of 40. At this point, several sequentially arranged sub-deformation field statistical shape models can be obtained, resulting in the model sequence {M_00-10→M_10-20→M_20-30→M_30-40→M_40-50→M_50-60→M_60-70→M_70-80→M_80-90}. A portion of the model sequence is extracted. The first sub-deformation field statistical shape model in this portion is associated with the target mobile breathing state, and the last sub-deformation field statistical shape model is associated with the target reference breathing state. That is, the partial model sequence is {M_00-10→M_10-20→M_20-30→M_30-40}. The sub-deformation field statistical shape models included in this partial model sequence can constitute the target sub-deformation field statistical shape model. The optimizable weights in M_00-10, M_10-20, M_20-30, and M_30-40 were adjusted multiple times. The value of .

[0180] The constructed deformation field statistical shape model belongs to the atlas space, while the non-real-time preoperative image and the real-time intraoperative image belong to the individual space where the target object is located. There are two methods for registration. In the first method, the operations related to steps S102 and S103 are performed in the individual space. In the second method, the operations related to steps S102 and S103 are performed in both the atlas space and the individual space.

[0181] Specifically, the first method can include, for example: Figure 7 The steps shown are as follows:

[0182] Before the operation, a statistical shape model of the deformation field can be obtained. Based on the deformation field between the preoperative image and the target template image, the statistical shape model of the deformation field can be transformed from the atlas space to the individual space.

[0183] During the operation, intraoperative images of the first respiratory cycle can be acquired. Based on the statistical shape model of the deformation field in the individual space, the preoperative images and the intraoperative images of the first respiratory cycle are registered, and the correlation between registration-related information and respiratory status is constructed.

[0184] Intraoperative images from non-first respiratory cycles are acquired, and the respiratory status of these intraoperative images is matched with the respiratory status of the first respiratory cycle to obtain the corresponding registration information.

[0185] Determine whether the target values ​​of the optimizable weights in the registration-related information need to be fine-tuned to register preoperative images with intraoperative images from non-first respiratory cycles;

[0186] The registered images are displayed on the navigation interface.

[0187] Specifically, the second approach can include, for example: Figure 8 The steps shown are as follows:

[0188] Before the operation, a statistical shape model of the deformation field is obtained, and the preoperative image is converted into the atlas space based on the deformation field between the preoperative image and the target template image.

[0189] During the operation, intraoperative images of the first respiratory cycle can be acquired and converted to atlas space. Within the atlas space, based on the deformation field statistical shape model, the preoperative images and the intraoperative images of the first respiratory cycle are registered, and the correlation between registration-related information and respiratory status is constructed.

[0190] Intraoperative images from non-first respiratory cycles are acquired, and the respiratory status of these intraoperative images is matched with the respiratory status of the first respiratory cycle to obtain the corresponding registration information.

[0191] Determine whether the target values ​​of the optimizable weights in the registration-related information need to be fine-tuned to register preoperative images with intraoperative images from non-first respiratory cycles;

[0192] The registered images are converted to individual space and displayed on the navigation interface.

[0193] The content displayed on the navigation interface can be as follows: Figure 9 As shown, the non-real-time preoperative image is transformed to register with the real-time intraoperative image, resulting in a registration result. This registration result includes the registration results of the transverse plane and the sagittal plane. The registration results of the transverse plane, the real-time intraoperative image of the transverse plane, and the registration results of the sagittal plane can be displayed in the navigation interface. In addition, the registration results can be fused with the deformed mesh image to display a 3D mesh image in the navigation interface.

[0194] In the first method, the operations related to steps S102 and S103 are performed within the individual space, which is faster. However, it involves transferring the statistical shape model of the deformation field from the spectral space to the individual space, resulting in some loss of accuracy. In the second method, registration is performed in the spectral space, which offers higher resolution and accuracy but is more time-consuming. After obtaining the relevant image through registration, it still needs to be transferred to the individual space, further increasing the time required. The first method is faster but slightly less accurate, while the second method is faster but more accurate; the choice can be made based on actual needs.

[0195] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0196] Based on the same inventive concept, this application also provides an image registration apparatus for implementing the image registration method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image registration apparatus embodiments provided below can be found in the limitations of the image registration method described above, and will not be repeated here.

[0197] In one embodiment, such as Figure 10 As shown, an image registration apparatus is provided, comprising:

[0198] The image acquisition module 1001 is used to acquire non-real-time preoperative images and real-time intraoperative images of the target object; the imaging field of view of the non-real-time preoperative image is wider than that of the imaging field of view of the real-time intraoperative image.

[0199] The registration processing module 1002 is used to adjust the value of the optimizable weight in the statistical shape model of the deformation field based on respiratory motion when the real-time intraoperative image belongs to the first respiratory cycle, to obtain the target deformation field for registering the non-real-time preoperative image and the real-time intraoperative image; to obtain registration-related information based on the target value of the optimizable weight used to obtain the target deformation field; and to obtain the correlation between the registration-related information and the respiratory state corresponding to the real-time intraoperative image.

[0200] The registration processing module 1002 is further configured to, when the real-time intraoperative image belongs to a non-first respiratory cycle, obtain registration-related information of the respiratory state associated with the real-time intraoperative image based on the correlation relationship, and register the non-real-time preoperative image with the real-time intraoperative image according to the target value of the optimizable weight in the registration-related information and the deformation field statistical shape model.

[0201] In one embodiment, the registration-related information further includes a deformation image, which is obtained by applying the target deformation field to the non-real-time preoperative image;

[0202] The registration processing module 1002 is further configured to: acquire the deformation image in the registration-related information; acquire the image portion in the deformation image that is within the same imaging field of view as the real-time intraoperative image; when the content similarity between the image portion and the real-time intraoperative image does not reach a threshold, adjust the value of the optimizable weight in the deformation field statistical shape model using the target value of the optimizable weight in the registration-related information as the initial adjustment value to obtain the target deformation field for registering the non-real-time preoperative image and the real-time intraoperative image.

[0203] In one embodiment, the registration processing module 1002 is further configured to: when the similarity between the image portion and the content of the real-time intraoperative image reaches a threshold, use the deformed image as the registration result between the non-real-time preoperative image and the real-time intraoperative image.

[0204] In one embodiment, the registration processing module 1002 is further configured to: acquire a plurality of the association relationships; determine, among the respiratory states corresponding to the plurality of association relationships, whether there exists a respiratory state consistent with the respiratory state corresponding to the real-time intraoperative image; if there is, take the respiratory state consistent with the respiratory state corresponding to the real-time intraoperative image as the target respiratory state; if there is not, take the respiratory state adjacent to the respiratory state corresponding to the real-time intraoperative image as the target respiratory state; take the association relationship corresponding to the target respiratory state as the target association relationship; and take the registration-related information corresponding to the target association relationship as the registration-related information associated with the respiratory state corresponding to the real-time intraoperative image.

[0205] In one embodiment, the deformation field statistical shape model based on respiratory motion includes several sub-deformation field statistical shape models. Each sub-deformation field statistical shape model is associated with a reference respiratory state and a moving respiratory state. The moving respiratory states associated with each sub-deformation field statistical shape model are the same, while the reference respiratory states associated with each sub-deformation field statistical shape model are different. The non-real-time preoperative image is collected when the target object is in the moving respiratory state.

[0206] The registration processing module 1002 is further configured to: when the real-time intraoperative image belongs to the first respiratory cycle, determine the target sub-deformation field statistical shape model among several sub-deformation field statistical shape models; match the reference respiratory state associated with the target sub-deformation field statistical shape model with the respiratory state corresponding to the real-time intraoperative image; and adjust the value of the optimizable weight in the target sub-deformation field statistical shape model.

[0207] In one embodiment, the deformation field statistical shape model based on respiratory motion includes several sub-deformation field statistical shape models. Each sub-deformation field statistical shape model is associated with a reference respiratory state and a moving respiratory state. The several sub-deformation field statistical shape models are arranged in sequence, and the reference respiratory state of the previous sub-deformation field statistical shape model is the moving respiratory state of the next sub-deformation field statistical shape model.

[0208] The registration processing module 1002 is further configured to: obtain a target moving respiratory state based on the respiratory state corresponding to the non-real-time preoperative image; obtain a target reference respiratory state based on the respiratory state corresponding to the real-time intraoperative image; determine a target sub-deformation field statistical shape model among several sub-deformation field statistical shape models; the moving respiratory state associated with the target sub-deformation field statistical shape model is the target moving respiratory state, and the reference respiratory state associated with the target sub-deformation field statistical shape model is the target reference respiratory state; and adjust the values ​​of the optimizable weights in the target sub-deformation field statistical shape model.

[0209] In one embodiment, the registration processing module 1002 is further configured to: obtain a model sequence based on a plurality of sequentially arranged sub-deformation field statistical shape models; extract a portion of the model sequence; the mobile breathing state associated with the first sub-deformation field statistical shape model in the portion of the model sequence is the target mobile breathing state, and the reference breathing state associated with the last sub-deformation field statistical shape model in the portion of the model sequence is the target reference breathing state; and obtain a target sub-deformation field statistical shape model based on the portion of the model sequence.

[0210] Each module in the aforementioned image registration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in the intraoperative real-time guidance system, or stored in software within the memory of the intraoperative real-time guidance system, so that the processor can call and execute the corresponding operations of each module.

[0211] In one exemplary embodiment, an intraoperative real-time guidance system is provided, the internal structure of which can be shown in the diagram below. Figure 11As shown, the real-time intraoperative guidance system includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores the data involved in the aforementioned methods. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an image registration method.

[0212] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the intraoperative real-time guidance system to which the present application is applied. A specific intraoperative real-time guidance system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0213] In one embodiment, an intraoperative real-time guidance system is provided, the system including a processor for performing the following steps:

[0214] During the first respiratory cycle, the non-real-time preoperative image of the target subject and the real-time intraoperative image corresponding to each respiratory state in the first respiratory cycle are registered according to the above method, so as to provide real-time intraoperative guidance during the first respiratory cycle.

[0215] During non-first respiratory cycles, the non-real-time preoperative images of the target subject and the real-time intraoperative images corresponding to each respiratory state during non-first respiratory cycles are registered according to the above method in order to provide real-time intraoperative guidance during non-first respiratory cycles.

[0216] After the robotic-guided surgery begins, a specific respiratory cycle can be designated as the first respiratory cycle based on actual needs. Respiratory cycles following this first respiratory cycle are referred to as non-first respiratory cycles.

[0217] When acquiring real-time intraoperative images of any respiratory state during the first respiratory cycle, the optimizable weights can be adjusted multiple times. The value. Each adjustment optimizes the weights. The value of can be used to obtain a new deformation field; apply this new deformation field to the non-real-time preoperative image to obtain a deformation image; obtain the image portion of the deformation image within the same imaging field of view as the real-time intraoperative image; obtain the content similarity between this image portion and the real-time intraoperative image, which can be obtained through normalized cross-correlation, mutual information, gradient correlation, or mean squared error; if the content similarity does not reach the threshold, it indicates that the new deformation field does not meet the requirements, and the optimizable weights can be further adjusted. The value; if the content similarity reaches the threshold, it means that the new deformation field meets the requirements and can be used as the target deformation field; apply the new deformation field to the non-real-time preoperative image, transform the non-real-time preoperative image to obtain a deformation image, process the deformation image and the real-time intraoperative image, and the processing result can be used for real-time intraoperative guidance.

[0218] Furthermore, it can convert the preoperative tissue segmentation map of the target object into a mesh map; it can also invert the target deformation field and apply it to the mesh map to obtain a deformed mesh map; it can apply the target deformation field to the non-real-time preoperative map to obtain a deformed image; and it can process the deformed image, the deformed mesh map, and the real-time intraoperative map.

[0219] Additionally, when the content similarity reaches a threshold, the optimizable weights obtained from this adjustment can be used. The value is used as the target value; based on the optimizable weights... The target value is obtained, and registration-related information is acquired; the correlation between registration-related information and the respiratory status corresponding to the real-time intraoperative image is constructed.

[0220] Understandably, with the continuous acquisition of real-time intraoperative images, the correlation between different respiratory states and registration-related information can be established in the first respiratory cycle.

[0221] Let's take the second respiratory cycle as an example, which is not the first respiratory cycle:

[0222] If the registration-related information includes the target value of the optimizable weights, the deformed image, and the deformed mesh image, when a real-time intraoperative image of the second respiratory cycle is acquired, the respiratory state corresponding to that real-time intraoperative image can be determined. The registration-related information associated with that respiratory state can then be identified within the correlation. The deformed image within this registration-related information is then acquired, and the portion of the deformed image within the same imaging field of view as the real-time intraoperative image is determined. The content similarity between this portion and the real-time intraoperative image is then calculated. If the content similarity reaches a threshold, the deformed image, the deformed mesh image within the registration-related information, and the real-time intraoperative image can be directly processed. The processing result can be used for real-time intraoperative guidance, eliminating the need to transform the non-real-time preoperative image using a deformation field. If the content similarity does not reach the threshold, the optimizable weights can be fine-tuned. At this point, the target value of the optimizable weights in the registration-related information is used to adjust the initial value, and the value of the optimizable weights is searched. Each time an optimizable weight value is found, a new deformation field is obtained by combining it with the deformation field statistical shape model. This new deformation field is applied to the non-real-time preoperative image to obtain a deformation image. The image portion within the same imaging field of view as the real-time intraoperative image is obtained, and the content similarity between this image portion and the real-time intraoperative image is calculated. If the content similarity does not reach the threshold, it indicates that the new deformation field does not meet the requirements, and the search for optimizable weights can continue. The value of the content similarity reaches the threshold, indicating that the new deformation field meets the requirements and is used as the target deformation field. The target deformation field is applied to the non-real-time preoperative image to obtain the deformation image, and the target deformation field is applied to the mesh image to obtain the deformed mesh image. The deformation image, the deformed mesh image and the real-time intraoperative image are processed, and the processing results can be used for real-time intraoperative guidance.

[0223] In one embodiment, the processor is also configured to perform the following steps:

[0224] The respiratory state of the target object is acquired through a respiratory gating device. If the deformation field statistical shape model is constructed under a fixed mobile respiratory state, the target object is scanned when its respiratory state matches the fixed mobile respiratory state to obtain a non-real-time preoperative image of the target object. The non-real-time preoperative image corresponds to the fixed mobile respiratory state. If the deformation field statistical shape model is constructed under a non-fixed mobile respiratory state, the target object is scanned when its respiratory state is in any respiratory state to obtain a non-real-time preoperative image of the target object.

[0225] When the deformation field statistical shape model is the deformation field statistical shape model of the independent breathing state, the deformation field statistical shape model of the independent breathing state is constructed under the condition of fixed moving breathing state.

[0226] If the fixed respiratory state is 00, the respiratory state of the target object can be monitored through a respiratory gating device during the preoperative stage. If the target object is detected to be in respiratory state 00, the target object is scanned. The scanned image is a non-real-time preoperative image, and the respiratory state corresponding to this non-real-time preoperative image is 00.

[0227] When the deformation field statistical shape model is a deformation field statistical shape model for the full breathing state or a deformation field statistical shape model for the continuous breathing state, the deformation field statistical shape model for the full breathing state or the deformation field statistical shape model for the continuous breathing state is constructed under the condition of non-fixed moving breathing state.

[0228] In the preoperative stage, the respiratory status of the target subject can be monitored through a respiratory gating device; when the target subject is detected to be in any respiratory state, the target subject can be scanned, and the scanned image is a non-real-time preoperative image.

[0229] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0230] In one embodiment, a computer program product is provided having a computer program stored thereon, the computer program being executed by a processor of the steps described in the various method embodiments above.

[0231] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0232] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0233] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0234] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image registration method, characterized in that, The method includes: Acquire non-real-time preoperative images and real-time intraoperative images of the target object; the imaging field of view of the non-real-time preoperative image is wider than that of the imaging field of view of the real-time intraoperative image. When the real-time intraoperative image belongs to the first respiratory cycle, the value of the optimizable weight in the deformation field statistical shape model based on respiratory motion is adjusted to obtain the target deformation field for registration of the non-real-time preoperative image and the real-time intraoperative image. Based on the target value of the optimizable weight used to obtain the target deformation field, the registration-related information is obtained, and the correlation between the registration-related information and the respiratory state corresponding to the real-time intraoperative image is obtained. When the real-time intraoperative image belongs to a non-first respiratory cycle, based on the correlation, the registration-related information of the respiratory state corresponding to the real-time intraoperative image is obtained. According to the target value of the optimizable weight in the registration-related information and the deformation field statistical shape model, the non-real-time preoperative image and the real-time intraoperative image are registered. The deformation field statistical shape model based on respiratory motion includes several sub-deformation field statistical shape models, each of which is associated with a reference respiratory state and a moving respiratory state. In a sequence of several sub-deformation field statistical shape models, where the reference respiratory state of the preceding sub-deformation field statistical shape model serves as the moving respiratory state of the following sub-deformation field statistical shape model, when the real-time intraoperative image belongs to the first respiratory cycle, the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion are adjusted, including: Based on the respiratory status corresponding to the non-real-time preoperative image, the target movement respiratory status is obtained; Based on the respiratory status corresponding to the real-time intraoperative image, the target reference respiratory status is obtained; Among several sub-deformation field statistical shape models, a target sub-deformation field statistical shape model is determined; the mobile breathing state associated with the target sub-deformation field statistical shape model is the target mobile breathing state, and the reference breathing state associated with the target sub-deformation field statistical shape model is the target reference breathing state. Adjust the values ​​of the optimizable weights in the statistical shape model of the target sub-deformation field.

2. The method according to claim 1, characterized in that, The registration-related information also includes a deformation image, which is obtained by applying the target deformation field to the non-real-time preoperative image. Based on the target values ​​of the optimizable weights in the registration-related information and the deformation field statistical shape model, the non-real-time preoperative image and the real-time intraoperative image are registered, including: Obtain the deformation image from the registration-related information; Obtain the portion of the deformed image that shares the same imaging field of view as the real-time intraoperative image; When the similarity between the image portion and the content of the real-time intraoperative image does not reach a threshold, the target value of the optimizable weight in the registration-related information is used as the initial value for adjustment, and the value of the optimizable weight in the deformation field statistical shape model is adjusted to obtain the target deformation field for registration of the non-real-time preoperative image and the real-time intraoperative image.

3. The method according to claim 2, characterized in that, The method further includes: When the similarity between the image portion and the content of the real-time intraoperative image reaches a threshold, the deformed image is used as the registration result between the non-real-time preoperative image and the real-time intraoperative image.

4. The method according to claim 1, characterized in that, Based on the aforementioned correlation, registration-related information concerning the respiratory status associated with the real-time intraoperative image is obtained, including: Obtain several of the aforementioned relationships; Among the respiratory states corresponding to each of the aforementioned relationships, determine whether there is a respiratory state that is consistent with the respiratory state corresponding to the real-time intraoperative image. If it exists, the respiratory state that is consistent with the respiratory state corresponding to the real-time intraoperative image will be taken as the target respiratory state. If it does not exist, the respiratory state adjacent to the respiratory state corresponding to the real-time intraoperative image will be taken as the target respiratory state. The association relationship corresponding to the target respiratory state is taken as the target association relationship; the registration-related information corresponding to the target association relationship is taken as the registration-related information associated with the respiratory state corresponding to the real-time intraoperative image.

5. The method according to claim 1, characterized in that, When the mobile respiratory state associated with each sub-deformation field statistical shape model is the same, but the reference respiratory state associated with each sub-deformation field statistical shape model is different, the non-real-time preoperative image is collected when the target object is in the mobile respiratory state. When the real-time intraoperative image belongs to the first respiratory cycle, the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion are adjusted, including: Among several sub-deformation field statistical shape models, a target sub-deformation field statistical shape model is determined; the reference respiratory state associated with the target sub-deformation field statistical shape model is matched with the respiratory state corresponding to the real-time intraoperative image; Adjust the values ​​of the optimizable weights in the statistical shape model of the target sub-deformation field.

6. The method according to claim 1, characterized in that, Among several statistical shape models of sub-deformation fields, the statistical shape model of the target sub-deformation field is determined, including: Based on a series of sequentially arranged statistical shape models of sub-deformation fields, a model sequence is obtained; In the model sequence, a portion of the model sequence is extracted; the mobile breathing state associated with the first sub-deformation field statistical shape model in the portion of the model sequence is the target mobile breathing state, and the reference breathing state associated with the last sub-deformation field statistical shape model in the portion of the model sequence is the target reference breathing state. Based on the aforementioned partial model sequence, the statistical shape model of the target sub-deformation field is obtained.

7. An image registration device, characterized in that, The device includes: The image acquisition module is used to acquire non-real-time preoperative images and real-time intraoperative images of the target object; the imaging field of view of the non-real-time preoperative image is wider than that of the imaging field of view of the real-time intraoperative image. The registration processing module is used to adjust the value of the optimizable weight in the statistical shape model of the deformation field based on respiratory motion when the real-time intraoperative image belongs to the first respiratory cycle, to obtain the target deformation field for registering the non-real-time preoperative image and the real-time intraoperative image. Based on the target value of the optimizable weight used to obtain the target deformation field, the module obtains registration-related information and obtains the correlation between the registration-related information and the respiratory state corresponding to the real-time intraoperative image. The registration processing module is further configured to, when the real-time intraoperative image belongs to a non-first respiratory cycle, obtain registration-related information of the respiratory state associated with the real-time intraoperative image based on the correlation relationship, and register the non-real-time preoperative image with the real-time intraoperative image according to the target value of the optimizable weight in the registration-related information and the deformation field statistical shape model; The deformation field statistical shape model based on respiratory motion includes several sub-deformation field statistical shape models, each of which is associated with a reference respiratory state and a moving respiratory state. In a sequence of several sub-deformation field statistical shape models, where the reference respiratory state of the preceding sub-deformation field statistical shape model serves as the moving respiratory state of the following sub-deformation field statistical shape model, when the real-time intraoperative image belongs to the first respiratory cycle, the values ​​of the optimizable weights in the deformation field statistical shape model based on respiratory motion are adjusted, including: Based on the respiratory status corresponding to the non-real-time preoperative image, the target movement respiratory status is obtained; Based on the respiratory status corresponding to the real-time intraoperative image, the target reference respiratory status is obtained; Among several sub-deformation field statistical shape models, a target sub-deformation field statistical shape model is determined; the mobile breathing state associated with the target sub-deformation field statistical shape model is the target mobile breathing state, and the reference breathing state associated with the target sub-deformation field statistical shape model is the target reference breathing state. Adjust the values ​​of the optimizable weights in the statistical shape model of the target sub-deformation field.

8. An intraoperative real-time guidance system, the system comprising a processor, characterized in that, The processor is used to perform the following steps: During the first respiratory cycle, the non-real-time preoperative image of the target object and the real-time intraoperative image corresponding to each respiratory state in the first respiratory cycle are registered according to the method of any one of claims 1 to 6, so as to provide real-time intraoperative guidance in the first respiratory cycle. During non-first respiratory cycles, the non-real-time preoperative image of the target subject and the real-time intraoperative image corresponding to each respiratory state during non-first respiratory cycles are registered according to the method of any one of claims 1 to 6, so as to provide real-time intraoperative guidance during non-first respiratory cycles.

9. The system according to claim 8, characterized in that, The processor is also configured to perform the following steps: The respiratory status of the target object is obtained through a respiratory gating device; If the deformation field statistical shape model is constructed under a fixed mobile breathing state, then when the breathing state of the target object matches the fixed mobile breathing state, the target object is scanned to obtain a non-real-time preoperative image of the target object; the non-real-time preoperative image corresponds to the fixed mobile breathing state. If the deformation field statistical shape model is constructed under non-fixed mobile breathing state, then when the target object is in any breathing state, the target object is scanned to obtain a non-real-time preoperative image of the target object.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images