A method and apparatus for image registration

Abstract

The specification discloses a registration method and a registration device of images, the registration method of images comprising: acquiring a first type image and a second type image of a patient at a body tissue, the first type image and the second type image being different types of images, a device based on which the first type image is acquired, and a device based on which the second type image is acquired, being different types of devices; inputting the first type image into a pre-trained image conversion model to determine a simulated second type image corresponding to the first type image; inputting the second type image and the simulated second type image into a pre-trained image registration model to determine a global conversion parameter between pixels contained in the first type image and pixels contained in the second type image; and performing registration on the first type image and the second type image through the global conversion parameter.

Description

Technical Field

[0001] This specification relates to the field of medical image processing technology, and in particular to an image registration method and registration device. Background Technology

[0002] As an important branch of medical image processing, medical image registration technology has been widely used in the diagnosis and treatment of patients. For example, before surgery, because the display effects of magnetic resonance (MR) images and computed tomography (CT) images of the affected area differ for different tissues in the body, it is usually necessary to register the MR images and CT images. The fused images after registration can provide doctors with both anatomical information and functional imaging results at the same location, playing an important role in doctors' clinical diagnosis and operation.

[0003] However, current registration methods typically employ feature-based or edge-point detection approaches. These methods suffer from poor registration accuracy, making it impossible to precisely fuse every corresponding location between MR and CT images. Furthermore, the registration process is time-consuming, hindering its application in surgical procedures where real-time performance and accuracy are critical. Additionally, some methods require auxiliary tools for registration, the placement of which is easily affected by various environmental factors during surgery, compromising registration accuracy.

[0004] Therefore, how to improve image registration accuracy while increasing image registration efficiency is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides an image registration method and registration apparatus to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides an image registration method, including:

[0008] Acquire a first type of image and a second type of image of the patient in the body tissues. The first type of image and the second type of image are images of different types. The device on which the first type of image is acquired is different from the device on which the second type of image is acquired.

[0009] The first type of image is input into a pre-trained image conversion model to determine the simulated second type of image corresponding to the first type of image;

[0010] The second type image and the simulated second type image are input into a pre-trained image registration model to determine the global transformation parameters between the pixels contained in the first type image and the pixels contained in the second type image.

[0011] The first type of image and the second type of image are registered using the global transformation parameters.

[0012] Optionally, the first type image and the second type image are registered using the global transformation parameters, specifically including:

[0013] The first type of image is adjusted using the global transformation parameters, and the adjusted first type of image is then registered with the second type of image.

[0014] Optionally, the global conversion parameters include at least one of the following: the grayscale adjustment amount corresponding to each pixel in the first type of image, and the translation amount of each pixel in the first type of image relative to each pixel in the second type of image;

[0015] The first type of image is adjusted using the global transformation parameters, and the adjusted first type of image is then registered with the second type of image, specifically including:

[0016] The position and grayscale of each pixel in the first type of image are adjusted using the global transformation parameters, and the adjusted first type of image is then registered with the second type of image.

[0017] Optionally, the second type image and the simulated second type image are input into a pre-trained image registration model to determine global transformation parameters between pixels in the first type image and pixels in the second type image, specifically including:

[0018] The second type image and the simulated second type image are input into a pre-trained image registration model to determine the feature point transformation parameters between the feature points in the first type image and the feature points in the second type image based on the differences between the feature points in the second type image and the feature points in the simulated second type image, and the feature point transformation parameters are used as global transformation parameters.

[0019] Optionally, training the image conversion model specifically includes:

[0020] Acquire a first sample image, wherein the first sample image includes a first type of historical image of the patient at the body tissue and an actual second type of historical image corresponding to the first type of historical image;

[0021] The first type of historical image is input into the image conversion model to obtain a simulated second type of historical image corresponding to the first type of historical image;

[0022] The image conversion model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical image and the actual second-type historical image.

[0023] Optionally, the image conversion model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical image and the actual second-type historical image, specifically including:

[0024] The deviation between each pixel in the simulated second-type historical image and each pixel in the actual second-type historical image is used as the pixel deviation of the image registration model, and the recognition result of body tissue identification in the simulated second-type historical image is determined.

[0025] The overall deviation of the image registration model is determined based on the pixel deviation of the image conversion model and the deviation between the recognition result and the actual recognition result corresponding to the actual second type of historical image.

[0026] The image conversion model is trained with the goal of minimizing the overall bias of the model.

[0027] Optionally, training the image registration model specifically includes:

[0028] Acquire a second sample image, wherein the second sample image includes a first type of historical sample image of the patient at the body tissue and an actual second type of historical sample image corresponding to the first type of historical sample image;

[0029] The first type of historical sample image is input into a pre-trained image conversion model to determine the simulated second type of historical sample image corresponding to the first type of historical sample image;

[0030] The actual second-type historical sample image and the simulated second-type historical sample image are input into the image registration model to determine the global transformation parameters to be optimized between the pixels contained in the first-type historical sample image and the pixels contained in the actual second-type historical sample image.

[0031] The first type of historical sample image and the actual second type of historical sample image are registered using the global transformation parameters to be optimized.

[0032] The image registration model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical sample image corresponding to the registered first-type historical sample image and the actual second-type historical sample image.

[0033] Optionally, the image registration model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical sample image corresponding to the registered first-type historical sample image and the actual second-type historical sample image. Specifically, this includes:

[0034] The deviation between each pixel in the simulated second-type historical sample image corresponding to the registered first-type historical sample image and each pixel in the actual second-type historical sample image is taken as the pixel deviation of the image registration model, and the recognition result of body tissue identification of the simulated second-type historical sample image is taken as the second recognition result.

[0035] The overall deviation of the image registration model is determined based on the pixel deviation of the image registration model and the deviation between the second recognition result and the actual recognition result corresponding to the actual second type of historical sample image.

[0036] The image registration model is trained with the goal of minimizing the overall bias of the model.

[0037] Optionally, the method further includes:

[0038] The registered second-type image is fused with the registered first-type image to obtain a fused image containing features of both the second-type image and the first-type image. Medical tasks are then performed based on the fused image.

[0039] Optionally, the first type of image includes: magnetic resonance (MR) images, and the second type of image includes: computed tomography (CT) images.

[0040] This specification provides an image registration device, comprising:

[0041] The acquisition module acquires a first type of image and a second type of image of the patient in the body tissue. The first type of image and the second type of image are images of different types. The device on which the first type of image is acquired is different from the device on which the second type of image is acquired.

[0042] The conversion module inputs the first type of image into a pre-trained image conversion model to determine the simulated second type of image corresponding to the first type of image;

[0043] The determination module inputs the second type image and the simulated second type image into a pre-trained image registration model to determine the global transformation parameters for all pixels in the first type image or the second type image;

[0044] The registration module registers the first type image and the second type image using the global conversion parameters.

[0045] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image registration method described above.

[0046] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0047] In the image registration method provided in this specification, a first type image is converted through an image conversion model to obtain a simulated second type image corresponding to the first type image. Then, the simulated second type image and the actual second type image are input into the image registration model to obtain global conversion parameters between the first type image and the second type image. Finally, the first type image and the second type image are registered through these global conversion parameters.

[0048] As can be seen from the above method, this specification first converts the first type image into a corresponding simulated second type image. By comparing the deviation between each pixel in the simulated second type image and the actual second type image, the global transformation parameters for the transformation between each pixel in the simulated second type image and the actual second type image are obtained. Then, the transformation parameters between each pixel in the first type image and the second type image are determined, and image registration is performed accordingly. This improves the image registration accuracy and provides accurate image references for medical tasks such as patient diagnosis and surgical procedures. Attached Figure Description

[0049] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0050] Figure 1 This is a schematic diagram of an image registration method provided in this specification;

[0051] Figure 2 This diagram illustrates a training method for an image conversion model provided in this specification.

[0052] Figure 3 This is a schematic diagram of an image registration device provided in this specification;

[0053] Figure 4 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0055] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0056] Figure 1 This is a flowchart illustrating an image registration method provided in this specification, including the following steps:

[0057] S101: Acquire a first type image and a second type image of the patient at the site of body tissue, wherein the first type image and the second type image are images of different types, and the device on which the first type image is acquired is a different type of device from the device on which the second type image is acquired.

[0058] In medical tasks such as preoperative diagnosis and surgical procedures, surgical personnel typically need to obtain CT images of body tissues using computed tomography (CT) or MRI images of body tissues using magnetic resonance imaging (MRI) to obtain information about the body tissues.

[0059] Because different tissues within a patient's body have varying structures and compositions, and because MR and CT images operate on different imaging principles, their visualization effects on different tissues differ significantly. For example, CT images are more effective at displaying tissues with density differences (such as the liver, pancreas, and adrenal glands), while MR images are significantly better at displaying soft tissues like bones, joints, blood vessels, rectum, muscles, and the bladder, as well as the nervous system. In complex diagnostic or surgical procedures, a single CT or MR image may not provide sufficient information about the disease. Therefore, it is necessary to combine the visualization characteristics of body tissues from both CT and MR images to obtain more comprehensive information, thereby ensuring the accuracy and safety of medical procedures.

[0060] However, some medical tasks require high precision in image information, which necessitates strict registration of CT and MR images to ensure that the same areas in the CT and MR images are precisely aligned. This allows for accurate fusion of the CT or MR images, providing operators with accurate reference information to guarantee surgical precision.

[0061] Based on this, this specification provides an image registration method, wherein the server can acquire CT images of body tissues using a computed tomography scanner provided in the surgical environment and MR images of body tissues using an magnetic resonance imaging (MRI) scanner.

[0062] In this specification, the first type of image at the body tissue site can be an MR image at the body tissue site, and the second type of image is a CT image at the body tissue site. Of course, the first type of image at the body tissue site can also be a CT image at the body tissue site, and correspondingly, the second type of image can be an MR image at the body tissue site. For ease of description, the following will only use the MR image at the body tissue site as the first type of image and the CT image at the body tissue site as the second type of image as an example to explain the image registration method provided in this specification.

[0063] Furthermore, the execution subject of the image registration method described in this specification can refer to a designated device such as a server set up in a medical environment. For ease of description, the following description will only use the server as the execution subject to illustrate one image registration method provided in this specification.

[0064] S102: Input the first type of image into the pre-trained image conversion model to determine the simulated second type of image corresponding to the first type of image.

[0065] When the MR image of the body tissue is used as the first type of image and the CT image of the body tissue is used as the second type of image, after the server obtains the MR image and CT image of the patient's body tissue, the server can input the above MR image and CT image into a pre-trained image conversion model. The convolutional layer, pooling layer, normalization layer and other feature processing layers of the image conversion model are used to perform feature processing, and the gray level of each pixel in the MR image is adjusted accordingly to convert it into a simulated CT image.

[0066] Before using the above image conversion model on the server, the image conversion model needs to be trained first. In this manual, the execution subject used for training each model can refer to the server or a specified device such as a desktop computer or laptop computer. For ease of description, the following will only use the server as the execution subject for model training as an example to explain the training of each model.

[0067] Specifically, the service can obtain first sample images of different patients in their body tissues. These sample images contain first type historical images and second type historical images. When the MR image of the body tissue is the first type image and the CT image of the body tissue is the second type image, the first type historical image in the first sample image is the first historical MR image, and the second type historical image is the first historical actual CT image.

[0068] It should be noted that the first historical MR image and the first historical actual CT image are pre-registered. There are various methods for registering the first historical MR image and the first historical actual CT image. For example, the first historical MR image or the first historical CT image can be manually adjusted for manual registration. Another method is to use a positioning device such as a tracer to register the first historical MR image and the first historical actual CT image. To enhance the robustness and generalization ability of the image conversion model, the sample images should, as far as possible, include images corresponding to different body tissues of the human body.

[0069] Furthermore, the server can input the first historical MR image into the image conversion model to obtain the first historical simulated CT image corresponding to the first historical MR image. Then, the server can determine the pixel deviation of the image conversion model based on the deviation of each pixel in the first historical simulated CT image from each pixel in the first historical actual CT image, such as the pixel grayscale value. Based on this, the server can determine the loss value of the grayscale loss function corresponding to the image conversion model. For example, the server can calculate the loss value of the grayscale loss function using the minimum absolute value error between each pixel in the first historical simulated CT image and each pixel in the first historical actual CT image. The formula for the grayscale loss function corresponding to the image conversion model can then be:

[0070]

[0071] in, This refers to the pixel deviation of the image conversion model. Let y be the first historical simulated CT image, y be the first historical actual CT image, and i be the corresponding pixel in the two images. Then, the loss value L1 of the grayscale loss function corresponding to the image conversion model can represent the deviation between each pixel in the first historical simulated CT image and the corresponding pixel in the first historical actual CT image.

[0072] Alternatively, the server can use the loss value of the grayscale loss function calculated from the least square error between each pixel in the first historical simulated CT image and each pixel in the first historical actual CT image as the pixel deviation of the image conversion model. The formula for calculating the pixel deviation of the image conversion model can also be:

[0073]

[0074] in, This represents the pixel deviation of the image conversion model.

[0075] Of course, other loss functions can also be used to calculate the pixel deviation of the image conversion model, and this manual does not make specific restrictions on this.

[0076] After the server determines the pixel deviation of the image conversion model, it can determine the deviation between the above recognition results based on the first recognition result of body tissue recognition of the first historical simulated CT image through the image conversion model and the actual recognition result corresponding to the first historical actual CT image. Based on the deviation between the above recognition results and the pixel deviation of the image conversion model, the comprehensive deviation of the image conversion model is determined.

[0077] The recognition result of the image conversion model for body tissue identification in the first historical simulated CT image and the first historical actual CT image can be a segmentation map (such as the image region corresponding to human tissues such as bones and blood vessels identified by the segmentation network in the image conversion model) after segmenting the first historical simulated CT image and the first historical actual CT image. The deviation between the above recognition results can be the deviation between the image region corresponding to the body tissue identified in the first historical simulated CT image and the image region corresponding to the body tissue in the first historical actual CT image.

[0078] The formula for calculating the overall bias of the image conversion model can be:

[0079] Lseg = L1(Iseg1,Iseg2)

[0080] or

[0081] Lseg = L2(Iseg1,Iseg2)

[0082] Wherein, Iseg1 is the recognition result of the image conversion model for body tissue identification in the first historical simulated CT image, Iseg2 is the recognition result of the image conversion model for body tissue identification in the first historical actual CT image, L1 is the loss value of the gray-scale loss function corresponding to the image conversion model determined by calculating the minimum absolute value error between each pixel in the first historical simulated CT image and each pixel in the first historical actual CT image, L2 is the loss value of the gray-scale loss function corresponding to the image conversion model determined by calculating the minimum squared error between each pixel in the first historical simulated CT image and each pixel in the first historical actual CT image, and Lseg is the comprehensive deviation of the image conversion model.

[0083] The server can train the image conversion model with the optimization objective of minimizing its overall bias until the model meets the training objective. The trained model parameters are then determined and saved for deployment within the image conversion model. The training objective can be: the overall bias of the image conversion model converges to a preset threshold range, or a preset number of training iterations are reached, to ensure that the image conversion model can accurately convert MR images into corresponding simulated CT images. The preset threshold range and the preset number of training iterations can be set according to actual conditions and are not specifically limited in this specification. For ease of understanding, this specification provides a schematic diagram of the image conversion model training method, as shown below. Figure 2 As shown.

[0084] Figure 2 This is a schematic diagram illustrating a training method for an image conversion model provided in this specification.

[0085] The server inputs the patient's MR image at the body tissue into the image conversion model, obtains the simulated CT image corresponding to the MR image through the image conversion model, and then trains the image conversion model based on the deviation between the simulated CT image corresponding to the MR image and the actual CT image corresponding to the MR image.

[0086] During the image registration process, the server can load the model parameters of the image conversion model obtained after training the image conversion model when it receives the registration instruction, and deploy them into the image conversion model. Alternatively, the server can deploy the CT and MR images of the body tissue into the image conversion model after acquiring them. Of course, the server can also load the model parameters of the image conversion model and deploy them into the image conversion model immediately after startup. This manual does not make specific limitations on this.

[0087] S103: Input the second type image and the simulated second type image into a pre-trained image registration model to determine the global transformation parameters between the pixels contained in the first type image and the pixels contained in the second type image.

[0088] When the MR image of the body tissue is used as the first type of image and the CT image of the body tissue is used as the second type of image, after the server obtains the simulated CT image corresponding to the MR image of the patient's body tissue, the server can input the simulated CT image and the actual CT image into a pre-trained image registration model. The image registration model will determine the offset of each pixel in the MR image and the grayscale adjustment amount between the pixels in the CT image and the pixels in the CT image based on the deviation between each pixel in the simulated CT image and the actual CT image. The server can use this to construct a deformation field that adjusts the position and grayscale of each pixel (such as a matrix used to represent the positional relationship and grayscale change between the pixels in the MR image and the pixels in the CT image) and use it as a global transformation parameter between the pixels in the MR image and the pixels in the CT image.

[0089] It should be noted that the grayscale adjustment amount determined here is intended to enhance the display effect of MR image features and CT image features in the registered image, and will not make the grayscale of the MR image and CT image more similar.

[0090] Before the server uses the aforementioned image registration model, it needs to train the image conversion model. Specifically, the server needs to acquire a second sample image for training the image registration model. This second sample image contains first-type historical sample images and second-type historical sample images of different patients at body tissues. When the MR image at the body tissue is used as the first-type image and the CT image at the body tissue is used as the second-type image, the corresponding first-type historical sample image becomes the second-historical MR image, and the second-type historical sample image becomes the second-historical actual CT image. It should be noted that, in this specification, the second sample image used to train the image registration model and the first sample image used to train the image conversion model can be the same sample image. That is, the first-type historical image and the first-type historical sample image contained in the second sample image and the first sample image can be the same sample image, and the second-type historical image and the second-type historical sample image can be the same sample image.

[0091] The server can input the aforementioned second historical MR image into the image registration model to obtain the second historical simulated CT image corresponding to the second historical MR image. Then, the server can input the second historical simulated CT image and the second historical actual CT image into the image registration model to determine the global transformation parameters to be optimized between the pixels contained in the second historical MR image and the pixels contained in the second historical actual CT image. The server can then register the second historical MR image and the second historical CT image using the global transformation parameters to be optimized.

[0092] During the training of the image registration model, the server can use the deviation between each pixel in the simulated CT image obtained from the registered second historical MR image through the image conversion model and each pixel in the second historical actual CT image as the pixel deviation of the image registration model, and then determine the loss value of the gray-level loss function corresponding to the image registration model. For example, the server can determine the loss value of the gray-level loss function corresponding to the image registration model by calculating the least square error between each pixel in the simulated CT image obtained from the registered second historical MR image through the image conversion model and each pixel in the second historical actual CT image, or by calculating the minimum absolute error. In this specification, the loss value of the gray-level loss function corresponding to the image registration model can be calculated using the same formula as the image conversion model for calculating the least square error or the minimum absolute error. This specification will not elaborate further on this.

[0093] After the server determines the pixel deviation of the image registration model, it can determine the deviation between the image registration model and the above recognition results based on the second recognition result of the simulated CT image obtained by the image conversion model from the registered second historical MR image, and the recognition result corresponding to the second historical actual CT image. Based on the deviation between the image registration model and the above recognition results, as well as the pixel deviation of the image registration model, the overall deviation of the image registration model is determined.

[0094] The identification results of the image registration model for body tissue recognition of the registered second historical MR image through the simulated CT image obtained by the image conversion model, and the identification results of the second historical actual CT image for body tissue recognition can be the segmentation map obtained by the segmentation network in the image registration model for the registered second historical MR image through the simulated CT image obtained by the image conversion model and the second historical actual CT image. In this specification, the server can use the same formula as the above formula for calculating the comprehensive deviation of the image registration model when calculating the comprehensive deviation of the image conversion model. This specification will not elaborate further on this.

[0095] The server can train the image registration model with the optimization objective of minimizing its overall bias until the model meets the training objective. The trained model parameters are then determined and saved for deployment within the image registration model. The training objective can be either for the overall bias of the image registration model to converge to a preset threshold range, or for reaching a preset number of training iterations, to ensure accurate registration of MR and CT images. The preset threshold range and the preset number of training iterations can be set according to actual conditions and are not specifically limited in this specification.

[0096] During the image registration process, the server can load the model parameters of the image registration model obtained after training the image registration model when it receives the registration instruction, and deploy them into the image registration model. Alternatively, the server can deploy the model parameters into the image registration model after obtaining the simulated CT image corresponding to the MR image through the aforementioned image conversion model. Of course, the server can also load the model parameters of the image registration model and deploy them into the image registration model immediately after startup. This specification does not specifically limit this.

[0097] Of course, in this specification, the server can also, after inputting CT images and simulated CT images into the image registration model, extract the contours and edge feature points of body tissues in the images (such as the contours and edge feature points of the spine) through the image registration model, and then, based on the differences between the feature points corresponding to the body tissues in the CT images and the feature points corresponding to the body tissues in the simulated CT images, determine the feature point conversion parameters between each feature point in the MR image and each feature point in the CT image, and use the feature point conversion parameters as global conversion parameters.

[0098] S104: Register the first type image and the second type image using the global conversion parameters.

[0099] When an MR image of a body tissue is used as the first type of image and a CT image of the same body tissue is used as the second type of image, after the server obtains the global conversion parameters, it can adjust the spatial position of each pixel in the MR image to strictly align it with the same part in the CT image, and adjust the grayscale value of each pixel in the MR image to improve its display effect, thereby completing the registration of the MR image and the CT image.

[0100] Of course, the server can also use this global conversion parameter to adjust the spatial position of each pixel in the CT image so that it is in the same spatial position as the corresponding pixel in the MR image, and adjust the grayscale value of each pixel in the CT image to improve its display effect, thereby completing the registration of the CT image and the MR image.

[0101] Furthermore, when the MR image of the body tissue is used as the second type of image and the CT image of the body tissue is used as the first type of image, the server can train another image conversion model. This model converts the patient's actual CT image at the body tissue into a simulated MR image. The simulated MR image and the actual MR image at the body tissue are then input into an image registration model to obtain global conversion parameters between the actual MR image and the actual CT image at the body tissue. This allows for the registration of the actual MR image and the actual CT image at the body tissue. The training method for this other image conversion model can be the same as the training method for the image conversion model described above, and will not be elaborated upon further in this specification.

[0102] After registering MR and CT images, the server can fuse the registered MR and CT images and send the fused image to medical personnel, providing relevant information for medical tasks. Alternatively, during robotic-guided surgery using surgical equipment, the registered MR and CT images can be sent separately to the surgical equipment for navigation.

[0103] As can be seen from the above method, the image registration method provided in this specification first converts the MR image into a corresponding simulated CT image. By comparing the deviation between each pixel in the simulated CT image and the actual CT image, the global conversion parameters for transforming each pixel in the simulated CT image and the actual CT image are obtained. Then, the conversion parameters between each pixel in the MR image and the CT image are determined. The position of each pixel in the MR image or CT image is adjusted by the conversion parameters, and the grayscale is adjusted accordingly to improve the display effect. This completes the registration of the CT image and the MR image, improves the image registration accuracy, and provides accurate image reference for medical tasks such as patient diagnosis and surgical operations.

[0104] The above describes one or more image registration methods for embodiments of this specification. Based on the same approach, this specification also provides corresponding image registration devices, such as... Figure 3 As shown.

[0105] Figure 3 A schematic diagram of an image registration device provided in this specification includes:

[0106] The acquisition module 301 acquires a first type image and a second type image of the patient in the body tissue. The first type image and the second type image are images of different types. The device on which the first type image is acquired is different from the device on which the second type image is acquired.

[0107] The conversion module 302 is used to input the first type image into a pre-trained image conversion model to determine the simulated second type image corresponding to the first type image;

[0108] The determination module 303 is used to input the second type image and the simulated second type image into a pre-trained image registration model to determine the global transformation parameters between the pixels contained in the first type image and the pixels contained in the second type image;

[0109] The registration module 304 is used to register the first type image and the second type image using the global conversion parameters.

[0110] Optionally, the registration module 304 is specifically used to adjust the first type image through the global conversion parameters, and register the adjusted first type image with the second type image.

[0111] Optionally, the global conversion parameters include at least one of the following: the grayscale adjustment amount corresponding to each pixel in the first type of image, and the translation amount of each pixel in the first type of image relative to each pixel in the second type of image;

[0112] The registration module 304 is specifically used to adjust the position and grayscale of each pixel in the first type image through the global conversion parameters, and to register the adjusted first type image with the second type image.

[0113] Optionally, the determining module 303 is specifically used to input the second type image and the simulated second type image into a pre-trained image registration model, so as to determine the feature point transformation parameters between the feature points in the first type image and the feature points in the second type image based on the differences between the feature points in the second type image and the feature points in the simulated second type image, and use the feature point transformation parameters as global transformation parameters.

[0114] Optionally, the device further includes:

[0115] Training module 305 is used to acquire a first sample image, wherein the first sample image includes a first type of historical image of the patient at the body tissue and an actual second type of historical image corresponding to the first type of historical image; inputting the first type of historical image into the image conversion model to obtain a simulated second type of historical image corresponding to the first type of historical image; and training the image conversion model with the optimization objective of minimizing the deviation between the simulated second type of historical image and the actual second type of historical image.

[0116] Optionally, the training module 305 is specifically used to: use the deviation between each pixel in the simulated second-type historical image and each pixel in the actual second-type historical image as the pixel deviation of the image registration model; and determine the recognition result of body tissue recognition in the simulated second-type historical image; determine the comprehensive deviation of the image registration model based on the pixel deviation of the image conversion model and the deviation between the recognition result and the actual recognition result corresponding to the actual second-type historical image; and train the image conversion model with minimizing the comprehensive deviation of the image conversion model as the optimization objective.

[0117] Optionally, the training module 305 is specifically configured to: acquire a second sample image, wherein the second sample image includes a first type of historical sample image of the patient at the body tissue and an actual second type of historical sample image corresponding to the first type of historical sample image; input the first type of historical sample image into a pre-trained image conversion model to determine a simulated second type of historical sample image corresponding to the first type of historical sample image; input the actual second type of historical sample image and the simulated second type of historical sample image into the image registration model to determine the global conversion parameters to be optimized between the pixels contained in the first type of historical sample image and the pixels contained in the actual second type of historical sample image; register the first type of historical sample image and the actual second type of historical sample image using the global conversion parameters to be optimized; and train the image registration model with the optimization objective of minimizing the deviation between the simulated second type of historical sample image corresponding to the registered first type of historical sample image and the actual second type of historical sample image.

[0118] Optionally, the training module 305 is specifically used to train the image registration model with the optimization objective of minimizing the deviation between the simulated second-type historical sample image corresponding to the registered first-type historical sample image and the actual second-type historical sample image. Specifically, this includes: using the deviation between each pixel in the simulated second-type historical sample image corresponding to the registered first-type historical sample image and each pixel in the actual second-type historical sample image as the pixel deviation of the image registration model; determining the recognition result of body tissue identification on the simulated second-type historical sample image as the second recognition result; determining the comprehensive deviation of the image registration model based on the pixel deviation of the image registration model and the deviation between the second recognition result and the actual recognition result corresponding to the actual second-type historical sample image; and training the image registration model with the optimization objective of minimizing the comprehensive deviation of the image registration model.

[0119] Optionally, the registration module 304 is further configured to fuse the registered second type image with the registered first type image to obtain a fused image containing the features of the second type image and the features of the first type image, and to perform a medical task based on the fused image.

[0120] Optionally, the first type of image includes: magnetic resonance (MR) images, and the second type of image includes: computed tomography (CT) images.

[0121] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This provides an image registration method.

[0122] This instruction manual also provides Figure 4 The one shown corresponds to Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 4 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The image registration method described above. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0123] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0124] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0125] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0126] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0127] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0133] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0134] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0137] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0138] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. An image registration method, characterized in that, include: First-type images and second-type images of the patient's body tissues are acquired from different types of devices, and the first-type images and the second-type images are images of different types. The first type of image is input into a pre-trained image conversion model to determine the simulated second type of image corresponding to the first type of image; The second type image and the simulated second type image are input into a pre-trained image registration model to determine the global transformation parameters between the pixels contained in the first type image and the pixels contained in the second type image. The first type image and the second type image are registered using the global transformation parameters; wherein, the first type image is adjusted using the global transformation parameters so that the corresponding pixels in the adjusted first type image and the second type image are located in the same spatial position, thereby improving the display effect of the first type image; and the adjusted first type image and the second type image are fused to obtain a fused image containing the features of the second type image and the features of the first type image.

2. The method as described in claim 1, characterized in that, The global conversion parameters include at least one of the following: the grayscale adjustment amount corresponding to each pixel in the first type of image, and the translation amount of each pixel in the first type of image relative to each pixel in the second type of image; The first type of image is adjusted using the global transformation parameters, and the adjusted first type of image is then registered with the second type of image, specifically including: The position and grayscale of each pixel in the first type of image are adjusted using the global transformation parameters, and the adjusted first type of image is then registered with the second type of image.

3. The method as described in claim 1, characterized in that, The second type image and the simulated second type image are input into a pre-trained image registration model to determine the global transformation parameters between the pixels contained in the first type image and the pixels contained in the second type image, specifically including: The second type image and the simulated second type image are input into a pre-trained image registration model to determine the feature point transformation parameters between the feature points in the first type image and the feature points in the second type image based on the differences between the feature points in the second type image and the feature points in the simulated second type image, and the feature point transformation parameters are used as global transformation parameters.

4. The method as described in claim 1, characterized in that, Training the image conversion model specifically includes: Acquire a first sample image, wherein the first sample image includes a first type of historical image of the patient at the body tissue and an actual second type of historical image corresponding to the first type of historical image; The first type of historical image is input into the image conversion model to obtain a simulated second type of historical image corresponding to the first type of historical image; The image conversion model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical image and the actual second-type historical image.

5. The method as described in claim 4, characterized in that, The image conversion model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical image and the actual second-type historical image, specifically including: The deviation between each pixel in the simulated second-type historical image and each pixel in the actual second-type historical image is taken as the pixel deviation of the image conversion model, and the recognition result of body tissue identification in the simulated second-type historical image is taken as the first recognition result. The overall deviation of the image conversion model is determined based on the pixel deviation of the image conversion model and the deviation between the first recognition result and the actual recognition result corresponding to the actual second type of historical image. The image conversion model is trained with the goal of minimizing the overall bias of the model.

6. The method as described in claim 1, characterized in that, Training the image registration model specifically includes: Acquire a second sample image, wherein the second sample image includes a first type of historical sample image of the patient at the body tissue and an actual second type of historical sample image corresponding to the first type of historical sample image; The first type of historical sample image is input into a pre-trained image conversion model to determine the simulated second type of historical sample image corresponding to the first type of historical sample image; The actual second-type historical sample image and the simulated second-type historical sample image are input into the image registration model to determine the global transformation parameters to be optimized between the pixels contained in the first-type historical sample image and the pixels contained in the actual second-type historical sample image. The first type of historical sample image and the actual second type of historical sample image are registered using the global transformation parameters to be optimized. The image registration model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical sample image corresponding to the registered first-type historical sample image and the actual second-type historical sample image.

7. The method as described in claim 6, characterized in that, The image registration model is trained with the optimization objective of minimizing the deviation between the simulated second-type historical sample image corresponding to the registered first-type historical sample image and the actual second-type historical sample image. The training specifically includes: The deviation between each pixel in the simulated second-type historical sample image corresponding to the registered first-type historical sample image and each pixel in the actual second-type historical sample image is taken as the pixel deviation of the image registration model, and the recognition result of body tissue identification of the simulated second-type historical sample image is taken as the second recognition result. The overall deviation of the image registration model is determined based on the pixel deviation of the image registration model and the deviation between the second recognition result and the actual recognition result corresponding to the actual second type of historical sample image. The image registration model is trained with the goal of minimizing the overall bias of the model.

8. The method according to any one of claims 1 to 7, characterized in that, The first type of image includes: magnetic resonance (MR) images, and the second type of image includes: computed tomography (CT) images.

9. An image registration device, characterized in that, include: The acquisition module acquires first-type images and second-type images of the patient's body tissues from different types of devices, wherein the first-type images and the second-type images are images of different types. The conversion module inputs the first type of image into a pre-trained image conversion model to determine the simulated second type of image corresponding to the first type of image; The determination module inputs the second type image and the simulated second type image into a pre-trained image registration model to determine the global transformation parameters for all pixels in the first type image or the second type image; The registration module registers the first type image and the second type image using the global conversion parameters; wherein, the first type image is adjusted using the global conversion parameters so that the corresponding pixels in the adjusted first type image and the second type image are located in the same spatial position, thereby improving the display effect of the first type image; and the adjusted first type image and the second type image are fused to obtain a fused image containing the features of the second type image and the features of the first type image.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 8.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 8.

Images