Three-dimensional medical image registration method and device, electronic equipment and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI
- Filing Date
- 2023-04-14
- Publication Date
- 2026-06-26
Smart Images
Figure CN116385512B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical image processing technology, specifically to a three-dimensional medical image registration method, apparatus, electronic device, and storage medium. Background Technology
[0002] In recent years, many scholars have proposed a number of 3D image registration methods, which can be generally divided into three categories: classic iterative optimization registration algorithms, image registration paradigms based on convolutional neural networks (CNNs), and registration methods based on Transformers.
[0003] Classic iterative optimization registration algorithms show good results in image registration, but they require defining a new objective function for each pair of images, making the registration process time-consuming and inefficient.
[0004] Image registration paradigms based on Convolutional Neural Networks (CNNs) learn the parameters of their models on training datasets to predict deformation fields between unseen image pairs. However, currently used medical image registration paradigms rely on a single model to register all images based on similarity metrics, which can lead to mismatches in complex scenes and when images are distorted.
[0005] Supervised registration introduces a synthetic deformation field as supervision, but realistic deformation fields are difficult to obtain. To effectively complete deformable registration, it is necessary to infer the semantic correspondences of fine-grained structures within the region of interest. However, due to the varying shapes and scales of volumetric images, identifying truly matching anatomical structures is challenging. Unsupervised registration models only have correspondence capabilities but lack perceptual capabilities, making it difficult to handle misalignments in blurred anatomical structures. For image pairs with large initial deviations, existing registration techniques often fail.
[0006] Transformer-based registration methods are better suited for representing spatial relationships in large deformation registration, and their application in 3D registration has been explored. However, they are mainly used in brain map registration and are not yet applicable to large deformation scenarios. Summary of the Invention
[0007] This application provides a three-dimensional medical image registration method to solve the problems of misregistration and registration failure that occur in the prior art when dealing with complex scenes, image distortion, and image pairs with large initial deviations.
[0008] Accordingly, embodiments of this application also provide an image registration device, an electronic device, and a storage medium to ensure the implementation and application of the above methods.
[0009] To address the aforementioned technical problems, this application discloses a three-dimensional medical image registration method, the method comprising:
[0010] A target point is determined from the moving image that matches a preset reference point on the reference image, and the displacement deviation of the target point relative to the reference point is obtained; one or more reference points are preset.
[0011] The global displacement of the moving image is determined based on the displacement deviation of all the target points.
[0012] The moving image is registered based on the global displacement.
[0013] This application also discloses a three-dimensional medical image registration device, the device comprising:
[0014] The feature query module is used to determine a target point from the moving image that matches a preset reference point on the reference image, and to obtain the displacement deviation of the target point relative to the reference point; one or more reference points are preset.
[0015] The position reset module is used to determine the global displacement of the moving image based on the displacement deviation of all the target points;
[0016] The position reset module is also used to register the moving image based on the global displacement.
[0017] This application also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements one or more of the methods described in this application.
[0018] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements one or more of the methods described in this application.
[0019] In this embodiment, target points that match preset reference points on a reference image are determined from the moving image. Since one or more reference points are preset, there are also one or more target points, enabling matching between reference points on the reference image and target points on the moving image. Based on the displacement deviations of all target points, the global displacement of the moving image is determined, which constrains all target points in the moving image and establishes spatial relationships between them. Registering the moving image based on the global displacement ensures that the points in the registered moving image conform as closely as possible to the requirements of anatomical structures.
[0020] Additional aspects and advantages of the embodiments of this application will be set forth in the following description, and will become apparent from the description or may be learned by practice of this application. Attached Figure Description
[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0022] Figure 1 A flowchart of a three-dimensional medical image registration method provided in the embodiments of this application;
[0023] Figure 2 A network framework diagram of the three-dimensional medical image registration method provided in the embodiments of this application;
[0024] Figure 3 A flowchart illustrating the three-dimensional medical image registration method provided in the embodiments of this application;
[0025] Figure 4 A schematic diagram of a transponder module provided in an embodiment of this application;
[0026] Figure 5 A schematic diagram of the structure of the three-dimensional medical image registration device provided in the embodiments of this application;
[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0028] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.
[0029] Those skilled in the art will understand that, unless explicitly stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.
[0030] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.
[0031] The solutions provided in this application can be executed by any electronic device, such as a terminal device or a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. The three-dimensional medical image registration method, apparatus, electronic device, and storage medium provided in this application aim to solve at least one of the technical problems existing in the prior art.
[0032] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0033] This application provides a possible implementation method, such as... Figure 1 The diagram shows a flowchart of a three-dimensional medical image registration method. This method can be executed by any electronic device, optionally on a server or a terminal device.
[0034] like Figure 1 As shown, the method may include the following steps:
[0035] Step 101: Determine a target point from the moving image that matches a preset reference point on the reference image, and obtain the displacement deviation of the target point relative to the reference point.
[0036] There are one or more preset reference points, therefore, there can also be one or more target points obtained.
[0037] The reference point can be a global point on the reference image or a point within a local region of interest on the reference image. In this embodiment, for each reference point on the reference image, a corresponding target point is obtained from the moving image, and the displacement deviation of the target point relative to the reference point is obtained. Specifically, this can be done as follows:
[0038] Reference Figure 2 The reference feature vector corresponding to the pre-obtained reference image, the moving feature vector corresponding to the pre-obtained moving image, and the reference point vector corresponding to the pre-set reference point are input into the query responder. With the assistance of the reference feature vector and the moving feature vector, the query responder continuously optimizes the network by calculating the squared error loss between the coordinates of the corresponding target point on the moving image and its true coordinates, so that the network outputs the displacement deviation between the coordinates of the target point on the moving image and the coordinates of the reference point on the reference image.
[0039] In this embodiment, the position coordinates of the target point can be calculated based on the position coordinates of the reference point and the displacement deviation of the corresponding target point, so as to determine the target point that matches the reference point.
[0040] Step 102: Determine the global displacement of the moving image based on the displacement deviation of all the target points.
[0041] Figure 2 In this process, the global displacement of the moving image is determined based on the displacement deviations of all the target points, which is achieved through a position resetter. Specifically, the displacement deviations of the target points include deviations in the X direction, Y direction, and Z direction. The overall deviation of the moving image in the Z direction can be obtained from the deviation of each target point in the X direction; the overall deviation of the moving image in the Y direction can be obtained from the deviation of each target point in the Y direction; and the overall deviation of the moving image in the Z direction can be obtained from the deviation of each target point in the Z direction. In this embodiment, the overall deviations of all moving images in the X direction, Y direction, and Z direction are used as the global displacement of the moving image. The global displacement of the moving image obtained from the displacement deviations of all target points can represent the overall displacement direction of the moving image.
[0042] Step 103: Register the moving image according to the global displacement.
[0043] Since the overall displacement and transformation of the moving image can be determined by the global displacement, the positional relationship between all points in the moving image can be constrained, and points with unreasonable deformation can be restricted, so that all points conform to the requirements of the anatomical structure as much as possible, making the final registration result clinically significant.
[0044] Optionally, such as Figure 2 As shown, the registration direction can be determined based on the global displacement, and the moving image can be registered by grid sampling to obtain the response image (i.e., the registered moving image).
[0045] The loss value between the moving image and the response image can be calculated using a cross-correlation loss function to optimize the network.
[0046] In this embodiment, if the reference point is a point in a local region of interest on the reference image, the network can be driven to learn more features of the region of interest, thereby obtaining a more accurate and finer-grained registration result.
[0047] In this embodiment, target points that match preset reference points on a reference image are determined from the moving image. Since one or more reference points are preset, there are also one or more target points, enabling matching between reference points on the reference image and target points on the moving image. Based on the displacement deviations of all target points, the global displacement of the moving image is determined, which constrains all target points in the moving image and establishes spatial relationships between them. Registering the moving image based on the global displacement ensures that the points in the registered moving image conform as closely as possible to the requirements of anatomical structures.
[0048] In an optional embodiment, refer to Figure 3 Before determining the target point that matches a preset reference point on the reference image from the moving image and obtaining the displacement deviation of the target point relative to the reference point, the method further includes:
[0049] The feature map of the reference image and the feature map of the moving image are extracted using an encoder.
[0050] The encoder's role is to extract features from the input image pair (reference image and moving image). In this embodiment, a convolutional neural network (CNN) trained on the backbone of the Med3D network can be used as the encoder for feature extraction. Optionally, the encoder extracting feature maps from the reference image and the encoder extracting features from the moving image share weights. Sharing weights reduces the number of parameters in the convolutional neural network, thereby reducing the risk of overfitting and improving the model's generalization ability. Furthermore, the same weights can be reused at different time steps or locations, reducing the number of weight updates and making the training process more efficient.
[0051] Optionally, feature maps of the reference image and the moving image are extracted separately using a weighted encoder, as follows:
[0052] Will I r and Is ∈R D×H×W The reference image and the moving image are respectively cropped and resized to D,H,W=128. Then the feature maps of the reference image and the moving image are extracted respectively.
[0053] In an optional embodiment, determining a target point from the moving image that matches a preset reference point on the reference image, and obtaining the displacement deviation of the target point relative to the reference point, includes:
[0054] Based on the feature map of the reference image, the feature map of the moving image, and the reference point, a target point matching the reference point is determined, and the displacement deviation is obtained.
[0055] Specifically, the correlation between and within the reference image and the moving image can be simulated using the feature map of the reference image, the feature map of the moving image, and the reference point, thereby determining the target point corresponding to the reference point.
[0056] In an optional embodiment, determining the target point matching the reference point based on the feature map of the reference image, the feature map of the moving image, and the reference point, and obtaining the displacement deviation, includes:
[0057] The feature map of the reference image, the feature map of the moving image, and the reference point are positionally encoded to generate a reference feature vector, a moving feature vector, and a reference point vector, respectively.
[0058] The displacement deviation of the target point relative to the reference point is obtained by processing the reference feature vector, the moving feature vector, and the reference point vector through a self-attention mechanism network.
[0059] Optionally, refer to Figure 3 The feature map of the reference image and the feature map of the moving image are merged into a feature sequence;
[0060] The feature sequence is then input into a linear mapping layer, and the feature maps in the feature sequence are then positionally encoded.
[0061] Specifically, the feature maps of the reference image and the moving image are merged along the Z-axis to obtain a feature sequence, as shown in the figure below. Among them, D C H C , C = 128.
[0062] Alternatively, for each feature map (h,w,d), its sine and cosine encoded vectors can be calculated using the following formulas:
[0063]
[0064]
[0065] Where h represents the height of the feature map, w represents the width of the feature map, d represents the dimension of the feature map, and i represents the i-th feature map.
[0066] Finally, the sine and cosine encoded vectors can be concatenated according to certain rules, for example, as follows:
[0067]
[0068] After concatenating the features in the above manner, the final feature vector (including the reference feature vector and the moved feature vector) is obtained.
[0069] The reference point can be position-encoded as follows:
[0070] The position of each reference point is encoded into a reference point vector i (i∈R). M ), where M = 192.
[0071] In this embodiment, the reference point vector has the same size as the reference feature vector and the moving feature vector.
[0072] In this embodiment, refer to Figure 3 The aforementioned feature vectors (including reference feature vectors and moving feature vectors) and reference point vectors can be input. Figure 3 The query responder shown includes a responder module, which is deployed with a self-attention mechanism network, or the responder module is composed of a self-attention mechanism network. The self-attention mechanism network can simulate the correlation between and within a reference image and a moving image. The specific algorithm for simulating the correlation between and within a reference image and a moving image using the self-attention mechanism network is as follows:
[0073]
[0074] Where Q is the reference point vector; K and V are both vectors obtained by mapping the feature map of the reference image and the feature map of the moving image.
[0075] The multilayer perceptron can calculate and process the feature parameters obtained after processing by the transponder module to determine the target point and the displacement deviation of the target point relative to the reference point, and output the displacement deviation of the target point.
[0076] Both the reference point and the target point have corresponding positional information, such as coordinates. The displacement deviation of the target point relative to the reference point can be calculated based on the positional information of both the target point and the reference point.
[0077] In this embodiment, the position coordinates (x1, y1, z1) of the target point can be obtained using the position coordinates (x, y, z) of the reference point and the displacement deviation of the target point. Specifically, the position coordinates (x1, y1, z1) of the target point are obtained by adding the position coordinates (x, y, z) of the reference point to the displacement deviation of the target point. Figure 3 The response results (x1, y1, z1) are shown in the figure.
[0078] Optionally, refer to Figure 4 The transponder module consists of multiple sets of encoders and decoders connected in series. Each set of encoders and decoders comprises a multi-head attention layer, a normalization layer, and a feedforward network layer. The process of simulating the correlation between and within the reference image and the moving image in the transponder module can be as follows:
[0079] In the encoder section, the reference points and the reference point vector formed by position encoding the reference points are input into the multi-head attention layer and the normalization layer (each set of multi-head attention layer and normalization layer can be looped multiple times); then the result is used as the input to the decoder section.
[0080] In the decoder section, the feature vector is input into the multi-head attention layer, then passes through a normalization layer, followed by a feedforward network layer and another normalization layer. This normalization layer, along with the result received from the encoder section, is then input into the multi-head attention layer and normalization layer, the feedforward network layer and another normalization layer, and finally, the prediction result is output. Each set of multi-head attention layers and normalization layers can be iterated multiple times, and each set of feedforward network layers and normalization layers can be iterated multiple times.
[0081] In an optional embodiment, determining the global displacement of the moving image based on the displacement deviation of all the target points includes:
[0082] The average displacement deviation of all the target points in the x, y and z directions is obtained, and the average displacement deviation in the x, y and z directions is used as the global displacement of the moving image.
[0083] The average displacement deviation of the target point in the x, y, and z directions can reflect the overall offset direction of the moving image.
[0084] In an optional embodiment, registering the moving image based on the global displacement includes:
[0085] Mesh sampling is performed based on the global displacement to obtain a translation-corrected moving image.
[0086] When moving or transforming an image based on global displacement, it can prevent some local points in the moving image from being incorrectly identified, which could lead to incorrect interpretations or decisions.
[0087] Based on the same principle as the method provided in the embodiments of this application, the embodiments of this application also provide an image registration device, such as... Figure 5 As shown, the device includes:
[0088] The feature query module 501 is used to determine a target point that matches a preset reference point on a reference image from the moving image, and to obtain the displacement deviation of the target point relative to the reference point.
[0089] There are one or more preset reference points, therefore, there can also be one or more target points obtained.
[0090] The reference point can be a global point on the reference image or a point within a local region of interest on the reference image. In this embodiment, for each reference point on the reference image, a corresponding target point is obtained from the moving image, and the displacement deviation of the target point relative to the reference point is obtained. Specifically, this can be done as follows:
[0091] Reference Figure 2 The reference feature vector corresponding to the pre-obtained reference image, the moving feature vector corresponding to the pre-obtained moving image, and the reference point vector corresponding to the pre-set reference point are input into the query responder. With the assistance of the reference feature vector and the moving feature vector, the query responder continuously optimizes the network by calculating the squared error loss between the coordinates of the corresponding target point on the moving image and its true coordinates, so that the network outputs the displacement deviation between the coordinates of the target point on the moving image and the coordinates of the reference point on the reference image.
[0092] In this embodiment, the position coordinates of the target point can be calculated based on the position coordinates of the reference point and the displacement deviation of the corresponding target point, so as to determine the target point that matches the reference point.
[0093] The position reset module 502 is used to determine the global displacement of the moving image based on the displacement deviation of all the target points.
[0094] Figure 2In this process, the global displacement of the moving image is determined based on the displacement deviations of all the target points, which is achieved through a position resetter. Specifically, the displacement deviations of the target points include deviations in the X direction, Y direction, and Z direction. The overall deviation of the moving image in the Z direction can be obtained from the deviation of each target point in the X direction; the overall deviation of the moving image in the Y direction can be obtained from the deviation of each target point in the Y direction; and the overall deviation of the moving image in the Z direction can be obtained from the deviation of each target point in the Z direction. In this embodiment, the overall deviations of all moving images in the X direction, Y direction, and Z direction are used as the global displacement of the moving image. The global displacement of the moving image obtained from the displacement deviations of all target points can represent the overall displacement direction of the moving image.
[0095] The position reset module 502 is also used to register the moving image according to the amount of displacement.
[0096] Since the overall displacement and transformation of the moving image can be determined by the global displacement, the positional relationship between all points in the moving image can be constrained, and points with unreasonable deformation can be restricted, so that all points conform to the requirements of the anatomical structure as much as possible, making the final registration result clinically significant.
[0097] Optionally, such as Figure 2 As shown, the registration direction can be determined based on the global displacement, and the moving image can be registered by grid sampling to obtain the response image (i.e., the registered moving image).
[0098] The loss value between the moving image and the response image can be calculated using a cross-correlation loss function to optimize the network.
[0099] In this embodiment, if the reference point is a point in a local region of interest on the reference image, the network can be driven to learn more features of the region of interest, thereby obtaining a more accurate and finer-grained registration result.
[0100] In this embodiment, the feature query module determines target points on the moving image that match preset reference points on the reference image; wherein, one or more reference points are preset, and therefore, one or more target points are also preset, enabling matching between reference points on the reference image and target points on the moving image; the position determination module determines the global displacement of the moving image based on the displacement deviation of all the target points, which can constrain all target points in the moving image and establish spatial relationships between all target points; the position reset module also registers the moving image based on the global displacement, which can make the points in the registered moving image conform to the requirements of anatomical structures as much as possible.
[0101] In one optional embodiment of this application, the apparatus further includes:
[0102] The feature extraction module is used to extract feature maps of the reference image and the moving image using the encoder.
[0103] In one optional embodiment of this application, the feature query module 501 includes:
[0104] The first feature query submodule determines the target point that matches the reference point based on the feature map of the reference image, the feature map of the moving image, and the reference point, and obtains the displacement deviation.
[0105] In one optional embodiment of this application, the first feature query submodule includes:
[0106] The feature encoding unit is used to perform position encoding on the feature map of the reference image, the feature map of the moving image, and the reference point, respectively generating a reference feature vector, a moving feature vector, and a reference point vector.
[0107] The feature processing unit is used to process the reference feature vector, the moving feature vector, and the reference point vector through a self-attention mechanism network to obtain the displacement deviation of the target point relative to the reference point.
[0108] In one optional embodiment of this application, the reference point vector has the same size as the reference feature vector and the moving feature vector.
[0109] In one optional embodiment of this application, the position reset module 502 includes:
[0110] The first position reset submodule is used to obtain the average displacement deviation of the displacement deviation of all the target points in the x, y and z directions, respectively, and use the average displacement deviation in the x, y and z directions as the global displacement of the moving image.
[0111] In one optional embodiment of this application, the position reset module 502 further includes:
[0112] The second position reset submodule is used to perform grid sampling based on the global displacement to obtain a translation-corrected moving image.
[0113] The image registration device provided in this application embodiment can achieve... Figures 1 to 5 The various processes implemented in the method embodiments are not described in detail here to avoid repetition.
[0114] The three-dimensional medical image registration device of this application embodiment can execute the three-dimensional medical image registration method provided in this application embodiment. The implementation principle is similar. The actions performed by each module and unit in the three-dimensional medical image registration device in each embodiment of this application are corresponding to the steps in the three-dimensional medical image registration method in each embodiment of this application. For detailed functional descriptions of each module of the three-dimensional medical image registration device, please refer to the descriptions in the corresponding three-dimensional medical image registration methods shown above. They will not be repeated here.
[0115] Based on the same principles as the methods shown in the embodiments of this application, embodiments of this application also provide an electronic device, which may include, but is not limited to, a processor and a memory; the memory for storing computer programs; and the processor for executing the three-dimensional medical image registration method shown in any optional embodiment of this application by calling the computer program. Compared with the prior art, the three-dimensional medical image registration method provided by this application determines target points on a moving image that match preset reference points on a reference image; wherein, one or more reference points are preset, therefore, one or more target points are also preset, which can realize the matching of reference points on the reference image and target points on the moving image; based on the displacement deviation of all the target points, the global displacement of the moving image is determined, which can constrain all target points in the moving image and establish the spatial relationship between all target points; registering the moving image based on the global displacement can make the points in the registered moving image conform as closely as possible to the requirements of anatomical structures.
[0116] In an alternative embodiment, an electronic device, such as Figure 6 As shown, Figure 6 The illustrated electronic device 600 can be a server, including a processor 601 and a memory 603. The processor 601 and the memory 603 are connected, for example, via a bus 602. Optionally, the electronic device 600 may also include a transceiver 604. It should be noted that in practical applications, the transceiver 604 is not limited to one type, and the structure of this electronic device 600 does not constitute a limitation on the embodiments of this application.
[0117] Processor 601 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 601 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
[0118] Bus 602 may include a pathway for transmitting information between the aforementioned components. Bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 602 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0119] The memory 603 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.
[0120] The memory 603 stores application code that executes the scheme of this application, and its execution is controlled by the processor 601. The processor 601 executes the application code stored in the memory 603 to implement the content shown in the foregoing method embodiments.
[0121] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0122] The server provided in this application can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.
[0123] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.
[0124] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0125] It should be noted that the computer-readable storage medium described above in this application can also be a computer-readable signal medium or a combination of computer-readable storage media and computer-readable storage media. Computer-readable storage media can be, for example,—but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0126] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0127] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.
[0128] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the three-dimensional medical image registration method provided in the various alternative implementations described above.
[0129] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0131] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules are not necessarily limiting in certain circumstances; for example, a position reset module can also be described as "a position reset module for registering the moving image based on the global displacement".
[0132] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.
Claims
1. A three-dimensional medical image registration method, characterized in that, The method includes: The encoder is used to extract feature maps from the reference image and the moving image; Determining a target point from a moving image that matches a preset reference point on a reference image, and obtaining the displacement deviation of the target point relative to the reference point, includes: performing position encoding on the feature map of the reference image, the feature map of the moving image, and the reference point to generate a reference feature vector, a moving feature vector, and a reference point vector, respectively; processing the reference feature vector, the moving feature vector, and the reference point vector through a self-attention mechanism network to obtain the displacement deviation of the target point relative to the reference point, wherein the reference point vector has the same size as the reference feature vector and the moving feature vector; one or more reference points are preset. The global displacement of the moving image is determined based on the displacement deviation of all the target points. The moving image is registered based on the global displacement.
2. The three-dimensional medical image registration method according to claim 1, characterized in that, Determining the global displacement of the moving image based on the displacement deviation of all the target points includes: The average displacement deviation of all the target points in the x, y and z directions is obtained, and the average displacement deviation in the x, y and z directions is used as the global displacement of the moving image.
3. The three-dimensional medical image registration method according to any one of claims 1-2, characterized in that, Registering the moving image based on the global displacement includes: Mesh sampling is performed based on the global displacement to obtain a translation-corrected moving image.
4. A three-dimensional medical image registration device, characterized in that, The device includes: The feature query module is used to extract feature maps of a reference image and a moving image using an encoder, determine a target point from the moving image that matches a preset reference point on the reference image, and obtain the displacement deviation of the target point relative to the reference point. This includes: performing position encoding on the feature maps of the reference image, the moving image, and the reference point to generate a reference feature vector, a moving feature vector, and a reference point vector, respectively; processing the reference feature vector, the moving feature vector, and the reference point vector through a self-attention mechanism network to obtain the displacement deviation of the target point relative to the reference point; wherein the reference point vector has the same size as the reference feature vector and the moving feature vector; and one or more reference points are preset. The position reset module is used to determine the global displacement of the moving image based on the displacement deviation of all the target points; The position reset module is also used to register the moving image based on the global displacement.
5. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method of any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 3.