Methods, systems, apparatus, and storage media of visualizing registered medical images

Abstract

The embodiment of the specification provides a method, system, device and storage medium for visualizing registration of medical images, the method comprising: acquiring a plurality of medical images; dividing the plurality of medical images into at least one group according to a registration relationship between the plurality of medical images; acquiring a registration relationship between the plurality of medical images in the at least one group, and visualizing the registration relationship; wherein the visualizing the registration relationship comprises visualizing a registration method and a registration direction between a reference medical image and a registered medical image in the plurality of medical images in the at least one group.

Description

Technical Field

[0001] This specification relates to the field of medical technology, and in particular to a method and system for visualizing and registering medical images. Background Technology

[0002] With advancements in medical imaging technology, doctors can integrate images from multiple modalities and examinations to make more comprehensive clinical diagnoses. This comprehensive information improves diagnostic accuracy and represents the current trend in imaging diagnosis. Integrating information from multiple images requires registering them within a single spatial dimension for comparative observation. However, with the increasing number and variety of images in the current environment, registering all images would result in an explosive increase in the number of registration attempts. This leads to both lengthy registration times and complex relationships between images, making it difficult for users to troubleshoot registration failures and adjust registration relationships when desired.

[0003] Therefore, it is desirable to provide a method and system for visualizing and registering medical images. Summary of the Invention

[0004] One embodiment of this specification provides a method for visualizing registered medical images. The method includes: acquiring multiple medical images; dividing the multiple medical images into at least one group based on registration relationships between the multiple medical images; acquiring the registration relationships between the multiple medical images in the at least one group; and visualizing the registration relationships; the visualization of the registration relationships includes visualizing a registration method and / or a registration direction between a reference medical image and a medical image to be registered in the at least one group of multiple medical images; the registration method includes at least one of coordinate alignment registration, registration using a registration algorithm, and manual registration; the registration direction is used to indicate the relationship between the reference medical image and the medical image to be registered.

[0005] In some embodiments, the method for visualizing and registering medical images may further include: acquiring user operations on a registration relationship visualization interface, and adjusting the registration relationship between the plurality of medical images based on the operations; the operations include modifying the registration method, deleting registration relationships, and adding registration relationships.

[0006] In some embodiments, the registration relationship between the plurality of medical images can be adjusted in the following way: when the user changes the registration method between the reference medical image and the registered medical image from the first registration method to the second registration method, the reference medical image and the registered medical image are registered based on the second registration method, and the registration relationship between the reference medical image and the registered medical image is visualized as the second registration method.

[0007] In some embodiments, the registration relationship between the plurality of medical images can be adjusted in the following way: when the user deletes the existing registration relationship between the reference medical image and the registered medical image, the registration relationship between the reference medical image and the registered medical image is updated, and the updated registration relationship between the reference medical image and the registered medical image is visualized.

[0008] In some embodiments, the registration relationship between the plurality of medical images can be adjusted in the following manner: when a user increases the registration relationship between the reference medical image and the registered medical image, the reference medical image and the registered medical image are registered based on the selected registration method, and the registration relationship between the reference medical image and the registered medical image is visualized.

[0009] In some embodiments, the method for visualizing registered medical images may further include: responding to a user's selection operation on a browsing interface for multiple registered medical images; displaying the multiple registered medical images; opening the registration relationship visualization interface; and adjusting the registration relationship between the multiple registered medical images.

[0010] In some embodiments, the registration between the reference medical image and the registered medical image may include coarse registration and fine registration. The coarse registration may include: performing preliminary intra-group registration based on the coordinate positions of the reference medical image and the registered medical image within the same group to obtain an intra-group registration result; and performing preliminary inter-group registration based on key points extracted from the reference medical image and the registered medical image and the intra-group registration result to obtain a coarse registration result. The fine registration may include: performing precise registration based on the similarity between the reference medical image and the registered medical image and the coarse registration result to obtain a target registration result, whereby the target registration result represents the spatial transformation relationship between the reference medical image and the registered medical image during the registration process.

[0011] One embodiment of this specification provides a system for visualizing and registering medical images, including a grouping acquisition module and a visualization module. The grouping acquisition module is used to acquire multiple medical images and divide the multiple medical images into at least one group according to the registration relationship between the multiple medical images. The visualization module is used to acquire the registration relationship between the multiple medical images in the at least one group and visualize the registration relationship. The visualization of the registration relationship includes visualizing the registration method and / or registration direction between the reference medical image and the registered medical image in the at least one group of multiple medical images. The registration method includes at least one of coordinate alignment registration, registration using a registration algorithm, and manual registration. The registration direction is used to indicate the relationship between the reference medical image and the registered medical image.

[0012] In some embodiments, the system for visualizing and registering medical images may further include an adjustment module; the adjustment module may be used to acquire user operations on the registration relationship visualization interface and adjust the registration relationship between the plurality of medical images based on the operations; the operations include modifying the registration method, deleting registration relationships, and adding registration relationships.

[0013] One embodiment of this specification provides an apparatus for visualizing and registering medical images, including a processor for performing a method for visualizing and registering the medical images.

[0014] One embodiment of this specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the method for visual registration of medical images.

[0015] In some embodiments of this specification, a large number of images (e.g., different modalities, different times, different beds, etc.) are grouped based on registration relationships, and registration-related operations are performed directly based on a visualized registration relationship diagram. The process is clear and convenient, which facilitates the management of registration relationships between large-scale images. This solves the problem of poor image management caused by excessive registration, improves the convenience of registration for large-scale images, and allows for easy adjustment of registration relationships to meet the needs of different users and ensure the diagnostic and treatment effects based on registered images. Attached Figure Description

[0016] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0017] Figure 1 These are schematic diagrams illustrating application scenarios of a system for visually registering medical images, based on some embodiments of this specification.

[0018] Figure 2 This is a schematic diagram of a system for visualizing and registering medical images according to some embodiments of this specification;

[0019] Figure 3 This is an exemplary flowchart of a method for visualizing and registering medical images according to some embodiments of this specification;

[0020] Figure 4 This is a schematic diagram of a registration relationship visualization interface shown in some embodiments of this specification;

[0021] Figure 5 This is a schematic diagram illustrating the visual modification of the registration relationship according to some embodiments of this specification;

[0022] Figure 6 This is a schematic diagram illustrating the visual deletion registration relationship according to some embodiments of this specification;

[0023] Figure 7 This is a schematic diagram illustrating the visualization of increased registration relationships according to some embodiments of this specification. Detailed Implementation

[0024] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0025] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0026] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0027] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0028] Figure 1 This is a schematic diagram illustrating the application scenario of a system for visualizing and registering medical images, based on some embodiments of this specification.

[0029] like Figure 1 As shown, in some embodiments, system 100 may include medical imaging equipment 110, processing equipment 120, storage device 130, terminal 140, and network 150.

[0030] Medical imaging device 110 refers to a medical device that uses different media to reproduce the internal structures of the human body as images. In some embodiments, medical imaging device 110 can be any medical device capable of imaging or treating a specified body part of a patient, such as one or any combination of computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), etc. The medical imaging device 110 described above is for illustrative purposes only and is not intended to limit its scope. In some embodiments, medical imaging device 110 may include multiple different types of medical imaging devices for acquiring multiple medical images of different modalities. In some embodiments, medical imaging device 110 can transmit the scanned medical images to processing device 120. Medical imaging device 110 can receive instructions from doctors via terminal 140 and perform related operations according to the instructions, such as scanning and imaging. In some embodiments, the medical imaging device 110 may exchange data and / or information with other components in the system 100 (e.g., processing device 120, storage device 130, terminal 140) via network 150. In some embodiments, the medical imaging device 110 may be directly connected to other components in the system 100. In some embodiments, one or more components in the system 100 (e.g., processing device 120, storage device 130) may be included within the medical imaging device 110.

[0031] Processing device 120 can process data and / or information obtained from other devices or system components, and perform the visualization registration method for medical images shown in some embodiments of this specification based on this data, information, and / or processing results to accomplish one or more functions described in some embodiments of this specification. For example, processing device 120 can group multiple medical images from medical imaging device 110 according to registration relationships. As another example, processing device 120 can acquire the registration relationships between grouped medical images and visualize these relationships. In some embodiments, processing device 120 can acquire user operations on the registration relationship visualization interface and adjust the registration relationships between medical images based on these operations. In some embodiments, processing device 120 can perform coarse and fine registration of medical images within and between groups to obtain registration results, such as registration matrices. In some embodiments, processing device 120 can send the visualized registration relationships to terminal 140 for users (e.g., doctors) to view and adjust. In some embodiments, processing device 120 can receive instructions sent by users through the visualization interface of terminal 140 to adjust the registration relationships.

[0032] In some embodiments, the processing device 120 may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-chip processing device). By way of example only, the processing device 120 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction processor (ASIP), a graphics processing unit (GPU), a physical processor (PPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, or any combination thereof.

[0033] Storage device 130 can store data or information generated by other devices. In some embodiments, storage device 130 can store data and / or information acquired by medical imaging device 110, such as multiple modal medical images. In some embodiments, storage device 130 can store data and / or information processed by processing device 120, such as registration relationships between medical images. Storage device 130 may include one or more storage components, each of which may be an independent device or part of other devices. Storage device can be local or implemented via the cloud.

[0034] Terminal 140 can control the operation of medical imaging equipment 110. Doctors can issue operating instructions to medical imaging equipment 110 through terminal 140 to enable it to perform specified operations, such as scanning and imaging a specified body part of a patient. In some embodiments, terminal 140 can instruct processing device 120 to perform a method for visually registering medical images as shown in some embodiments of this specification. In some embodiments, terminal 140 can receive the registration relationship between visualized medical images from processing device 120 and display a registration relationship visualization interface on a screen, allowing doctors to accurately grasp the registration relationship between multiple medical images for effective and targeted examination and / or treatment of patients. In some embodiments, doctors can issue operating instructions to processing device 120 through the registration relationship visualization interface displayed on the screen of terminal 140 to adjust the registration relationship. In some embodiments, terminal 140 can be one or any combination of mobile device 140-1, tablet computer 140-2, laptop computer 140-3, desktop computer, or other devices with display screens, input and / or output functions.

[0035] Network 150 can connect the various components of the system and / or connect the system to external resources. Network 150 enables communication between the components and with other parts outside the system, facilitating the exchange of data and / or information. In some embodiments, one or more components in system 100 (e.g., medical imaging device 110, processing device 120, storage device 130, terminal 140) can send data and / or information to other components via network 150. In some embodiments, network 150 can be any one or more of a wired network or a wireless network.

[0036] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. Various changes and modifications can be made by those skilled in the art based on the content of this specification. Features, structures, methods, and other features of the exemplary embodiments described herein can be combined in various ways to obtain other and / or alternative exemplary embodiments. For example, the processing device 120 may be based on a cloud computing platform, such as a public cloud, private cloud, community cloud, and hybrid cloud. However, these changes and modifications will not depart from the scope of this specification.

[0037] Figure 2 This is a schematic diagram of a system for visualizing and registering medical images according to some embodiments of this specification.

[0038] like Figure 2 As shown, in some embodiments, the system 200 for visualizing registered medical images may include a group acquisition module 210 and a visualization module 220.

[0039] In some embodiments, the group acquisition module 210 can be used to acquire multiple medical images and divide these medical images into at least one group according to the registration relationship between them.

[0040] In some embodiments, the visualization module 220 can be used to acquire and visualize the registration relationships between at least one set of multiple medical images. The visualization of the registration relationships can include visualizing the registration method and / or registration direction between a reference medical image and a registered medical image in at least one set of multiple medical images. The registration method can include at least one of coordinate alignment registration, registration using a registration algorithm, and manual registration. The registration direction can be used to indicate the relationship between the reference medical image and the registered medical image.

[0041] In some embodiments, the system 200 for visualizing and registering medical images may further include an adjustment module 230. The adjustment module 230 can be used to acquire user operations on the registration relationship visualization interface and adjust the registration relationships between multiple medical images based on these operations. These operations include modifying the registration method, deleting registration relationships, and adding registration relationships.

[0042] In some embodiments, the registration relationship between multiple medical images can be adjusted in the following way: when the user changes the registration method between the reference medical image and the registered medical image from the first registration method to the second registration method, the adjustment module 230 can register the reference medical image and the registered medical image based on the second registration method, and visually display the registration relationship between the reference medical image and the registered medical image as the second registration method.

[0043] In some embodiments, the registration relationship between multiple medical images can be adjusted in the following way: when the user deletes the existing registration relationship between the reference medical image and the registered medical image, the adjustment module 230 can update the registration relationship between the reference medical image and the registered medical image and visually display the updated registration relationship between the reference medical image and the registered medical image.

[0044] In some embodiments, the registration relationship between multiple medical images can be adjusted in the following way: when a user adds a registration relationship between a reference medical image and a medical image to be registered, the adjustment module 230 can register the reference medical image and the medical image to be registered based on the selected registration method, and visualize the registration relationship between the reference medical image and the medical image to be registered.

[0045] In some embodiments, the system 200 for visualizing registered medical images may further include a browsing module (not shown in the figures). The browsing module can be used to respond to a user's selection operation of multiple registered medical images on a browsing interface; display the multiple registered medical images; open the registration relationship visualization interface; and adjust the registration relationship between the multiple registered medical images.

[0046] In some embodiments, the registration between a reference medical image and a medical image to be registered may include coarse registration and fine registration. Coarse registration may include: performing preliminary intra-group registration based on the coordinate positions of the reference and registered medical images within the same group to obtain an intra-group registration result; and performing preliminary inter-group registration based on keypoints extracted from the reference and registered medical images and the intra-group registration result to obtain a coarse registration result. Fine registration may include: performing precise registration based on the similarity between the reference and registered medical images and the coarse registration result to obtain a target registration result, wherein the target registration result represents the spatial transformation relationship between the reference and registered medical images during the registration process.

[0047] Figure 3 This is an exemplary flowchart of a method for visualizing and registering medical images according to some embodiments of this specification.

[0048] like Figure 3As shown, process 300 includes the following steps. In some embodiments, process 300 may be executed by processing device 120.

[0049] Step 310: Acquire multiple medical images and group them into at least one group based on the registration relationship between the multiple medical images. In some embodiments, step 310 may be performed by the group acquisition module 210.

[0050] Medical images refer to images obtained by medical imaging equipment scanning a target object (e.g., the human body, phantom, etc.), such as CT images, MRI images, PET images, etc. In the field of medical imaging, doctors can register medical images from multiple modalities, multiple examinations, and different bed positions into the same coordinate space for comparative observation. By integrating these different medical image information, a more comprehensive clinical diagnosis can be made. In some embodiments, the group acquisition module 210 can acquire multiple medical images in various ways, such as directly acquiring them through medical imaging equipment, acquiring them through various data sources such as storage devices, etc. These acquired medical images may include the registration relationships between them.

[0051] In some embodiments, the grouping acquisition module 210 can group multiple medical images according to the registration relationship between them, and the number of groups can be one or more. Specifically, multiple medical images that have a registration relationship with each other can be grouped together. For example, Figure 4 As shown, images 1.1, 1.2, and 1.3 form group 1, where images 1.1 and 1.2 are both registered to image 1.3; images 2.1, 2.2, 2.3, and 2.4 form group 2, where images 2.1, 2.2, and 2.3 are all registered to image 2.4; and images 3.1, 3.2, 3.3, 3.5, and 3.4 form group 3, where images 3.1, 3.2, 3.3, and 3.5 are all registered to image 3.4.

[0052] In some embodiments, the group acquisition module 210 may group multiple medical images based on other rules (e.g., at least one of medical imaging equipment, coordinate reference system, and anatomical location).

[0053] Step 320: Obtain the registration relationship between at least one set of multiple medical images and visualize the registration relationship. In some embodiments, step 320 may be performed by visualization module 220.

[0054] In some embodiments, the visualization module 220 can obtain the registration relationship between at least one group of medical images from the grouped medical images. For example, it can obtain... Figure 4The registration relationships between medical images in groups 1, 2, and 3 are shown. Intra-group registration relationships include: images 1.1 and 1.2 in group 1 are both registered to image 1.3; images 2.1, 2.2, and 2.3 in group 2 are all registered to image 2.4; and images 3.1, 3.2, 3.3, and 3.5 in group 3 are all registered to image 3.4. Inter-group registration relationships include: images 2.4 and 3.4 are both registered to image 1.3.

[0055] In some embodiments, the visualization module 220 can visualize the acquired registration relationship. Specifically, the acquired registration relationship can be displayed on a registration relationship visualization interface. For example, Figure 4 The registration relationship visualization interface is shown.

[0056] In some embodiments, registration relationship visualization may include visualizing a registration method and / or registration direction between a reference medical image and a registered medical image in at least one set of multiple medical images.

[0057] In some embodiments, the registration method may include at least one of coordinate alignment registration, registration using a registration algorithm, and manual registration. Coordinate alignment registration refers to transforming the reference medical image and the medical image to be registered into the same scanning coordinate system. Registration using a registration algorithm refers to using various registration algorithms (e.g., registration algorithms based on image grayscale statistical characteristics, registration algorithms based on image features, etc.) for registration. Manual registration refers to registration by the user manually adjusting the image.

[0058] In some embodiments, the registration direction can be used to indicate the reference medical image and the registered medical image, where registration between the reference medical image and the registered medical image refers to the registration of the registered medical image to the reference medical image.

[0059] For example only, such as Figure 4 As shown, the images are divided into three groups: Group 1, Group 2, and Group 3, connected by registration lines that indicate the registration method and direction between the images. In some embodiments, there can be multiple types of registration lines, depending on the type of registration algorithm, with each registration algorithm corresponding to one type of registration line. Figure 4 The registration methods include registration method 1, registration method 2, and manual registration, and the three lines correspond to these three registration methods respectively. In some embodiments, the arrow of the registration connection line points from the image to be registered to the reference medical image, indicating that the image to be registered is registered to the reference medical image.

[0060] like Figure 4As shown, in Group 1, images 1.1 and 1.2 are both registered to image 1.3 using registration method 1. The arrows indicate that images 1.1 and 1.2 are the registered medical images, and image 1.3 is the reference medical image. In Group 2, images 2.1, 2.2, and 2.3 are all registered to image 2.4 using registration method 1. The arrows indicate that images 2.1, 2.2, and 2.3 are the registered medical images, and image 2.4 is the reference medical image. In Group 3, images 3.1, 3.2, 3.3, and 3.5 are all registered to image 3.4 using registration method 1. The arrows indicate that images 3.1, 3.2, 3.3, and 3.5 are the registered medical images, and image 3.4 is the reference medical image. Image 2.4 is registered to image 1.3 using registration method 2, and image 3.4 is registered to image 1.3 using manual registration. The arrows indicate that in these two registration relationships, images 2.4 and 3.4 are the registered medical images, and image 1.3 is the reference medical image.

[0061] In some embodiments, a user (e.g., a doctor) can simultaneously select and open multiple medical images of interest in the browsing interface, wherein these medical images are already registered and aligned with each other. In some embodiments, the user can browse these medical images in the browsing interface and perform diagnostic operations based on these images. In some embodiments, during browsing, the user can open a registration relationship visualization interface to adjust these medical images.

[0062] In some embodiments, the registration between a reference medical image and a medical image to be registered may include coarse registration and fine registration. Coarse registration is performed based on the original medical image, and fine registration is performed based on the result of coarse registration.

[0063] In some embodiments, coarse registration may include coarse registration within and between groups.

[0064] In some embodiments, coarse registration within a group may include preliminary registration based on the coordinate positions of a reference medical image and the medical image to be registered within the same group, yielding an intra-group registration result. Specifically, a coordinate transformation matrix between the coordinate system of the medical image to the coordinate system of the reference medical image can be obtained using a third-party coordinate system and the coordinates of a common point between the third-party coordinate system, the coordinate system of the reference medical image, and the coordinate system of the medical image to be registered, serving as the intra-group registration result. For example... Figure 4 Registration method 1 can be an intra-group preliminary registration based on the coordinate positions of the reference medical image and the medical image to be registered within the same group.

[0065] As an example, assume OXYZ represents the patient coordinate system, o1x1y1z1 represents the coordinate system of the first image V1 scanned simultaneously, and o2x2y2z2 represents the coordinate system of the second image V2 scanned simultaneously. V1 can be the registered medical image, and V2 can be the reference medical image. Because it's a simultaneous scan, V1 and V2 not only reside in their own coordinate systems but also belong to the larger patient coordinate system OXYZ. Therefore, a point P in OXYZ can be denoted as point P1 in o1x1y1z1 and point P2 in o2x2y2z2. The shared coordinate system OXYZ can be used to correlate the positions of P1 and P2. In some embodiments, let M1 be the coordinate transformation matrix for transforming the o1x1y1z1 coordinate system to the OXYZ coordinate system, and M2 be the coordinate transformation matrix for transforming the o2x2y2z2 coordinate system to the OXYZ coordinate system. Then, the correlation formula for P1 and P2 can be shown as follows:

[0066]

[0067] in R1 = [o1x1; o1y1; o1z1], and so on. R2 = [o2x2;o2y2;o2z2].

[0068] In some embodiments, based on formula (1), the preliminary registration relationship between V1 and V2 can be shown in the following formula:

[0069]

[0070] Where T0 represents the coordinate transformation matrix between the coordinate system o1x1y1z1 where V1 is located and the coordinate system o2x2y2z2 where V2 is located; the meanings of M1 and M2 are the same as the parameters in formula (1).

[0071] In some embodiments, coarse registration between groups may include preliminary registration between groups based on key features (e.g., bounding boxes, lines, points, etc.) extracted from the reference medical image and the medical image to be registered, as well as the intra-group registration results, to obtain coarse registration results. Taking key features as key points as an example, specifically, key points can be extracted from the reference medical image and the medical image to be registered through various methods (e.g., deep learning models, etc.), and the common points of these key points in the reference medical image and the medical image to be registered can be obtained. Then, based on the intra-group coarse registration results, the coordinate transformation matrix from the medical image to the reference medical image is obtained according to the coordinates of these common points, as the coarse registration result.

[0072] Taking two images, 1 and 2, as examples, we will illustrate key feature matching. Assume that the point set extracted from image 1 is S1 = {q1, q2, ..., q...} mThe point set extracted from image 2 is S2={q1',q2',…,q n Because of differences in the scanning field of view (e.g., one includes the head, the other does not), different scanning times, different modalities, or other reasons, the points contained in S1 and S2 may be different. Points in S1 and S2 other than common points can be deleted, retaining only the common points. Common points represent the same body position of the scanned object (e.g., the center point of the left eyeball, etc.). Assume S1 and S2 contain k (k <= min(m,n)) common points, i.e., q1, q2, ..., q... k and q1',q2',…,q k ', then the points retained are q1,q2,…,q k and q1',q2',…,q k In some embodiments, the parameters for solving the registration relationship of points (i.e., the coordinate transformation matrix of common points in two different coordinate systems) may include translation, rotation, shearing, etc. Let the registration relationship of points be denoted as T0', then T0' can be calculated according to the following formula:

[0073]

[0074] in, (x i ,y i ,z i Let q1, q2, ..., q be points in S1. k The coordinates, (x i ',y i ',z i Let ') be points q1', q2', ..., q in S2. k The coordinates of ', where i is a numeric subscript, 1 <= i <= k.

[0075] In some embodiments, fine registration may include fine registration between groups. In some embodiments, precise registration may be performed based on the similarity between the reference medical image and the medical image to be registered, as well as the coarse registration result, to obtain the target registration result. The target registration result represents the spatial transformation relationship between the reference medical image and the medical image to be registered during the registration process, and may include coordinate transformation matrices, etc. For example, Figure 4 Registration method 2 in the text can be preliminary inter-group registration based on key points extracted from the reference medical image and the medical image to be registered, as well as intra-group registration results, and / or precise registration based on the similarity between the reference medical image and the medical image to be registered, as well as coarse registration results.

[0076] As an example only, for images I1 and I2, let the registration matrix for coarse registration be T0, and the registration matrix for precise registration be T1. Let I(T) represent the spatial transformation of image I using the transformation matrix T. Let S be the measure of similarity between the two images. Then the optimization process (i.e., the precise registration process) can be represented by the following formula:

[0077]

[0078] Here, argmin f(x) represents the variable value that minimizes the objective function f(x); T is equivalent to the variable x, representing the coordinate transformation matrix; S(I1,I2(T*T0)) calculates the similarity between two images, equivalent to the objective function f(x), meaning that the reference medical image I1 remains unchanged, and T*T0 is used to perform coordinate transformation (i.e., registration) on the medical image I2 to be registered; T1 represents the value of T that minimizes S(I1,I2(T*T0). The optimization process is to find the optimal T1 so that the similarity between the adjusted I1 and I2 meets the requirements, for example, the best similarity, that is, I1 and I2 are most similar.

[0079] In some embodiments, a target registration relationship can be determined based on coarse and fine registration relationships, wherein the target registration relationship represents the spatial transformation relationship between the medical image to be registered and the reference medical image during the registration process. In some embodiments, the target registration relationship may include a registration matrix. The target registration relationship is denoted as T. final Let the fine registration relationship be denoted as T1 and the coarse registration relationship as T0. Then the target registration relationship can be expressed by the following formula:

[0080] T final =T1*T0 (5)

[0081] In some embodiments of this specification, by grouping medical images of multiple modalities and multiple examinations and visualizing their registration relationships, users can intuitively and clearly obtain the registration relationships of complex and large medical images, which greatly facilitates registration management, is very user-friendly, and improves the efficiency of image registration.

[0082] In some embodiments, step 320 may be followed by step 330, in which step 330 responds to user operations on the registration relationship visualization interface and adjusts the registration relationship between multiple medical images based on these operations.

[0083] Step 330: Obtain the user's operations on the registration relationship visualization interface, and adjust the registration relationship between multiple medical images based on the operations. In some embodiments, step 330 may be performed by the adjustment module 230.

[0084] In some embodiments, based on the registration relationships of multiple medical images displayed on the registration relationship visualization interface, the adjustment module 230 can acquire user operations on the registration relationship visualization interface and adjust the registration relationships between these medical images based on the operations. In some embodiments, user operations may include modifying the registration method, deleting registration relationships, and adding registration relationships.

[0085] In some embodiments, user actions may include clicking, long-pressing, dragging, etc. In some embodiments, user actions may be implemented through various methods such as touching a touchscreen, clicking a mouse, etc.

[0086] In some embodiments, when a user changes the registration method between a reference medical image and a medical image to a second registration method, the adjustment module 230 can register the reference medical image and the medical image to be registered based on the second registration method, and visually display the registration relationship between the reference medical image and the medical image to be registered as the second registration method. The first registration method refers to the original registration method in the registration relationship between the reference medical image and the medical image to be registered, and the second registration method is a registration method different from the first registration method.

[0087] In some embodiments, in the registration relationship visualization interface, the user can click on the connection line; select the method to be changed, wherein the change can be deleting the connection, modifying the connection, and modifying the connection can be further subdivided into specific registration methods; the processing device (e.g., processing device 120) executes the selected registration method or operation; and finally the processing device updates the connection line on the interface.

[0088] As an example only, visually modifying the registration relationship can be done as follows: Figure 5 As shown, where, Figure 5 The registration relationship shown is based on Figure 4 The registration relationship in the code. For example... Figure 5 As shown in the left image, the original registration method between Image 1.1 and Image 1.3 is Registration Method 1. Users may find the result unsatisfactory and wish to modify it. Users can then perform the following operations: Click the connecting line between Image 1.1 and Image 1.3 to change the old Registration Method 1 to Registration Method 2. After clicking the connecting line, users can access the following options: Delete Connection, Modify Connection - Registration Method 1 (not selectable as Registration Method 1 is currently in use), Modify Connection - Registration Method 2, Modify Connection - Manual Registration. Figure 5If the user selects "Modify Connection - Registration Method 2", the processing device will execute the following steps: Activate Registration Method 2 to perform registration on images 1.1 and 1.3. The processing device will automatically perform registration method 2 on images 1.1 and 1.3. Then, the processing device will update the interface: after registration is complete, update the registration connection lines, that is, update the connection lines in the interface to the connection lines of registration method 2, resulting in the following... Figure 5 The registration relationship is shown in the image on the right.

[0089] In some embodiments, when a user deletes the existing registration relationship between the reference medical image and the registered medical image, the adjustment module 230 can update the registration relationship between the reference medical image and the registered medical image and visually display the updated registration relationship between the reference medical image and the registered medical image.

[0090] As an example only, visualizing the deletion of registration relationships can be done as follows: Figure 6 As shown, where, Figure 6 The registration relationship shown is based on Figure 4 The registration relationship in the code. For example... Figure 6 As shown in the left image, there is a registration relationship between image 1.1 and image 1.3. If the user does not need the registration between image 1.1 and image 1.3 and decides to delete the registration relationship, the following operation can be performed: Click the connecting line between image 1.1 and image 1.3 to delete the connection. The user can click the connecting line to open the operable options and select "Delete Connection". Then the processing device can perform the following steps: delete the registration relationship between image 1.1 and 1.3 maintained in the background. The processing device can delete the stored registration relationship between image 1.1 and image 1.3. Then the processing device can update the interface: delete the registration relationship between image 1.1 and image 1.3, that is, delete the connecting line between image 1.1 and image 1.3, to obtain the following result. Figure 6 The registration relationship is shown in the image on the right.

[0091] In some embodiments, when a user adds a registration relationship between a reference medical image and a medical image to be registered, the adjustment module 230 can register the reference medical image and the medical image to be registered based on the selected registration method, and visually display the registration relationship between the reference medical image and the medical image to be registered.

[0092] In some embodiments, if a user needs to directly register two unregistered images, i.e., add a direct registration line, the user can select any two images in the registration relationship visualization interface; open the add option, which may include optional registration methods; select the desired registration method; the processing device (e.g., processing device 120) executes the selected registration method; after registration is completed, the processing device connects the two images with the corresponding connection line.

[0093] As an example only, visualizing the registration relationship can be done as follows: Figure 7 As shown, where, Figure 7 The registration relationship shown is based on Figure 6 The registration relationship of the right-hand image. For example... Figure 7 As shown in the left image, there is currently no registration relationship between image 1.1 and image 1.3. If the user wishes to register images 1.1 and 1.3, they can perform the following steps: Select images 1.1 and 1.3, then add a connection – manual registration. A manual registration window will pop up, allowing the user to perform manual registration. After the user completes the manual registration, the processing device can update the interface by connecting 1.1 and 1.3 with a manual registration line, i.e., adding a connection line between images 1.1 and 1.3 to the interface.

[0094] It should be noted that the above description of process 300 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 300 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, if all images are not registered, the interface visualized in step 320 may only include the grouping results of the images.

[0095] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) By grouping a large number of images (e.g., different modalities, different times, different beds, etc.) based on the registration relationship, the registration-related operations can be performed directly based on the visualized registration relationship diagram. The process is clear and the operation is convenient, which facilitates the management of registration relationships between large-scale images. This solves the problem of poor image management caused by excessive registration times, improves the convenience of registration for users of large-scale images, and allows for easy adjustment of registration relationships to meet the needs of different users, ensuring the diagnostic and treatment effects of users based on registered images; (2) By visualizing the registration relationship and related registration operations, the human-computer interaction is improved, the user experience is enhanced, the user operation is facilitated, the user's learning cost is reduced, and the user can quickly master the registration operation. It should be noted that the beneficial effects that may be produced by different embodiments are different. In different embodiments, the beneficial effects that may be produced can be any one or a combination of the above, or any other possible beneficial effects.

[0096] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0097] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0098] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0099] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0100] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0101] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0102] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for visualizing and registering medical images, comprising: Acquire multiple medical images and divide the multiple medical images into at least one group according to the registration relationship between the multiple medical images; Obtain the registration relationship between the plurality of medical images in the at least one group, and visualize the registration relationship; The registration relationship visualization includes visualizing the registration method and / or registration direction between the reference medical image and the registered medical image in the at least one set of multiple medical images. The registration method includes at least one of coordinate alignment registration, registration using a registration algorithm, and manual registration. The registration direction indicates the registration between the reference medical image and the registered medical image. The registration between the reference medical image and the registered medical image refers to the registration from the registered medical image to the reference medical image. The reference medical image and the registered medical image are connected by a registration connection line, indicating the registration method and / or the registration direction between the reference medical image and the registered medical image.

2. The method of claim 1, further comprising: The system acquires user actions on the registration relationship visualization interface and adjusts the registration relationship between the multiple medical images based on these actions, wherein the actions include modifying the registration method.

3. The method of claim 1, further comprising: The system acquires user actions on the registration relationship visualization interface and adjusts the registration relationship between the multiple medical images based on these actions, wherein the actions include deleting registration relationships.

4. The method of claim 1, further comprising: The user's actions on the registration relationship visualization interface are obtained, and the registration relationship between the multiple medical images is adjusted based on the actions, wherein the actions include adding registration relationships.

5. The method according to any one of claims 2-4, further comprising: Respond to user selection of multiple registered medical images in the browsing interface; Display the multiple registered medical images; Open the registration relationship visualization interface and adjust the registration relationship between the multiple registered medical images.

6. The method of claim 1, wherein the registration between the reference medical image and the registered medical image includes coarse registration and fine registration.

7. A system for visualizing and registering medical images, comprising a group acquisition module and a visualization module; The group acquisition module is used to acquire multiple medical images and divide the multiple medical images into at least one group according to the registration relationship between the multiple medical images. The visualization module is used to obtain the registration relationship between the at least one set of the plurality of medical images and visualize the registration relationship; the visualization of the registration relationship includes visualizing the registration method and / or registration direction between the reference medical image and the registered medical image in the at least one set of the plurality of medical images, the registration method includes at least one of coordinate alignment registration, registration using a registration algorithm, and manual registration, the registration direction indicates the registration between the reference medical image and the registered medical image, the registration between the reference medical image and the registered medical image refers to the registration from the registered medical image to the reference medical image, the registration connection line connects the reference medical image and the registered medical image, and indicates the registration method and / or the registration direction between the reference medical image and the registered medical image.

8. The system of claim 7, further comprising an adjustment module; The adjustment module is used to acquire user operations on the registration relationship visualization interface and adjust the registration relationship between the multiple medical images based on the operations; the operations include modifying the registration method, deleting registration relationships, and adding registration relationships.

9. An apparatus for visualizing and registering medical images, comprising a processor for performing the method for visualizing and registering medical images according to any one of claims 1 to 6.

10. A computer-readable storage medium storing computer instructions, wherein when a computer reads the computer instructions in the storage medium, the computer executes the method for visual registration of medical images as described in any one of claims 1 to 6.

Images