Medical image registration method and related device

Abstract

The invention discloses a medical image registration method and a related device. The method comprises the following steps: acquiring a first medical image and a second medical image to be mutually registered; identifying a skeleton mark existing in the first medical image, and identifying a skeleton mark existing in the second medical image; same skeleton marks existing in the two images are determined; based on a hierarchical index of an image slice to which the same skeleton mark belongs in the first medical image, a z-direction pixel size of the first medical image, a hierarchical index of an image slice to which the same skeleton mark belongs in the second medical image, and a z-direction pixel size of the second medical image, obtaining a second skeleton mark; obtaining a z-direction offset between the first medical image and the second medical image; and registering the first medical image and the second medical image based on the z-direction offset. According to the technical scheme, the medical image registration precision can be improved, the registration effect is improved, and a more effective auxiliary effect is provided for film reading and clinical work of doctors.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a medical image registration method and related apparatus. Background Technology

[0002] In the diagnosis and treatment of medical diseases, patients often require long-term, regular follow-up. For example, when reviewing medical images of patients who have undergone cancer treatment, doctors often need to compare and analyze the current images with those from previous examinations to assess changes in the patient's condition or the effectiveness of the treatment. To achieve accurate comparison, medical images of the same patient from different periods need to be registered. Through registration, the algorithm automatically locates the corresponding position in previous medical images when the current time series is viewed, allowing doctors to easily compare information about the patient's lesions.

[0003] However, although the registration technology currently used in clinical practice can provide relatively comprehensive spatial information, it has high computational complexity and long processing time, which affects the efficiency of doctors in reading images and poses certain challenges to doctors' clinical work.

[0004] Current registration techniques primarily rely on aligning the center points of two medical images. However, the two images to be registered may differ in their scanning ranges. Therefore, center-point-based registration methods struggle to guarantee high-precision registration. In summary, existing medical image registration techniques suffer from low registration accuracy, long processing times, and high complexity. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a medical image registration method and related apparatus, aiming to improve the registration accuracy of medical images, reduce the computational complexity of registration, and shorten the registration time.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] The first aspect of this application provides a method for registering medical images, the method comprising:

[0008] Acquire the first and second medical images to be mutually registered;

[0009] Identify skeletal markers present in the first medical image and identify skeletal markers present in the second medical image;

[0010] Identify the same bone markers present in the first medical image and the second medical image;

[0011] Based on the hierarchical index of the image slice to which the same bone marker belongs in the first medical image, the pixel size in the z-direction of the first medical image, the hierarchical index of the image slice to which the same bone marker belongs in the second medical image, and the pixel size in the z-direction of the second medical image, the z-direction offset between the first medical image and the second medical image is obtained; the z-direction is perpendicular to the image slices in the first medical image and the second medical image.

[0012] The first medical image and the second medical image are registered based on the z-direction offset.

[0013] In an optional implementation, before registering the first medical image and the second medical image, the method further includes:

[0014] Obtain the x-direction offset between the first medical image and the second medical image, and the y-direction offset between the first medical image and the second medical image;

[0015] Based on the z-direction offset, the x-direction offset, and the y-direction offset, a first registration matrix is ​​constructed between the first medical image and the second medical image;

[0016] The registration of the first medical image and the second medical image based on the z-direction offset is specifically as follows:

[0017] The first medical image and the second medical image are registered based on the first registration matrix; the z-direction offset in the first registration matrix is ​​used to register the first medical image and the second medical image in the z-direction; the x-direction offset and the y-direction offset in the first registration matrix are used together to register the image center points of the image slices of the first medical image and the second medical image.

[0018] In an optional implementation, after constructing the first registration matrix between the first medical image and the second medical image, the method further includes:

[0019] By adjusting the translation and rotation parameters between the first and second medical images, the mutual information between the first and second medical images is iteratively calculated. The translation and rotation parameters that maximize the mutual information are then determined as the target translation and target rotation parameters, respectively. The target translation parameters include x-direction translation parameters, y-direction translation parameters, and z-direction translation parameters. The target rotation parameters include rotation angles around the x-axis, y-axis, and z-axis. The registration of the first and second medical images based on the first registration matrix includes:

[0020] Based on the target translation parameters and the target rotation parameters, a mathematical transformation is performed on the first registration matrix to obtain a second registration matrix;

[0021] The first medical image and the second medical image are registered based on the second registration matrix.

[0022] In the optional implementation, the method for identifying skeletal markers present in the target medical image is as follows:

[0023] Image preprocessing is performed on each layer of the target medical image to obtain processed image slices of each layer of the target medical image;

[0024] The image slices processed by each layer are input into the skeleton marker recognition model. The skeleton marker recognition model performs image analysis on the image slices processed by each layer to obtain the initial regression values ​​corresponding to the image slices processed by each layer output by the skeleton marker recognition model.

[0025] Based on the correspondence between regression values ​​and skeletal markers, the skeletal markers present in the target medical image are determined.

[0026] In an optional implementation, after obtaining the initial regression values ​​corresponding to the processed image slices output by the skeletal marker recognition model, the method further includes:

[0027] Based on the average regression value corresponding to the first bone marker and the average regression value corresponding to the second bone marker, the initial regression values ​​corresponding to the processed image slices of each layer are linearly transformed to a preset numerical range to obtain the new regression values ​​corresponding to the processed image slices of each layer; the first bone marker is the uppermost bone marker representing the body part among a plurality of predefined bone markers, and the second bone marker is the lowermost bone marker representing the body part among the plurality of predefined bone markers.

[0028] Each regression value in the correspondence is specifically obtained by transforming the regression values ​​output by the model through the linear transformation.

[0029] In an optional implementation, after obtaining the new regression values ​​corresponding to the image slices after each layer of processing, the method further includes:

[0030] Data points are constructed based on the new regression values ​​and hierarchical indices corresponding to the image slices after each layer of processing, and a straight line is fitted based on the constructed data points.

[0031] Based on the degree of deviation of each data point from the straight line, abnormal data points are filtered out;

[0032] Calculate the maximum and minimum values ​​among the new regression values ​​of the retained data points, and determine the new regression values ​​of the retained data points that are between the minimum and maximum values;

[0033] The determination of skeletal markers present in the target medical image based on the correspondence between regression values ​​and skeletal markers includes:

[0034] From the correspondence, determine the closest regression value for each new regression value between the minimum and maximum values;

[0035] Based on the correspondence, the skeletal marker corresponding to the closest regression value is determined.

[0036] In the optional implementation, the training process of the skeletal marker recognition model includes:

[0037] Obtain a first training dataset and a second training dataset; the first training dataset contains multiple preprocessed 3D medical images without bone markers, and the second training dataset contains multiple preprocessed 3D medical images with bone markers.

[0038] In the first training phase, the three-dimensional medical images in the first training dataset are input into the initial model to be trained. The initial model to be trained performs image analysis on each layer of the input three-dimensional medical images, and outputs the initial regression values ​​of each layer of the input three-dimensional medical images. After the first training phase is completed, the skeleton marker recognition model is obtained.

[0039] In the second training phase, the three-dimensional medical images in the second training dataset are input into the bone marker recognition model. The bone marker recognition model performs image analysis on each layer of the input three-dimensional medical images to obtain the initial regression values ​​output by the bone marker recognition model for multi-layer image slices with the same bone marker. Based on multiple initial regression values ​​of the same bone marker, the average regression value corresponding to the bone marker is calculated. The correspondence between different bone markers and the average regression value is recorded in the bone marker recognition model.

[0040] Among the optional implementation methods, medical image registration methods also include:

[0041] Based on the average regression value corresponding to the first bone marker and the average regression value corresponding to the second bone marker, the calculated average regression value corresponding to each bone marker is linearly transformed to a preset numerical range, and the correspondence between the new linearly transformed regression value and each bone marker is constructed and recorded in the bone marker recognition model.

[0042] In an optional implementation, obtaining the x-direction offset between the first medical image and the second medical image, and the y-direction offset between the first medical image and the second medical image, includes:

[0043] The x-direction offset between the first medical image and the second medical image is calculated based on their respective x-direction dimensions and x-direction pixel sizes.

[0044] The y-direction offset between the first medical image and the second medical image is calculated based on their respective y-direction dimensions and y-direction pixel sizes.

[0045] In the optional implementation, the mutual information is calculated as follows:

[0046] The gray-level histogram of the first medical image, the gray-level histogram of the second medical image, and the joint gray-level histogram of the first and second medical images are calculated using the coordinate increment method.

[0047] The gray-level probability distribution of the first medical image is obtained based on the gray-level histogram of the first medical image, the gray-level probability distribution of the second medical image is obtained based on the gray-level histogram of the second medical image, and the joint gray-level probability distribution of the first medical image and the second medical image is obtained based on the joint gray-level histogram.

[0048] The entropy of the first medical image is calculated based on the gray-level probability distribution of the first medical image, the entropy of the second medical image is calculated based on the gray-level probability distribution of the second medical image, and the joint entropy of the first medical image and the second medical image is calculated based on the joint gray-level probability distribution.

[0049] The mutual information between the first medical image and the second medical image is calculated based on the entropy of the first medical image, the entropy of the second medical image, and the joint entropy.

[0050] A second aspect of this application provides a medical image registration device, the device comprising:

[0051] The image acquisition module is used to acquire the first and second medical images to be mutually registered.

[0052] A marker recognition module is used to identify bone markers present in the first medical image and to identify bone markers present in the second medical image;

[0053] A marker determination module is used to determine the same bone markers present in the first medical image and the second medical image;

[0054] The offset acquisition module is used to obtain the z-direction offset between the first medical image and the second medical image based on the layer index of the image slice to which the same bone marker belongs in the first medical image, the z-direction pixel size of the first medical image, the layer index of the image slice to which the same bone marker belongs in the second medical image, and the z-direction pixel size of the second medical image; the z-direction is perpendicular to the image slices in the first medical image and the second medical image.

[0055] The image registration module is used to register the first medical image and the second medical image based on the z-direction offset.

[0056] A third aspect of this application provides a medical image registration device, the device comprising: a processor and a memory communicatively connected to each other;

[0057] The memory stores computer programs;

[0058] The processor is configured to run the computer program to implement the medical image registration method as described in any implementation of the first aspect.

[0059] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when processed, implements the steps of the medical image registration method as described in any implementation of the first aspect.

[0060] Compared with the prior art, this application has the following beneficial effects:

[0061] This application proposes to acquire a first medical image and a second medical image to be registered; identify skeletal markers in the first medical image and the second medical image; determine the same skeletal markers in the first and second medical images; obtain the z-direction offset between the first and second medical images based on the layer index of the image slice to which the same skeletal marker belongs in the first medical image, the z-direction pixel size of the first medical image, the layer index of the image slice to which the same skeletal marker belongs in the second medical image, and the z-direction pixel size of the second medical image; the z-direction is perpendicular to the image slices in the first and second medical images; and register the first and second medical images based on the z-direction offset. In this application, by identifying the same skeletal markers in the two medical images to be registered, registration of the two medical images in the z-direction can be achieved. Even if the scanning ranges of the two medical images are different, since the image registration operation of this application does not completely depend on the image center point, but is based on the same skeletal markers and uses body parts as the registration guide, the registration accuracy is higher than that of the prior art. Furthermore, the technical solution of this application uses skeletal markers as the basis for registration, which reduces the complexity of registration calculations and also solves the problem of long registration time. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart illustrating a medical image registration method provided in this application embodiment;

[0064] Figure 2 This is a flowchart illustrating a preprocessing procedure for image slices in an embodiment of this application.

[0065] Figure 3 A flowchart illustrating another medical image registration method provided in this application embodiment;

[0066] Figure 4 A flowchart illustrating the process of obtaining the second registration matrix through coarse and fine registration;

[0067] Figure 5 A schematic diagram of fitting a straight line to data points constructed based on the new regression values ​​and hierarchical indices of image slices;

[0068] Figure 6 This is a schematic diagram of the data structure of a training dataset provided in an embodiment of this application;

[0069] Figure 7 This is a schematic diagram of a medical image registration method provided in an embodiment of this application. Detailed Implementation

[0070] As described earlier, current medical image registration techniques rely on the image center point as a coarse registration method. However, if the two medical images have different scanning ranges—for example, one scans the upper part of the body and the other the lower part, or one has a larger scanning range and the other a smaller one—center-point-based registration is likely to result in low registration accuracy. Furthermore, existing registration techniques are computationally complex and time-consuming, impacting registration and image interpretation efficiency.

[0071] To address the above problems, the inventors have proposed a new solution. This application provides a medical image registration method and related apparatus, which achieves registration between medical images in the z-direction based on common skeletal markers in the images. Since the registration in this solution does not entirely depend on the image center point, even if the scanning ranges of the two images being registered differ, the consistency of body parts can be accurately identified, thereby improving registration accuracy. Furthermore, it also solves the problems of high computational complexity and long processing time.

[0072] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0073] See Figure 1 This figure is a flowchart of a medical image registration method provided in an embodiment of this application. Figure 1 As shown, the registration methods for medical images include:

[0074] S11. Acquire the first and second medical images to be registered with each other.

[0075] In this embodiment, the registration process is illustrated using pairwise registration of medical images as an example. The two images to be registered are exemplified as a first medical image and a second medical image, where both originate from the same individual. For instance, the first medical image is a medical image taken by patient A at a first moment, and the second medical image is a medical image taken by patient A at a second moment, with the first moment preceding the second moment. As an example, the first moment could be the imaging time when patient A seeks medical attention one month after undergoing a certain medical treatment, and the second moment could be the imaging time when patient A seeks medical attention six months after undergoing the same medical treatment.

[0076] The first and second medical images use the same imaging techniques. For example, both the first and second medical images are MRI images. Or, for example, both the first and second medical images are CT images.

[0077] In this embodiment, both the first medical image and the second medical image are three-dimensional data structures. Each of the first and second medical images has multiple layers of image slices arranged in the z-direction, and each image slice is a two-dimensional image. The data of the two-dimensional image is arranged along the x and y directions.

[0078] S12. Identify skeletal markers present in the first medical image and identify skeletal markers present in the second medical image.

[0079] In an optional implementation, the recognition work in step S12 is performed using a pre-trained skeletal marker recognition model. For ease of explanation, "target medical image" refers to the medical image requiring skeletal marker recognition; that is, the target medical image can be either the first medical image or the second medical image in step S12. Since the method for recognizing skeletal markers in the first and second medical images is the same, for the sake of brevity, an exemplary method for recognizing skeletal markers in the target medical image is described below:

[0080] First, image preprocessing is performed on each layer of the target medical image to obtain processed image slices of each layer of the target medical image.

[0081] In the embodiments of this application, CT images are used as an example of medical images. The preprocessing of image slices may include one or more of the following: data cleaning, image resampling, CT value cropping and normalization, filtering, image cropping or padding.

[0082] In the specific implementation of this application, data cleaning mainly involves removing image slices whose relationship with the body trunk is difficult to determine. This is primarily because the skeletal marker recognition model learns based on the relative positions of the body. Some images are isolated from the trunk, such as CT images of the upper limbs (wrist, elbow, etc.), whose relationship with the trunk is difficult to determine and have limited value in training the skeletal marker recognition model and in medical image registration. Therefore, they can be removed through cleaning to reduce the amount of image data that needs to be processed. In addition, for CT images with a small shooting range, such as coronary CT images, although the relationship with the trunk can be determined, the small shooting range and the large difference from images of nearby body parts make it difficult for the training process to converge, for example, when used in model training. Therefore, these types of images are removed from the dataset.

[0083] Because the pixel spacing of different CT image data can vary significantly, to ensure that the image data input into the skeleton marker recognition model has the same physical meaning, it is necessary to resample these image slices to a uniform pixel spacing when necessary. As an example, the target pixel spacing (target_spacing) for resampling is 7mm. If the size of the resampled image is not 64 pixels × 64 pixels, it can be processed to 64 pixels × 64 pixels using zero-padding or center cropping. This ensures that the resampled pixel spacing guarantees a final field of view of 44.8cm × 44.8cm. Verification has shown that this field of view can cover the entire cross-section of a typical human body.

[0084] When cropping CT values, the CT values ​​of the image slices are cropped to between -1000 HU and 1500 HU. Afterwards, they are normalized to between -1 and 1.

[0085] In practical applications, a Gaussian filter can be used to filter the image slices obtained after the aforementioned processing, thereby avoiding aliasing artifacts. In the example, for the downsampling factor a = 0.25, visual inspection was performed after resampling, and the experiment determined that the smoothing standard deviation σ of the Gaussian kernel was 0.8. This Gaussian filter configuration causes only a small loss of edge information while minimizing aliasing artifacts, thus it is relatively reliable. The downsampling factor a and the smoothing standard deviation σ of the Gaussian kernel can be regarded as reference values ​​for the downsampling factor and the smoothing standard deviation of the Gaussian kernel, respectively, when obtaining better filtering results. In practical applications, different downsampling factors a' can be obtained based on the original pixel spacing (origin_spacing) of the image slices. The expression for the downsampling factor a' is:

[0086]

[0087] Then, based on the downsampling factor, the smoothing standard deviation σ' of the Gaussian kernel that should be configured for the Gaussian filter can be obtained. See the formula below:

[0088]

[0089] Figure 2 This is a flowchart illustrating a preprocessing procedure for image slices in an embodiment of this application. Figure 2 This demonstrates a preprocessing execution order that differs from those described above. Figure 2 The example first performs data cleaning, followed by resampling, CT value cropping and normalization, and Gaussian blurring. Subsequently, center cropping or surrounding padding is performed based on the difference between the pixel size of the image slice and the preset size. During resampling, each layer of the target medical image is resampled at a target pixel spacing of 7mm, the CT value is cropped to between -1000HU and 1500HU, and then normalized to between -1 and 1. Next, Gaussian filtering is used to remove image artifacts. Finally, if the pixel size of the image slice in the x or y direction is greater than 64 pixels, center cropping is performed; otherwise, zero padding is performed, making the image slice 64 pixels × 64 pixels in the x and y directions. The specific order of the preprocessing operations for image slices is not limited in this application. Since the images used in the training phase of the skeletal marker recognition model are also preprocessed using the above methods, processing the image slices in the target medical image in this way during the model inference phase (i.e., the skeletal marker recognition phase) ensures the correspondence between the input data in the model inference phase and the training data used in the model training phase. This allows the input data to be adapted to the skeletal marker recognition model, which is beneficial to ensuring the accuracy of skeletal marker recognition and the precision of medical image registration.

[0090] Next, the processed image slices are input into the skeletal landmark recognition model. The model analyzes each slice to obtain initial regression values ​​for each layer. For example, if a target medical image, after preprocessing, is input into the model with 30 processed image slices, the model can analyze each of these 30 slices individually, determining the regression value for each slice. The regression values ​​obtained directly from the model's calculations are referred to as initial regression values. In practical applications, the model pre-records the correspondence between regression values ​​and skeletal landmarks. Therefore, after obtaining the initial regression values, this correspondence can be used to determine which skeletal landmark corresponds to each image slice. This allows for the identification of all skeletal landmarks present in the target medical image.

[0091] Following the implementation process of S12 described above, this application can determine each skeletal marker present in the first medical image and each skeletal marker present in the second medical image.

[0092] As an example, the skeletal landmarks identified from the first medical image include: [C5,C6,C7,T1,T2,T3,T4,T5,T6,T7,T8,T9,T10], and the skeletal landmarks identified from the second medical image include: [C7,T1,T2,T3,T4,T5,T6,T7,T8,T9,T10,T11,T12,L1]. Each skeletal landmark in [ ] has a corresponding meaning, as described in Tables 1-1 and 1-2.

[0093] Table 1-1 Description of Skeletal Markers, Part 1

[0094]

[0095] Table 1-2 Skeletal Marker Description Part 2

[0096]

[0097] It should be noted that both Tables 1-1 and 1-2 present the descriptions of the skeletal markers in order of body part from top to bottom. In Table 1-1, head_end is the uppermost skeletal marker representing the body part in both Tables 1-1 and 1-2, which can be called the first skeletal marker. In Table 1-2, tibia-start is the lowermost skeletal marker representing the body part in both Tables 1-1 and 1-2, which can be called the second skeletal marker. Furthermore, the bottommost skeletal marker T12 in Table 1-1 and the topmost skeletal marker L1 in Table 1-2 are two adjacent skeletal markers among a predefined set of markers. T12 is closer to the first skeletal marker than L1, meaning that the body part represented by T12 is higher than the body part represented by L1.

[0098] S13. Identify the same bone markers present in the first and second medical images.

[0099] Based on the examples of skeletal markers identified from the first and second medical images above, it can be determined that the first and second medical images share 11 identical skeletal markers, including: [C7, T1, T2, T3, T4, T5, T6, T7, T8, T9, T10]. Finding these identical skeletal markers in the first and second medical images facilitates accurate registration in the medical image registration method proposed in this application.

[0100] The registration of the first and second medical images can be divided into two stages: coarse registration and fine registration. "Coarse" and "fine" are relative terms and do not imply that the coarse registration stage cannot achieve improved registration accuracy compared to existing technologies. In this embodiment, S12 to S13 represent the preliminary work for coarse registration. The execution of coarse registration also depends on S14 to S15, which will be described later. Furthermore, fine registration is a further optional optimization step and will be introduced as another embodiment after the method embodiments in S11 to S15 have been described.

[0101] S14. Based on the hierarchical index of the image slice to which the same bone marker belongs in the first medical image, the pixel size in the z-direction of the first medical image, the hierarchical index of the image slice to which the same bone marker belongs in the second medical image, and the pixel size in the z-direction of the second medical image, obtain the z-direction offset between the first medical image and the second medical image.

[0102] In order to achieve registration between medical images, this application embodiment proposes that the z-direction offset between the first medical image and the second medical image needs to be clearly defined, so that the z-direction offset can be used to complete the z-direction registration operation in the subsequent step S15.

[0103] Regarding the z-direction offset between the first and second medical images, assuming that during the registration process, the first medical image is a floating image and the second medical image is a fixed image (meaning that the first medical image is registered with the fixed second medical image by moving), the calculation method is as follows: ;

[0104] In the formula, This represents the z-direction offset between the first and second medical images. `slice1` and `slice2` represent the layer indices of the image slices containing the same bone marker in the first and second medical images, respectively. `spacingZ1` represents the pixel size in the z-direction of the first medical image, and `spacingZ2` represents the pixel size in the z-direction of the second medical image. As mentioned earlier, both the first and second medical images are three-dimensional medical images, each with multiple multi-layered image slices arranged perpendicular to the z-direction (i.e., extending along the x and y directions). Taking the same bone marker T5 in both the first and second medical images as an example, the layer index of the image slice to which T5 belongs in the first medical image can be determined. Similarly, the layer index of T5 in the second medical image can also be determined, thus obtaining the specific values ​​of `slice1` and `slice2`. Substituting these values ​​into the above formula, the z-direction offset between the first and second medical images can be calculated smoothly. .

[0105] It should be noted that since the physical scenes of the first and second medical images taken by the same individual are similar, the calculation of the z-direction offset between the first and second medical images is different. In this case, you only need to substitute slice1 and slice2 of the same bone marker. For example, you can substitute slice1 and slice2 of T5 into the formula to calculate the z-direction offset between the first and second medical images. It is not necessary to repeatedly substitute and calculate the z-direction offset multiple times. .

[0106] S15. Register the first medical image and the second medical image based on the z-direction offset.

[0107] Since the z-direction offset is calculated based on the same bone markers in the first and second medical images to be registered, the z-direction offset can represent the deviation of the same bone markers in the first and second medical images in the z-direction. Based on this, z-direction registration of the same bone markers in the first and second medical images can be achieved simply by using this z-direction offset, thus realizing the z-direction registration of the first and second medical images.

[0108] The above describes the medical image registration method according to the embodiments of this application, focusing on the coarse registration process between two medical images to be registered. The method proposes acquiring a first and a second medical image to be registered; identifying skeletal markers in the first and second medical images; determining the same skeletal markers in the first and second medical images; obtaining the z-direction offset between the first and second medical images based on the layer index of the image slice to which the same skeletal marker belongs in the first medical image, the z-direction pixel size of the first medical image, the layer index of the image slice to which the same skeletal marker belongs in the second medical image, and the z-direction pixel size of the second medical image; and registering the first and second medical images based on the z-direction offset. In this application, by identifying the same skeletal markers in the two medical images to be registered, registration of the two medical images in the z-direction can be achieved. Even if the two medical images have different scanning ranges, the image registration operation in this application does not rely on the image center point but is based on the same skeletal markers, using body parts as the registration guide. Therefore, the registration accuracy is higher than that of existing technologies. In addition, the technical solution of this application uses skeletal markers as the registration basis, which reduces the complexity of registration calculations and also solves the problem of long registration time.

[0109] The z-direction offset only determines the z-direction registration relationship between the first and second medical images based on the layer-to-layer relationship. However, in practice, how to place the corresponding layers in the z-direction requires analysis of the offsets in the x and y directions. Coarse registration can take into account the registration in the z, x, and y directions. When registering corresponding layers in the first and second medical images in the x and y directions, the center point alignment in the x and y directions can be selected, or the principal axis method can be used. This application does not limit the registration method in the x and y directions.

[0110] As an example, in practical applications, to improve registration accuracy, the offsets of the first and second medical images in the x-direction and the y-direction offset between them can be further obtained. Based on the z-direction, x-direction, and y-direction offsets, a first registration matrix between the first and second medical images is constructed. The form of the first registration matrix is ​​as follows:

[0111]

[0112] in, As described above, this represents the z-direction offset between the first and second medical images; and These represent the offsets of the first and second medical images in the x-direction and the offset in the y-direction between them, respectively. It is evident that once the offsets in these three directions are calculated, they can be used as key parameters to fill the first registration matrix above. This first registration matrix can also be considered as the coarse registration matrix in the coarse registration stage.

[0113] Assuming that during the registration process, the first medical image is a floating image (mov) and the second medical image is a fixed image (fix), this means that the first medical image is registered with the fixed second medical image by moving. Regarding the offset in the x-direction... and y-direction offset The calculation method involves determining the x-axis offset between the first and second medical images based on their respective x-axis dimensions and pixel sizes. Similarly, the y-axis offset is calculated based on the y-axis dimensions and pixel sizes of the first and second medical images. See the following formulas for details:

[0114] ;

[0115] ;

[0116] In the above formula, SizeX_mov and SizeX_fix represent the dimensions of the first and second medical images in the x-direction, respectively; SizeY_mov and SizeY_fix represent the dimensions of the first and second medical images in the y-direction, respectively. SpacingX_mov and SpacingX_fix represent the pixel size of the first and second medical images in the x-direction, respectively; SpacingY_mov and SpacingY_fix represent the pixel size of the first and second medical images in the y-direction, respectively. All eight parameters are known, thus allowing for the successful calculation of the x-direction offset. and y-direction offset .

[0117] To further improve registration accuracy, registration is performed in all three directions. Therefore, the specific implementation of step S15 in the above method embodiment can be: registering the first medical image and the second medical image based on the first registration matrix. The z-direction offset in the first registration matrix is ​​used for z-direction registration of the first and second medical images. The x-direction and y-direction offsets in the first registration matrix are used together to register the image center points of image slices of the first and second medical images. By registering the image slice center points in the x and y directions, the registration effect can be optimized, further improving the registration accuracy.

[0118] As mentioned earlier, in the optional implementation methods, fine registration can be further performed after coarse registration. The following example illustrates the registration implementation process of coarse registration + fine registration. Figure 3 A flowchart of another medical image registration method provided in this application embodiment.

[0119] S31. Acquire the first and second medical images to be registered with each other.

[0120] S32. Identify skeletal markers present in the first medical image and identify skeletal markers present in the second medical image.

[0121] S33. Identify the same bone markers present in the first and second medical images.

[0122] S34. Based on the hierarchical index of the image slice to which the same bone marker belongs in the first medical image, the pixel size in the z-direction of the first medical image, the hierarchical index of the image slice to which the same bone marker belongs in the second medical image, and the pixel size in the z-direction of the second medical image, obtain the z-direction offset between the first medical image and the second medical image.

[0123] S35. Obtain the x-direction offset between the first medical image and the second medical image, and the y-direction offset between the first medical image and the second medical image.

[0124] It should be noted that the x-direction offset and y-direction offset can be calculated later than, earlier than, or simultaneously with the z-direction offset. Therefore, S32~S35 in the embodiments of this application are merely examples and are not intended to limit the specific implementation order.

[0125] S36. Based on the offset in the z direction, the offset in the x direction, and the offset in the y direction, construct the first registration matrix between the first medical image and the second medical image.

[0126] The implementation methods of steps S31 to S36 have been described above and will not be repeated in this embodiment. Please refer to the previous descriptions for relevant details. In the following text, steps S37 to S39 focus on the fine registration operation. In this embodiment, the further fine registration of medical images mainly considers that the first and second medical images may have translational and rotational deviations, which could affect the registration effect. Therefore, this embodiment proposes to determine the translation and rotation parameters in step S37 so that the mutual information between the two medical images meets specific conditions.

[0127] S37. By adjusting the translation and rotation parameters between the first medical image and the second medical image, the mutual information between the first medical image and the second medical image is iteratively calculated, and the translation and rotation parameters that make the mutual information reach the maximum value are determined as the target translation parameters and the target rotation parameters, respectively.

[0128] In the optional implementation, the mutual information is calculated as follows:

[0129] The coordinate increment method is used to calculate the gray-level histograms of the first and second medical images, as well as the joint gray-level histogram of the first and second medical images. The gray-level probability distribution of the first medical image is obtained based on its gray-level histogram, the gray-level probability distribution of the second medical image is obtained based on its gray-level histogram, and the joint gray-level probability distribution of the first and second medical images is obtained based on the joint gray-level histogram. When calculating the gray-level probability distribution, the count of each histogram bucket is divided by the total number of sampling points, thus transforming it into a probability distribution. The total number of sampling points is:

[0130] (m_iWidth / m_iSrcScale)×(m_iHeight / m_iSrcScale)×m_iSlice;

[0131] In the formula, m_iWidth and m_iHeight represent the width and height of the image, respectively, m_iSrcScale represents the scaling ratio of the image from the original scale to the current scale, and m_iSlice represents the number of layers of the image slice in the image.

[0132] Having obtained the individual grayscale probability distributions of the first and second medical images, as well as their joint grayscale probability distribution, we can then calculate the entropy of each image, and the joint entropy of the two images. Specifically, the entropy of the first medical image is calculated based on its grayscale probability distribution, the entropy of the second medical image is calculated based on its grayscale probability distribution, and the joint entropy of the two images is calculated based on their joint grayscale probability distribution.

[0133] The entropy of the first medical image is calculated as follows:

[0134] ;

[0135] The entropy of the second medical image is calculated as follows:

[0136] ;

[0137] The method for calculating the joint entropy of the first and second medical images is as follows:

[0138] ;

[0139] In the above formula, A represents the first medical image, and B represents the second medical image; H(A) and H(B) represent the entropy of the first medical image and the entropy of the second medical image, respectively; H(A,B) represents the joint entropy. In the formula for calculating H(A), n represents the total number of gray levels in the first medical image, and P...i Let H(B) represent the probability of the i-th gray level in the gray-level probability distribution of the first medical image. In the formula for calculating H(B), n represents the total number of gray levels in the second medical image, and P... i Let H(A,B) represent the probability of the i-th gray level in the gray-level probability distribution of the second medical image. In the formula for calculating H(A,B), i represents the i-th gray level in the first medical image, j represents the j-th gray level in the second medical image, and P... AB (i,j) represents the probability of the i-th gray level in the first medical image and the j-th gray level in the second medical image in the joint gray-level probability distribution. Finally, the mutual information between the first and second medical images is calculated based on the entropy of the first medical image, the entropy of the second medical image, and the joint entropy. The formula for calculating the mutual information is as follows:

[0140] ;

[0141] The absolute value of mutual information I(A,B) reflects the correlation between the first and second medical images. A larger absolute value of I(A,B) indicates a higher correlation in the information content between the two images, which also affects the registration effect to some extent. Maximizing the absolute value of I(A,B) is crucial; I(A,B) should reach its maximum value. If the translation and rotation parameters used for the two images change, the mutual information will also change. When iteratively seeking the optimal rotation and translation parameters, the Power optimization algorithm requires the objective function to be concave and its minimum value to be found. Therefore, the value of the mutual information can be inverted when iteratively seeking the optimal rotation and translation parameters. By adjusting the translation and rotation parameters between the first and second medical images, the mutual information between them is iteratively calculated, and the translation and rotation parameters that maximize the mutual information are determined as the target translation and target rotation parameters, respectively. When the mutual information reaches its minimum value, the target translation and target rotation parameters are beneficial for achieving fine-grained registration between the two images.

[0142] The target translation parameters include translation parameters in the x, y, and z directions, and the target rotation parameters include rotation angles around the x-axis, y-axis, and z-axis. As an example, if the first medical image is a floating image and the second medical image is a fixed image, this means that the first medical image is registered with the fixed second medical image by movement. Therefore, the above target translation and rotation parameters refer to the parameters that the first medical image, as a floating image, should be registered with the second medical image.

[0143] Having obtained the target translation and rotation parameters, further adjustments can be made to the first registration matrix to apply the target translation and rotation parameters calculated for fine registration to the image registration work, see step S38.

[0144] S38. Based on the target translation parameters and target rotation parameters, perform mathematical transformations on the first registration matrix to obtain the second registration matrix.

[0145] As an example, the target translation parameters and target rotation parameters are represented as [Tx, Ty, Tz, Rx, Ry, Rz]. Where Tx, Ty, and Tz are the translation parameters in the x, y, and z directions, respectively; and Rx, Ry, and Rz are the rotation angles about the x-axis, y-axis, and z-axis, respectively. (This is related to the first registration matrix introduced earlier.) Based on this, the first registration matrix can be mathematically transformed using [Tx, Ty, Tz, Rx, Ry, Rz], and the result of the transformation is called the second transformation matrix.

[0146] S39. Register the first medical image and the second medical image based on the second registration matrix.

[0147] Based on the form of the input data for registration, two exemplary registration scenarios are provided below:

[0148] The first registration scenario: Given a point P(x,y,z,1) in a first medical image (a floating image), we need to determine the position of the corresponding point in a second medical image. The coordinates of P(x,y,z,1) in the second medical image (a fixed image) are then calculated using the second registration matrix. Where T is the second registration matrix.

[0149] In the first registration scenario, a point P(x,y,z,1) in the first medical image can be determined by a user's trigger operation on the first medical image displayed on the medical image interface. For example, if a user clicks on a point in the first medical image as point P(x,y,z,1), then after registration is completed, the position of P'(x',y',z',1) can be marked in the second medical image displayed on the interface.

[0150] The second registration scenario: A z-axis value is provided in a first medical image serving as a floating image. This value corresponds to the z-th layer image slice in the first medical image. It is necessary to determine the corresponding image slice in the second medical image that corresponds to the z-th layer image slice in the first medical image. Then, z' is calculated using the second registration matrix, and the z-th layer image slice in the first medical image corresponds to the z'-th layer image slice in the second medical image.

[0151] In the second registration scenario, the z-axis value in the first medical image can be determined by the user's trigger operation on a certain image slice in the medical image display interface. For example, if the user selects the z-th layer image slice in the first medical image, then after registration is completed, the z'-th layer image slice can be displayed in the second medical image with a color different from other layers in the display interface. Alternatively, after registration is completed, the z'-th layer image slice of the second medical image can be directly tiled and unfolded.

[0152] The above embodiments focus on the implementation process of fine registration. Figure 4 This is a flowchart illustrating the process of obtaining the second registration matrix through coarse and fine registration. (This can be combined with...) Figure 4 Please understand the description above. Furthermore, this application embodiment demonstrates the application effect of the technical solution in two different registration scenarios. By combining coarse and fine registration, a second registration matrix is ​​obtained to further improve registration accuracy, significantly improving registration efficiency and effect, facilitating doctors' image comparison, and achieving the goal of saving time and effort. In addition, the computational complexity is low, and the registration time is effectively controlled.

[0153] As mentioned in the embodiments described above, in order to identify the skeletal markers present in the first and second medical images, the image slices of each layer in the first and second medical images need to be preprocessed and then input into a pre-trained skeletal marker recognition model. The skeletal marker recognition model obtains the initial regression values ​​corresponding to each processed image slice. In other words, in this embodiment, the skeletal marker recognition model can complete the analysis and calculation from image slices to regression values. CT images can be viewed as three-dimensional medical images composed of multiple two-dimensional cross-sections (image slices) stacked along the z-direction. In the z-direction, there are body structures such as the spine that are relatively easy to locate. Therefore, by determining the location of each image slice in the human body, the approximate range of various parts of the human body can be determined. The purpose of regression values ​​and constructing regression values ​​and skeletal markers is precisely for this purpose.

[0154] To construct a more accurate correspondence between regression values ​​and skeletal markers and reduce computational and registration biases, this application further proposes that, after obtaining the initial regression values ​​corresponding to each layer of processed image slices output by the skeletal marker recognition model, the initial regression values ​​corresponding to each layer of processed image slices can be linearly transformed to a preset numerical range based on the average regression value corresponding to the first skeletal marker and the average regression value corresponding to the second skeletal marker, thereby obtaining new regression values ​​corresponding to each layer of processed image slices.

[0155] Specifically, the first skeletal marker refers to the uppermost skeletal marker representing the body part in Table 1-1, i.e., head_end, meaning the last slice where visible bones and brain tissue are found. The second skeletal marker refers to the lowermost skeletal marker representing the body part in Table 1-2, i.e., tibia-start, meaning the slice before the tibia and fibula begin to appear. The first and second skeletal markers essentially define the most valuable body part intervals for registration between medical images, or they can be understood as defining the upper and lower boundaries of several skeletal markers most valuable for registration between medical images. The average regression value (avg) corresponding to the first skeletal marker is... head_end The average regression value (avg) can be calculated by averaging the regression values ​​output by the model for different slices with the first bone marker. The average regression value (avg) corresponding to the second bone marker is... tibia_start The average regression value (avg) corresponding to the first bone marker can be calculated by averaging the regression values ​​output by the model for different slices with the second bone marker. The following formula illustrates this: head_end The average regression value avg corresponding to the second skeletal marker tibia_start Application in linear transformation of initial regression values.

[0156]

[0157] In the formula, score ori The score represents the initial regression value of an image slice calculated by the model. new This represents the new regression value of the image slice obtained through linear transformation. This transforms the regression values ​​to the range of 0 to 100. This generalized scale conversion facilitates observation. When identifying skeletal landmarks in medical images, the correspondence between the regression values ​​and the skeletal landmarks used specifically refers to the regression values ​​obtained during the model training phase, which are also obtained through a similar linear transformation process described above.

[0158] In an optional implementation, after obtaining the new regression values ​​corresponding to the image slices after each layer of processing, the method further includes:

[0159] Data points are constructed based on the new regression values ​​and hierarchical indices corresponding to the image slices after each layer of processing, and a straight line is fitted based on these data points. Outlier data points are filtered out according to the degree of deviation of each data point from the straight line. The maximum and minimum values ​​of the new regression values ​​of the retained data points are calculated, and the new regression values ​​of the retained data points that fall between the minimum and maximum values ​​are determined.

[0160] Figure 5 A schematic diagram illustrating the fitting of a straight line to data points constructed based on the new regression values ​​and hierarchical indices of the image slices. Figure 5 The horizontal axis represents the layer index of the image slice, increasing from small to large, while the vertical axis represents the new regression value of the image slice. These data points are fitted to a straight line using the least squares method. Regression values ​​that deviate excessively from this line are removed. This is because, according to the model's training logic, these new regression values ​​should normally follow the slice layer index linearly; therefore, regression values ​​that deviate too much from this line can be considered prediction errors. Figure 5 As shown in the image, the yellow dots are the data points used to fit the straight line, and the blue dots are the fitted straight line. Some yellow dots in the upper left corner deviate significantly from the blue dots; the regression values ​​of these image slices are incorrect predictions and need to be removed. The maximum value (score) in the new regression value can be determined from the remaining data points. max and minimum score min Furthermore, it can identify new regression values ​​among the retained data points that fall between the aforementioned minimum and maximum values. For these identified new regression values, the regression value closest to each new regression value can be found through a pre-constructed correspondence between regression values ​​and skeletal markers, and the corresponding skeletal marker can be identified based on the correspondence, thus completing the identification of skeletal markers in medical images.

[0161] Regarding the skeletal landmark recognition model mentioned and used earlier, the training process of this model is described below. The training process can be understood as comprising two stages: a first training stage and a second training stage. In the first training stage, self-supervised learning is used to regress body parts. The second training stage can be understood as the post-processing stage of the model. After the first training stage, the model cannot be directly applied; it can only output regression values ​​for image slices, but it cannot find skeletal landmarks corresponding to body parts. The second training stage establishes the correspondence between regression values ​​and skeletal landmarks, thereby serving the model's inference.

[0162] In the optional implementation, the initial model to be trained includes a base model (which can be VGG11), a 1×1 convolutional kernel, a global average pooling layer, and a fully connected layer. The base model takes a two-dimensional image slice as input and is primarily used to extract features from the slice. The 1×1 convolutional kernel is located at the end of the base model's output. This kernel does not change the number of channels but can fuse channel information, increasing non-linear expressive power. The global average pooling layer is located at the end of the 1×1 convolutional kernel's output and is used to calculate the average of the last two dimensions, resulting in a batch_size×512×1 feature map. For example, before the average pooling operation, the feature map dimensions should be batch_size×512×64×64, meaning the last two dimensions are 64×64. After average pooling, a batch_size×512×1 feature map is obtained. The fully connected layer is connected to the end of the global average pooling layer, with one output channel, used to predict the regression value.

[0163] In the optional implementation, the training process of the skeletal marker recognition model includes:

[0164] Obtain the first training dataset and the second training dataset. The first training dataset contains multiple preprocessed 3D medical images without skeleton markers, used for self-supervised learning in the first training phase; the second training dataset contains multiple preprocessed 3D medical images with skeleton markers, used for the second training phase. It should be noted that the preprocessing method for image slices in medical images was described earlier, and the preprocessing method for the data in the model's training dataset is similar, and will not be repeated here; please refer to the previous description.

[0165] Figure 6 This is a schematic diagram of the data structure of a training dataset provided in an embodiment of this application. For example... Figure 6 As shown, the data volume X of the three-dimensional medical images used to train the model i It contains m (e.g., m=3) equally spaced image slices (distributed along the z-direction), denoted as X. i,j X i,j+1 and X i,j+2 Here, j, j+1, and j+2 represent the hierarchical indices of the image slices. The loss function can be defined based on the model output s of this set of data and applied to the first training phase. Optionally, the loss function includes two parts: a sequential loss L... order and distance loss L dist The loss function L is expressed as follows: L = L dist +L order .

[0166] Among them, the sequential loss L order Defined as:

[0167]

[0168]

[0169] Where σ is the sigmoid activation function, B is the batch size, and m is the number of image slices sampled for each 3D medical image data volume. ij The model output s represents the j-th layer image slice of the i-th 3D medical image data volume in the sampled data. i,j+1 This represents the model output of the (j+1)th layer image slice of the i-th three-dimensional medical image data volume in the sampled data. This represents the difference between the two outputs of the model. The expression for the sigmoid function is:

[0170]

[0171] If the model output of this set of slices does not monotonically increase with the hierarchical index, then the sequential loss L order It will be very big.

[0172] Distance loss L dist Defined as:

[0173]

[0174] Where l1 is the smoothing L1 loss, expressed as:

[0175]

[0176] If the model outputs the difference between two consecutive sampled image slices... The difference is particularly large. According to the above formula, the distance loss will be very large, and the distance loss prompts the model to maintain a linear relationship between the hierarchical index of the image slice and the model output.

[0177] The hierarchical index difference k of the sampled image slices varies for different 3D medical images. This is mainly because the physical distance z between adjacent image slices differs in different 3D medical images. To ensure that the spacing between each image slice is equal in physical length, the hierarchical index difference k of different 3D medical image data volumes is obtained using the following formula:

[0178]

[0179] in, This refers to the physical distance between two randomly sampled image slices. Ensuring that the spacing between all image slices is equal in physical length is crucial because one term in the training loss is distance loss, which helps the model generate regression values ​​with equal spacing. If the physical lengths of the slices are unequal, the regression values ​​with equal spacing will not match the unequal-length slices, causing the trained model to fail to accurately map skeletal landmarks in the body.

[0180] As an example, during model training, the number of image slices sampled (m) is 4, the batch size is 64, the number of training iterations is 960, and the initial learning rate is 0.0001.

[0181] Self-supervised learning is employed to regress body parts. By comparing the positional distance differences between different cross-sections (i.e., image slices) in the z-direction, the algorithm learns and determines the relationships between these cross-sections, thereby achieving axial body part regression in CT images. The regression algorithm obtains the regression value for each cross-section, and the body part located at that cross-section is determined based on the regression value.

[0182] In the first training phase, 3D medical images from the first training dataset are input into the initial training model. The initial training model performs image analysis on each layer of the input 3D medical images, and outputs initial regression values ​​for each layer of the input 3D medical images. After the first training phase, a skeletal marker recognition model is obtained.

[0183] In the second training phase, the 3D medical images from the second training dataset are input into the skeleton marker recognition model. The model performs image analysis on image slices of each layer in the input 3D medical images to obtain initial regression values ​​for multi-layer image slices with the same skeleton markers. Based on multiple initial regression values ​​for the same skeleton marker, the average regression value corresponding to that skeleton marker is calculated. The correspondence between different skeleton markers and the average regression value is recorded in the skeleton marker recognition model.

[0184] Among the optional implementation methods, medical image registration methods also include:

[0185] Based on the average regression values ​​corresponding to the first and second bone markers, the calculated average regression values ​​corresponding to each bone marker are linearly transformed to a preset numerical range. The correspondence between the linearly transformed regression values ​​and each bone marker is then constructed and recorded in the bone marker recognition model. Specifically, the linear transformation can refer to transforming to the range of 0 to 100.

[0186] Through the above training and recording, when using the skeletal marker recognition model for model inference, it can output regression values ​​based on the two-dimensional image slices of the input medical image and identify the skeletal markers present in the medical image.

[0187] This application first employs self-supervised learning to regress body parts, performing coarse registration based on common skeletal markers from the first and second medical images. This coarse registration offers higher accuracy compared to commonly used methods such as torque principal axis and centroid alignment. Fine registration can then be used to further improve registration accuracy and ease of use, thereby enhancing the efficiency of image interpretation for doctors.

[0188] Based on the medical image registration method described in the foregoing embodiments, this application also provides a medical image registration device. Figure 7 This is a schematic diagram of the device, as shown below. Figure 7 As shown, the device includes:

[0189] Image acquisition module 71 is used to acquire a first medical image and a second medical image to be mutually registered.

[0190] The marker recognition module 72 is used to identify bone markers present in the first medical image and to identify bone markers present in the second medical image;

[0191] The marker determination module 73 is used to determine the same bone markers present in the first medical image and the second medical image;

[0192] The offset acquisition module 74 is used to obtain the z-direction offset between the first medical image and the second medical image based on the layer index of the image slice to which the same bone marker belongs in the first medical image, the z-direction pixel size of the first medical image, the layer index of the image slice to which the same bone marker belongs in the second medical image, and the z-direction pixel size of the second medical image; the z-direction is perpendicular to the image slice in the first medical image and the second medical image.

[0193] The image registration module 75 is used to register the first medical image and the second medical image based on the z-direction offset.

[0194] In an optional implementation, the offset acquisition module 74 in the medical image registration device is further used for:

[0195] Obtain the x-direction offset between the first medical image and the second medical image, and the y-direction offset between the first medical image and the second medical image;

[0196] Medical image registration devices also include:

[0197] The first construction module is used to construct a first registration matrix between the first medical image and the second medical image based on the z-direction offset, the x-direction offset, and the y-direction offset.

[0198] The image registration module 75 is specifically used for:

[0199] The first medical image and the second medical image are registered based on the first registration matrix; the z-direction offset in the first registration matrix is ​​used to register the first medical image and the second medical image in the z-direction; the x-direction offset and the y-direction offset in the first registration matrix are used together to register the image center points of the image slices of the first medical image and the second medical image.

[0200] In optional implementations, the medical image registration device also includes:

[0201] The parameter iteration optimization module is used to iteratively calculate the mutual information between the first medical image and the second medical image by adjusting the translation and rotation parameters between them, and to determine the translation and rotation parameters that minimize the mutual information as the target translation parameters and target rotation parameters, respectively. The target translation parameters include translation parameters in the x-direction, y-direction, and z-direction, and the target rotation parameters include rotation angles around the x-axis, y-axis, and z-axis.

[0202] The image registration module 75 is specifically used for:

[0203] Based on the target translation parameters and the target rotation parameters, a mathematical transformation is performed on the first registration matrix to obtain a second registration matrix;

[0204] The first medical image and the second medical image are registered based on the second registration matrix.

[0205] In an optional implementation, the marker recognition module 72 is specifically used to identify skeletal markers present in the target medical image in the following ways:

[0206] Image preprocessing is performed on each layer of the target medical image to obtain processed image slices of each layer of the target medical image;

[0207] The image slices processed by each layer are input into the skeleton marker recognition model. The skeleton marker recognition model performs image analysis on the image slices processed by each layer to obtain the initial regression values ​​corresponding to the image slices processed by each layer output by the skeleton marker recognition model.

[0208] Based on the correspondence between regression values ​​and skeletal markers, the skeletal markers present in the target medical image are determined.

[0209] In optional implementations, the medical image registration device also includes:

[0210] The first transformation module is used to linearly transform the initial regression values ​​corresponding to each processed image slice to a preset numerical range based on the average regression value corresponding to the first bone marker and the average regression value corresponding to the second bone marker, so as to obtain the new regression values ​​corresponding to each processed image slice; the first bone marker is the uppermost bone marker representing the body part among a plurality of predefined bone markers, and the second bone marker is the lowermost bone marker representing the body part among the plurality of predefined bone markers.

[0211] Each regression value in the correspondence is specifically obtained by transforming the regression values ​​output by the model through the linear transformation.

[0212] In optional implementations, the medical image registration device also includes:

[0213] The data fitting module is used to construct data points based on the new regression values ​​and hierarchical indices corresponding to the image slices after processing at each layer, and to fit a straight line based on the constructed data points.

[0214] The anomaly removal module is used to remove abnormal data points based on the degree of deviation of each data point from the straight line.

[0215] The regression value determination module is used to calculate the maximum and minimum values ​​among the new regression values ​​of the retained data points, and to determine the new regression values ​​of the retained data points that are between the minimum and maximum values;

[0216] The tag recognition module 72 is specifically used for:

[0217] From the correspondence, determine the closest regression value for each new regression value between the minimum and maximum values;

[0218] Based on the correspondence, the skeletal marker corresponding to the closest regression value is determined.

[0219] In an optional implementation, the medical image registration device also includes a model training module for training the skeletal marker recognition model in the following manner:

[0220] Obtain a first training dataset and a second training dataset; the first training dataset contains multiple preprocessed 3D medical images without bone markers, and the second training dataset contains multiple preprocessed 3D medical images with bone markers.

[0221] In the first training phase, the three-dimensional medical images in the first training dataset are input into the initial model to be trained. The initial model to be trained performs image analysis on each layer of the input three-dimensional medical images, and outputs the initial regression values ​​of each layer of the input three-dimensional medical images. After the first training phase is completed, the skeleton marker recognition model is obtained.

[0222] In the second training phase, the three-dimensional medical images in the second training dataset are input into the bone marker recognition model. The bone marker recognition model performs image analysis on each layer of the input three-dimensional medical images to obtain the initial regression values ​​output by the bone marker recognition model for multi-layer image slices with the same bone marker. Based on multiple initial regression values ​​of the same bone marker, the average regression value corresponding to the bone marker is calculated. The correspondence between different bone markers and the average regression value is recorded in the bone marker recognition model.

[0223] In optional implementations, the medical image registration device also includes:

[0224] The second transformation module is used to linearly transform the calculated average regression values ​​corresponding to each bone marker to a preset numerical range based on the average regression values ​​corresponding to the first bone marker and the second bone marker, construct the correspondence between the linearly transformed new regression values ​​and each bone marker, and record it in the bone marker recognition model.

[0225] In an optional implementation, the offset acquisition module 74 is specifically used for:

[0226] The x-direction offset between the first medical image and the second medical image is calculated based on their respective x-direction dimensions and x-direction pixel sizes.

[0227] The y-direction offset between the first medical image and the second medical image is calculated based on their respective y-direction dimensions and y-direction pixel sizes.

[0228] In the optional implementation, the parameter iteration optimization module is specifically used to calculate mutual information in the following ways:

[0229] The gray-level histogram of the first medical image, the gray-level histogram of the second medical image, and the joint gray-level histogram of the first and second medical images are calculated using the coordinate increment method.

[0230] The gray-level probability distribution of the first medical image is obtained based on the gray-level histogram of the first medical image, the gray-level probability distribution of the second medical image is obtained based on the gray-level histogram of the second medical image, and the joint gray-level probability distribution of the first medical image and the second medical image is obtained based on the joint gray-level histogram.

[0231] The entropy of the first medical image is calculated based on the gray-level probability distribution of the first medical image, the entropy of the second medical image is calculated based on the gray-level probability distribution of the second medical image, and the joint entropy of the first medical image and the second medical image is calculated based on the joint gray-level probability distribution.

[0232] The mutual information between the first medical image and the second medical image is calculated based on the entropy of the first medical image, the entropy of the second medical image, and the joint entropy.

[0233] Based on the medical image registration method and apparatus described in the foregoing embodiments, this application also provides a medical image registration device, which includes a processor and a memory that are communicatively connected to each other.

[0234] The memory stores computer programs;

[0235] The processor is configured to run the computer program to implement the medical image registration method as described in any of the implementations in the method embodiments.

[0236] In addition, this application also provides a computer-readable storage medium storing a computer program that, when processed, implements the steps of the medical image registration method as described in any implementation of the method embodiments.

[0237] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and equipment embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0238] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for registering medical images, characterized in that, include: Acquire the first and second medical images to be mutually registered; Identify skeletal markers present in the first medical image and identify skeletal markers present in the second medical image; Identify the same bone markers present in the first medical image and the second medical image; Based on the hierarchical index of the image slice to which the same bone marker belongs in the first medical image, the pixel size in the z-direction of the first medical image, the hierarchical index of the image slice to which the same bone marker belongs in the second medical image, and the pixel size in the z-direction of the second medical image, the z-direction offset between the first medical image and the second medical image is obtained; the z-direction is perpendicular to the image slices in the first medical image and the second medical image. The first medical image and the second medical image are registered based on the z-direction offset.

2. The method according to claim 1, characterized in that, Before registering the first medical image and the second medical image, the method further includes: Obtain the x-direction offset between the first medical image and the second medical image, and the y-direction offset between the first medical image and the second medical image; Based on the z-direction offset, the x-direction offset, and the y-direction offset, a first registration matrix is ​​constructed between the first medical image and the second medical image; The registration of the first medical image and the second medical image based on the z-direction offset is specifically as follows: The first medical image and the second medical image are registered based on the first registration matrix; the z-direction offset in the first registration matrix is ​​used to register the first medical image and the second medical image in the z-direction; the x-direction offset and the y-direction offset in the first registration matrix are used together to register the image center points of the image slices of the first medical image and the second medical image.

3. The method according to claim 2, characterized in that, After constructing the first registration matrix between the first medical image and the second medical image, the method further includes: By adjusting the translation and rotation parameters between the first and second medical images, the mutual information between the first and second medical images is iteratively calculated. The translation and rotation parameters that maximize the mutual information are then determined as the target translation parameters and target rotation parameters, respectively. The target translation parameters include translation parameters in the x, y, and z directions, and the target rotation parameters include rotation angles around the x-axis, y-axis, and z-axis. The registration of the first medical image and the second medical image based on the first registration matrix includes: Based on the target translation parameters and the target rotation parameters, a mathematical transformation is performed on the first registration matrix to obtain a second registration matrix; The first medical image and the second medical image are registered based on the second registration matrix.

4. The method according to claim 1, characterized in that, The method for identifying skeletal markers in target medical images is as follows: Image preprocessing is performed on each layer of the target medical image to obtain processed image slices of each layer of the target medical image; The image slices processed by each layer are input into the skeleton marker recognition model. The skeleton marker recognition model performs image analysis on the image slices processed by each layer to obtain the initial regression values ​​corresponding to the image slices processed by each layer output by the skeleton marker recognition model. Based on the correspondence between regression values ​​and skeletal markers, the skeletal markers present in the target medical image are determined.

5. The method according to claim 4, characterized in that, After obtaining the initial regression values ​​corresponding to each processed image slice output by the skeletal marker recognition model, the method further includes: Based on the average regression value corresponding to the first bone marker and the average regression value corresponding to the second bone marker, the initial regression values ​​corresponding to the processed image slices of each layer are linearly transformed to a preset numerical range to obtain the new regression values ​​corresponding to the processed image slices of each layer; the first bone marker is the uppermost bone marker representing the body part among a plurality of predefined bone markers, and the second bone marker is the lowermost bone marker representing the body part among the plurality of predefined bone markers. Each regression value in the correspondence is specifically obtained by transforming the regression values ​​output by the model through the linear transformation.

6. The method according to claim 5, characterized in that, After obtaining the new regression values ​​corresponding to the image slices after each layer of processing, the method further includes: Data points are constructed based on the new regression values ​​and hierarchical indices corresponding to the image slices after each layer of processing, and a straight line is fitted based on the constructed data points. Based on the degree of deviation of each data point from the straight line, abnormal data points are filtered out; Calculate the maximum and minimum values ​​among the new regression values ​​of the retained data points, and determine the new regression values ​​of the retained data points that are between the minimum and maximum values; The determination of skeletal markers present in the target medical image based on the correspondence between regression values ​​and skeletal markers includes: From the correspondence, determine the closest regression value for each new regression value between the minimum and maximum values; Based on the correspondence, the skeletal marker corresponding to the closest regression value is determined.

7. The method according to claim 5, characterized in that, The training process of the skeletal marker recognition model includes: Obtain a first training dataset and a second training dataset; the first training dataset contains multiple preprocessed 3D medical images without bone markers, and the second training dataset contains multiple preprocessed 3D medical images with bone markers. In the first training phase, the three-dimensional medical images in the first training dataset are input into the initial model to be trained. The initial model to be trained performs image analysis on each layer of the input three-dimensional medical images, and outputs the initial regression values ​​of each layer of the input three-dimensional medical images. After the first training phase is completed, the skeleton marker recognition model is obtained. In the second training phase, the three-dimensional medical images in the second training dataset are input into the bone marker recognition model. The bone marker recognition model performs image analysis on each layer of the input three-dimensional medical images to obtain the initial regression values ​​output by the bone marker recognition model for multi-layer image slices with the same bone marker. Based on multiple initial regression values ​​of the same bone marker, the average regression value corresponding to the bone marker is calculated. The correspondence between different bone markers and the average regression value is recorded in the bone marker recognition model.

8. The method according to claim 7, characterized in that, Also includes: Based on the average regression value corresponding to the first bone marker and the average regression value corresponding to the second bone marker, the calculated average regression value corresponding to each bone marker is linearly transformed to a preset numerical range, and the correspondence between the new linearly transformed regression value and each bone marker is constructed and recorded in the bone marker recognition model.

9. A medical image registration device, characterized in that, include: The image acquisition module is used to acquire the first and second medical images to be mutually registered. A marker recognition module is used to identify bone markers present in the first medical image and to identify bone markers present in the second medical image; A marker determination module is used to determine the same bone markers present in the first medical image and the second medical image; The offset acquisition module is used to obtain the z-direction offset between the first medical image and the second medical image based on the layer index of the image slice to which the same bone marker belongs in the first medical image, the z-direction pixel size of the first medical image, the layer index of the image slice to which the same bone marker belongs in the second medical image, and the z-direction pixel size of the second medical image; the z-direction is perpendicular to the image slices in the first medical image and the second medical image. The image registration module is used to register the first medical image and the second medical image based on the z-direction offset.

10. A medical image registration device, characterized in that, include: Interconnected processors and memory; The memory stores computer programs; The processor is configured to run the computer program to implement the registration method for medical images as described in any one of claims 1-8.

Images