Image processing method and device, terminal equipment and computer readable storage medium

Abstract

The invention is suitable for the technical field of medical image registration, and provides an image processing method and device, terminal equipment and a computer readable storage medium, and the method comprises the steps: obtaining a first image which represents the target body tissue of a patient; based on a second image, first displacement distances in multiple directions corresponding to multiple first target points of the target body tissue in the first image are obtained, the second image is a reference image corresponding to the target body tissue in the first image, and the multiple directions are multiple directions of the sampling space; and for each first target point, based on the first displacement distance corresponding to the first target point, mapping the plurality of first target points to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image. According to the method, the topological structure after image registration can be ensured, and the accuracy of image registration can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of medical image registration, and in particular, relates to an image processing method, apparatus, terminal device and computer-readable storage medium. Background Art

[0002] Medical image registration is a fundamental task in the field of computer vision, which aims to establish the spatial correspondence between two images (referred to as the moving image and the fixed image).

[0003] The related art uses a pixel intensity consistency method to align the moving image with the fixed image, but this method cannot guarantee the topological structure of the moving image after deformation, resulting in low image registration performance. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, terminal device and computer-readable storage medium, which can solve the problem of ensuring the topological structure after image registration and improve the accuracy of image registration.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Acquire a first image, the first image being an image representing a target body tissue of a patient;

[0007] Based on the second image, obtaining displacement distances in multiple directions corresponding to multiple first target points of the target body tissue in the first image, wherein the second image is a reference image corresponding to the target body tissue in the first image, and the multiple directions are multiple directions of the sampling space;

[0008] For each first target point, based on the first displacement distance corresponding to the first target point, multiple first target points are mapped to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image.

[0009] In an embodiment of the present application, an image is selected or obtained from a series of medical images of a patient, and this image will be used for subsequent analysis and feature extraction. Usually, this image contains key information related to the patient's characteristics (such as lesions, structures, etc.), and then the displacement distances required to move multiple target points in the first image to the second image are obtained, and the first image is geometrically transformed using the obtained displacement distances. Since the result relationship between multiple first target points can be guaranteed during the transformation of the first image using the obtained displacement distances, the target point mapping can be spatially aligned with the reference image corresponding to the patient, which can improve the accuracy of image registration.

[0010] In a possible implementation manner of the first aspect, acquiring, based on the second image, first displacement distances in multiple directions corresponding to multiple first target points of the target body tissue in the first image, includes:

[0011] Inputting a plurality of first target points in the first image into the trained registration model;

[0012] The first displacement distances in multiple directions corresponding to the multiple first target points are generated according to the trained registration model.

[0013] In one embodiment, the method further comprises:

[0014] Acquire a first sample image, where the first sample image is an image representing a target body tissue of a patient;

[0015] Obtaining a registration model to be trained, where the registration model to be trained is used to obtain a second displacement distance that a first sample point in a first sample image needs to move, wherein the second displacement distance is a distance that the first sample point needs to move when mapping the first sample point of the first sample image to a corresponding position of a second sample image, and the second sample image is a reference image corresponding to the target body tissue in the first sample image;

[0016] Based on the first sample image, the registration model to be trained is adjusted to obtain a trained registration model.

[0017] In one embodiment, adjusting the registration model to be trained to obtain a trained registration model includes:

[0018] Setting a first function, the first function being used to represent the consistency between a first distance between a plurality of first sample points and a second distance between a plurality of second sample points, wherein the second sample points are sample points in the sample image when the first sample image is mapped to the same position of the second sample image in the sampling space;

[0019] Obtaining, by means of a first function, a first output result obtained after inputting a plurality of first sample points of the first sample image into the registration model to be trained, wherein the first output result indicates the second displacement distance that the first sample point needs to move when mapping the first sample point to the second sample image;

[0020] Acquire a first registration result between the first sample image and the second sample image according to the first output result;

[0021] The registration model parameters are adjusted according to the first function result until the first registration result is within a preset threshold, and then it is determined that the registration model training is completed, and the trained registration model is obtained.

[0022] In one embodiment, adjusting the registration model to be trained to obtain a trained registration model includes:

[0023] Setting a second function, wherein the second function is used to represent the similarity of pixel intensities of the first sample point and the second sample point;

[0024] Obtaining, by means of the second function, a second output result obtained after inputting the first sample image into the registration model to be trained;

[0025] Acquire a second registration result between the first sample image and the second sample image according to the second output result;

[0026] The registration model parameters are adjusted according to the result of the second function until the second registration result is within a preset threshold, and then it is determined that the training of the registration model is completed, and the trained registration model is obtained.

[0027] In one embodiment, mapping the plurality of first target points to the same position of the second image in the sampling space based on the first displacement distance corresponding to the first target point to obtain a registration result between the first image and the second image includes:

[0028] Moving the first target point by a corresponding first displacement distance to obtain a corresponding second target point;

[0029] The multiple second target points are mapped to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image.

[0030] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0031] A first acquisition module, configured to acquire a first image, wherein the first image is an image representing a target body tissue of a patient;

[0032] A second acquisition module is used to acquire displacement distances in multiple directions corresponding to multiple first target points of the target body tissue in the first image based on the second image, wherein the second image is a reference image corresponding to the target body tissue in the first image, and the multiple directions are multiple directions of the sampling space;

[0033] The mapping module is used to map multiple first target points to the same position of the second image in the sampling space based on the first displacement distance corresponding to each first target point, so as to obtain the registration result between the first image and the second image.

[0034] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an image processing method as described in any one of the first aspects above is implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the image processing method as described in any one of the first aspects above.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the image processing methods in the first aspect.

[0037] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 is a flowchart of an image processing method provided in an embodiment of the present application;

[0040] Figure 2 is a schematic diagram of a process for obtaining a first displacement distance provided in an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a process for obtaining a trained registration model provided in an embodiment of the present application;

[0042] Figure 4 This is a schematic diagram of the process of the registration model provided in the embodiment of the present application. Figure 1 ;

[0043] Figure 5 Schematic diagram of internal and external mapping provided by the embodiment of the present application;

[0044] Figure 6 It is a schematic diagram of the process of distance consistency generation provided in an embodiment of the present application;

[0045] Figure 7 This is a schematic diagram of the process of the registration model provided in the embodiment of the present application. Figure 2 ;

[0046] Figure 8 is a schematic block diagram of image registration provided by an embodiment of the present application;

[0047] Fig. 9 is a schematic diagram of qualitative comparison of data sets provided in the embodiments of the present application;

[0048] Fig.10 is the comparison result provided by the embodiment of the present application;

[0049] Fig.11 This is the ablation experiment 1 provided in the embodiment of the present application;

[0050] Fig.12 This is ablation experiment 2 provided in the embodiment of the present application;

[0051] Fig.13 This is ablation experiment 3 provided in the embodiment of the present application;

[0052] Fig.14 is a structural block diagram of an image processing device provided in an embodiment of the present application;

[0053] Fig.15 It is a structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0060] In the medical field, image registration is a basic task in the field of computer vision, which aims to establish the spatial correspondence between two images (referred to as moving images and fixed images respectively). The main goal of image registration is to achieve accurate superposition between images captured at different times, different perspectives, or by different sensors, ensuring the precise alignment of corresponding features. The mapping usually desired is a regular mapping that maintains smoothness and reversibility, such as a homeomorphic mapping. The continuity and reversibility of the homeomorphic mapping can preserve the anatomical structure, which is crucial in the field of medical imaging. This type of mapping ensures that the topological structure of the image remains unchanged, thus avoiding unnatural deformations such as folding or tearing.

[0061] In the related art, most successful methods are based on learning, using convolutional neural networks or transformers to model the transformation between images. However, these methods usually require a large amount of training data and often perform poorly in accurately capturing the diverse deformations in different image pairs. Some methods usually rely on predefined models or assumptions to maintain homeomorphism in images, such as solving the Lamé-Navier elastic equations or using models inspired by fluid mechanics. However, choosing a suitable transformation model remains challenging as different assumptions have different impacts on the performance. Recent advances in learning have improved homeomorphism modeling, especially in enforcing inverse consistency, by ensuring that the inverse mapping can be obtained simply by swapping the input order, thereby maintaining the topological structure. These models require a large number of medical image datasets for training, and they usually have difficulty capturing the complex and diverse deformations in different image pairs. To solve these problems, researchers have tried to use implicit neural representation (INR) to optimize the mapping between images and use only intensity consistency to find the alignment of two images in space. Although INR-based methods can capture the complex and diverse transformations between image pairs without a large training dataset, it is still challenging to implement inverse consistency-based methods in INR. The above methods cannot guarantee the topological structure of the images after registration, resulting in low image registration accuracy.

[0062] In order to solve the problems arising in the above-mentioned related technologies, an embodiment of the present application provides an image processing method. In the present application, an INR-based framework is proposed. By adding a new distance consistency optimization function, the new distance consistency enables a new branch to learn the distance order of voxels in the three-dimensional magnetic resonance imaging (MRI) volume in the medical image, which is used to establish a spatial correspondence between two images while maintaining the topological structure, thereby improving the image registration performance.

[0063] See also Figure 1 , is a flowchart of an image processing method provided in an embodiment of the present application. As an example but not a limitation, the method may include the following steps:

[0064] S101, acquiring a first image, where the first image is an image representing a target body tissue of a patient.

[0065] In the present application, "target body tissue" generally refers to a specific part or tissue of the body (target object) that the researcher is interested in. For example, in cancer diagnosis, the target body tissue may be the specific area where the tumor is located; in cardiac imaging, the target body tissue may be the heart muscle or valve.

[0066] Medical imaging technology can capture image data (first image) of a specific part of a patient's body, such as collecting images of a patient's target body tissue through medical equipment. Usually, this process involves using imaging technologies such as X-rays, computed tomography (CT) scans, MRI, ultrasound or other imaging technologies to generate images.

[0067] S102, based on the second image, obtaining displacement distances in multiple directions corresponding to multiple first target points of the target body tissue in the first image, wherein the second image is a reference image corresponding to the target body tissue, and the multiple directions are multiple directions of the sampling space.

[0068] In the present application, the second image is a reference image corresponding to the target body, which can be used as a reference image. After the first image is spatially transformed or mapped, it is aligned with the second image in space to detect the difference between the first image and the reference image, so as to confirm whether the current first image has pathological factors.

[0069] In image registration, the main goal is to analyze the spatial relationship between objects (first image and second image), while less attention is paid to other blank areas in the image. For example, in an MRI of the human brain, the brain (target body tissue) only occupies a portion of the image (first image) space, similar to a sphere that occupies approximately half of the available volume. Therefore, it is necessary to perform target sampling on the first image, obtain only the portion of the target body tissue, remove the blank area in the image, and perform image processing only on the brain portion. MRI images are generally displayed in a three-dimensional coordinate grid such as P (H×W×D×3), which contains multiple voxel points. The voxels in the image corresponding to the brain portion obtained by target sampling of the first image are called multiple first target points. When aligning the first image with the reference image, it is necessary to change the position of multiple first target points in the first image so that they are in the same space or the same coordinate system as the position points in the reference image. Therefore, it is necessary to obtain the displacement distance that the first target point needs to move in multiple directions in space through the framework proposed in this application, so as to align it with the reference image after the movement.

[0070] Compared with traditional learning-based methods, such as CNNs transformers, the above-mentioned method for target sampling usually represents the input image and the generated deformation field as a matrix due to the architectural limitations of these models. In contrast, the INR-based framework of the present application can provide greater flexibility, allowing any coordinates in the image domain to be directly used as input (multiple first target points obtained by target sampling). This flexibility enables the model to selectively focus on coordinates corresponding to areas where objects exist or areas close to objects, thereby improving computational efficiency, and enhancing model performance by focusing resources on the most relevant areas in the image, and then processing the multiple first target points obtained by target sampling to obtain the distances in multiple directions for spatial transformation, which can improve the efficiency and accuracy of image registration.

[0071] In one embodiment, see Figure 2 , is a schematic diagram of a process for obtaining a first displacement distance provided in an embodiment of the present application, step S102 includes:

[0072] S201, inputting a plurality of first target points in a first image into a trained registration model.

[0073] In an embodiment of the present application, the registration model is the distance consistency (Distance consistency Implicit Neural Representation, DisIR) model based on INR proposed in this application. After the model is trained to obtain a trained model, the multiple first target points obtained by target sampling are input into the trained model.

[0074] S202: Generate first displacement distances in multiple directions corresponding to multiple first target points according to the trained registration model.

[0075] In the embodiment of the present application, according to a trained registration model DisIR, for each detected target point (first target point), the DisIR model needs to calculate the first displacement distance of the point in multiple predefined directions. This displacement distance is determined by spatial transformation, that is, by simulating or actually applying a certain spatial transformation (such as translation) to measure the distance of the target point from the spatial position corresponding to the first image to the spatial position corresponding to the reference image (second image).

[0076] In one embodiment, see Figure 3, is a flow chart of obtaining a trained registration model provided in an embodiment of the present application. Before calculating the distances that multiple target points in the first image need to move according to the trained DisIR model, the DisIR model needs to be trained so that the trained DisIR model can calculate the optimal distances that multiple target points need to move, so as to improve the registration between the target points after movement or spatial transformation and the position points in the reference image. The training process of the DisIR model is as follows:

[0077] S301, acquiring a first sample image, where the first sample image is an image representing a target body tissue of a patient.

[0078] In an embodiment of the present application, when training the DisIR model, it is necessary to obtain multiple medical images representing specific important body parts or tissues related to the patient's health, such as using X-rays, MRI, CT scans or other imaging technologies to obtain images of the patient (first sample images).

[0079] S302, obtaining a registration model to be trained, where the registration model to be trained is used to obtain a second displacement distance that a first sample point in a first sample image needs to move, wherein the second displacement distance is a distance that the first sample point needs to move when mapping the first sample point of the first sample image to a corresponding position of a second sample image, and the second sample image is a reference image corresponding to the target body tissue in the first sample image.

[0080] In the embodiment of the present application, an untrained registration model INR is used to learn how to calculate the displacement distance (second displacement distance) of a specific point (first sample point) in the first sample image to the corresponding position in the second sample image. This displacement distance is determined by mapping the first sample point from the first image to the second sample image, which means that the model needs to learn how to calculate the relative position change between points based on the input image.

[0081] S303: Based on the first sample image, adjust the registration model to be trained to obtain a trained registration model.

[0082] In the embodiment of the present application, the first sample image is the basis for model learning and adjustment. The DisIR model to be trained is optimized and adjusted according to the image features and content in the first image to improve its performance. The model is trained using the first sample image and other data. During the training process, the model will learn how to extract features from the image and make predictions. According to the prediction results, the parameters of the model are adjusted to improve the accuracy and performance of the model.

[0083] In one embodiment, see Figure 4 , is a schematic diagram of the process of the registration model provided in the embodiment of the present application Figure 1, step S303 includes:

[0084] S401: Setting a first function, where the first function is used to represent consistency between a first distance between a plurality of first sample points and a second distance between a plurality of second sample points, wherein the second sample points are sample points in a sample image when the first sample image is mapped to the same position of a second sample image in a sampling space.

[0085] In an embodiment of the present application, the first function is a distance consistency loss function proposed in the present application, and the purpose of this function is to measure the consistency between two distances, that is, the degree of match between the distance between sample points in the first sample image and the distance between corresponding sample points after being mapped to the second image.

[0086] Image registration aims to align a moving image to a fixed image by maximizing similarity under certain constraints. Image registration is usually formulated as follows:

[0087]

[0088] Among them, u * Represents the optimal displacement field x∈Ω, H represents a Hilbert space, Function E D : is an energy function used to measure the fixed image I f And the deformed image Here, is the image domain, represents the deformation field, λ is the regularization weight, and R(u) represents the regularization term.

[0089] In Euclidean space, for an open set X, consider the boundary points The surrounding small neighborhood U x , where N x It's the border Normal vector at x. Satisfies the following conditions:

[0090]

[0091] This means that the tangent plane at x will be adjacent to u x Divided into two areas: one belongs to X (inside x+y i ), and the other one is outside X (external x+y e ),like Figure 5 As shown in the figure, the internal and external mapping diagram provided by the embodiment of the present application, in mathematics, an open mapping is defined as a function that maps an open set to an open set. Specifically, for an open set X, if is an open mapping, then It is also an open set.

[0092]

[0093] Specifically, during the registration process, the inside and outside should remain unchanged. This principle can be extended to a simpler statement based on distance, and the simplified problem statement is as follows:

[0094]

[0095] in, is the optimal solution that is closest to the homeomorphism, y is a grid point near x, sgn(x) is a sign function that takes the value of -1 when x < 0 and takes the value of 1 when x ≥ 0, and D(x) = |x|2. Formula 4 describes the requirement that the ordering of the distance from the origin should remain unchanged after the deformation process; points far from the origin should remain far away, and points close to the origin should remain close. This property is called distance consistency. The distance consistency loss aims to minimize the difference in distance ordering. The first function consists of two parts: distance field generation and similarity measurement.

[0096] The distance field uses the origin of the image as the reference point, see Figure 6 , is a flow chart of distance consistency generation provided by an embodiment of the present application, as shown in the figure, which corresponds to the anterior commissure (AC) of the human brain, because all MRI images have been pre-aligned to the MNI space, which is a standard human brain space defined by a doctor. However, using a single reference point is not enough to ensure the preservation of topology. To solve this problem, we introduced a set of additional eight reference points located at the vertices of a cube that surrounds the MRI image and has a side length twice that of the original image domain. First, nine distance fields are generated based on the reference points (the distance from all target points to the reference points in the first image). Next, the distorted distance field (the distance from all target points to the reference point in the image obtained after mapping) is obtained by applying a deformation field. Finally, LNCC is used to measure the similarity between the original distance field and the distorted distance field.

[0097] D i (x)=||xx i ||2, i=1, ...9 (5)

[0098] Similarity metric: To achieve differentiability and local sensitivity, we adopt the local normalized cross correlation (LNCC) as the metric instead of using a sign function:

[0099]

[0100] Where A, B: P is a point in a local window around ρ, with a window size of n3 , σ is the standard deviation, cov is the covariance, is the expectation and μ is the mean. A higher LNCC value indicates a stronger correspondence between the inputs. Combining the distance field with LNCC, we define the distance loss function (first function) as:

[0101]

[0102] This loss function encourages the deformation field to preserve the homeomorphic mapping while being computationally cheap. This approach is simpler to implement than inverse consistency and can be conveniently applied to any registration method.

[0103] S402, obtaining, through a first function, a plurality of first sample points of a first sample image and inputting them into a registration model to be trained to obtain a first output result, wherein the first output result indicates a second displacement distance that the first sample point needs to move when mapping the first sample point to a position corresponding to the second sample image.

[0104] In an embodiment of the present application, target sampling is also performed in the first sample image to obtain multiple first sample points of multiple target body tissues, and the multiple first sample points are input into the INR model to be trained. The model will calculate an initial displacement distance (the second displacement distance, i.e., the first output result). The second displacement distance is the spatial distance that needs to be moved when the multiple first sample points are mapped to the same position in the second sample image.

[0105] S403: Obtain a first registration result between the first sample image and the second sample image according to the first output result.

[0106] In the embodiment of the present application, multiple sample points in the first sample image are moved to corresponding positions in the second sample image according to the displacement distance in the first output result. After the movement, multiple first sample points generate a new image, that is, the mapped second sample image. The alignment result (first registration result) between the second sample image and the second image is calculated by a related method. For example, the Dice similarity coefficient (DSC) can be used to evaluate the registration accuracy. The higher the DSC value, the more accurate the registration between the images.

[0107] S404: Adjust the registration model parameters according to the result of the first function until the second registration result obtained is within a preset threshold, then determine that the registration model training is completed, and obtain the trained registration model.

[0108] In an embodiment of the present application, the output result of the INR model is adjusted by adjusting the registration model parameters according to the result of the first function to change the registration result until the obtained registration result is within a preset range. It can be determined that the training of the model is completed and the trained registration model is obtained.

[0109] In one embodiment, see Figure 7 , is a schematic diagram of the process of the registration model provided in the embodiment of the present application Figure 2 , step S303 further includes:

[0110] S501: Set a second function, where the second function is used to represent the similarity between the pixel intensities of the first sample point and the second sample point.

[0111] In the embodiment of the present application, the second function uses LNCC to measure the intensity similarity between two images:

[0112]

[0113] A second output result obtained by inputting the first sample image into the registration model to be trained is obtained through the second function, so as to obtain a second registration result between the second sample image and the first sample image.

[0114] In an embodiment of the present application, by integrating the second function into the INR model to be trained, and then inputting multiple first sample points into the integrated INR model, a second output result, i.e., the moving distance of the first sample point, can be obtained, and then the alignment result (second registration result) between the mapping image and the second sample image is obtained according to the preset distance. The process of obtaining the second registration result is the same as the process of the first registration result mentioned above, and will not be repeated here.

[0115] S504, adjusting the registration model parameters according to the second function result until the second registration result obtained is within a preset threshold, then determining that the registration model training is completed, and obtaining a trained registration model.

[0116] In an embodiment of the present application, the output result of the INR model is adjusted by adjusting the registration model parameters according to the result of the second function to change the registration result until the obtained registration result is within a preset range. It can be determined that the training of the model is completed and the trained registration model is obtained.

[0117] In another embodiment, to ensure local directional consistency, we use Jacobian regularization, which ensures that if the Jacobian is positive, the deformation field (spatial transformation) remains directional consistent. On the contrary, if the determinant is negative, it means that the direction of the local area has been flipped, which may lead to topological inconsistency. By using the ReLU activation function, we can effectively reduce the area where the Jacobian is negative, thereby maintaining the integrity of the local direction, where N is the number of points x:

[0118]

[0119] In order to prevent abnormal deformation, we introduce a spatial gradient constraint on the smoothness of the deformation field as a regularization component:

[0120]

[0121] The complete loss function is expressed as follows:

[0122]

[0123] Among them, λ1, λ2 and λ3 control the weights of distance consistency loss, direction consistency loss and deformation smoothness loss:

[0124] By integrating formula (11) into the INR model, training is performed to obtain the registration result as described above, so that the obtained registration result is optimal.

[0125] During the optimization process, the hyperparameters of Formula (11) are set to λ1=0.1, λ2=10 and λ3=1. In addition, we set the LNCC window size of the distance loss function to 9 and the window size of the distance loss function to 7. Referring to previous work, we adopt a SIREN-inspired strategy to effectively capture high-frequency content using a periodic sinusoidal activation function. Our implicit neural representation model consists of a 3-layer MLP with 512 hidden units in each layer. Due to GPU memory limitations, it is impractical to directly input all 3D coordinates (D×H×W×3) into the MLP. Therefore, we downsample the original coordinate grid by 1 / 2 and then upsample the generated deformation field to restore it to full resolution. For each pair of images, our model is set to perform 300 iterations. The network parameters are optimized using the Adam optimizer with a learning rate of 10 -4 The model is implemented in PyTorch and evaluated on a server equipped with an NVIDIA RTX 3090 GPU.

[0126] The above method can integrate the loss function into the INR model to be trained, and use the sample image to train the integrated model, and finally obtain the registration model to be trained. Under the trained model, the image registration result obtained can reach the optimal value.

[0127] S103 , for each first target point, based on a first displacement distance corresponding to the first target point, mapping multiple first target points to the same position of the second image in the sampling space, to obtain a registration result between the first image and the second image.

[0128] In an embodiment of the present application, for each first target point, its corresponding first displacement distance is found. This displacement distance may refer to the distance after transformation such as translation, rotation or scaling from the first image to the second image. According to the first displacement distance corresponding to each first target point, these points are mapped to the corresponding positions of the second image. This means that the corresponding position of each point in the first image is found in the second image, and through the above mapping, the registration result between the first image and the second image can be obtained. This usually means that the points or features in the two images have been correctly matched.

[0129] In one embodiment, step S103 includes:

[0130] The first target point is moved by a corresponding first displacement distance to obtain a corresponding second target point; a plurality of second target points are mapped to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image.

[0131] In an embodiment of the present application, a point in the first image is moved to a new position according to a first displacement distance corresponding to each first target point to obtain a corresponding second target point, and the obtained multiple second target points are mapped to the same position of the second image in the sampling space. By mapping the second target points, a registration result between the first image and the second image can be obtained.

[0132] Among them, the above registration results use volume overlap of body tissue segmentation because it is challenging to obtain a dense true deformable field. The Dice similarity coefficient (DSC) is used to evaluate the registration accuracy. The higher the DSC value, the more accurate the registration between images. In addition, the standard deviation (STD) of these indicators is reported to gain insight into the stability and robustness of the model, and the local characteristics of the deformation field are evaluated by calculating the Jacobian matrix (JAC). Voxels with negative Jacobian matrix determinants are identified as singularities, indicating that topological folding has occurred. In order to evaluate the ability to preserve topological structure, we calculated the proportion of voxels with negative Jacobian determinants, thereby deriving a measure of the degree of topological distortion.

[0133] See also Figure 8 , is a schematic block diagram of image registration provided in an embodiment of the present application, such as Figure 6 As shown, the steps are as follows:

[0134] ① Sample the voxels containing the object in the coordinate grid P (first image) to obtain multiple three-dimensional points in P′ (first target point)

[0135] ② Input multiple first target points into the trained INR model to generate displacement distances corresponding to the multiple first target points, and generate deformation fields according to the displacement distances of the multiple first target points;

[0136] The deformation field learning is supervised by intensity consistency (second function) and distance consistency (first function), where distance consistency is to keep the distance field (the distance from all first target points to the reference point) of the first image (moving image) before spatial transformation consistent with the distorted distance field (the distance from the first target point to the reference point after spatial transformation) of the first image; intensity consistency is to ensure that the distorted image obtained after the transformation of the first image is consistent with the image intensity in the fixed image (second image);

[0137] ③ moving the multiple first target points according to the first displacement distance to obtain multiple second target points;

[0138] ④ Mapping multiple second target points to the same position of the second image to obtain a distorted image;

[0139] ⑤ Obtain the registration result between the distorted image and the second image.

[0140] After obtaining the registration result, in order to verify the registration accuracy of the registration result, refer to Fig. 9 , is a qualitative comparison diagram of the datasets provided in the embodiments of the present application. The method proposed by us is evaluated on three publicly available three-dimensional brain MRI datasets: the IXI dataset, which contains 576 T1-weighted (T1W) brain MRI images; the OASIS dataset, which contains 414 T1W brain MRI images; and the LPBA40 dataset, which contains 40 T1W brain MRI images. Each MRI image is preprocessed using FreeSurfer, including skull stripping, spatial normalization, affine transformation, and automatic structural segmentation. For the OASIS and IXI datasets, the images are cropped to a size of 160×192×224. For the LPBA40 dataset, the images are cropped to 160×192×160. The LPBA40 dataset also provides segmentation maps of 56 anatomical structures, which are used as benchmarks for evaluation. For the IXI dataset, segmentation maps of 30 anatomical structures are used, while for the OASIS dataset, segmentation maps of 35 anatomical structures are used as benchmarks for evaluation. For IXI and OASIS, we extract 70% of the data as training data, 10% as validation data, and 20% as test data to test the model performance of the learning-based method. For LPBA40, we use 30 MRI images as training data and 10 MRI images as test data. We only use test data on traditional methods and INR-based methods because these methods do not require training.

[0141] See also Fig.10 is the comparison result provided by the embodiment of the present application, such as Fig.10 As shown:

[0142] IXI dataset: Among the three baselines, the learning-based method TransMatch achieves the highest Dice score (78.00%), while the traditional method SyN records the lowest Jacobian matrix. Our method DisIR achieves the highest Dice score (78.60%) with the smallest variance (0.77%), surpassing TransMatch, and obtains a lower Jacobian matrix compared to all baselines except SyN.

[0143] OASIS dataset: Among the baseline methods, INR obtains the highest Dice score (81.05%), while SyN records a Jacobian matrix close to zero. In contrast, our method obtains the highest Dice score (81.73%) and the second best Jacobian matrix (0.08%).

[0144] LPBA40 dataset: Our method DisIR outperforms all baseline methods and obtains the highest Dice score (71.87%). In addition, in terms of Jacobian matrix, our method surpasses all learning-based and INR-based techniques by 0.03%.

[0145] Adding the distance consistency loss to VoxelMorph and TransMorph significantly improves the Dice similarity coefficient and Jacobian matrix in all datasets. These findings demonstrate that our proposed distance consistency module better preserves the topological structure of the anatomy and can be seamlessly integrated into existing INR-based methods and learning-based methods.

[0146] See also Fig.11 This is the ablation experiment 1 provided in the embodiment of the present application. When λ1 is set to 0.1, the best Dice similarity coefficient can be obtained. It is worth noting that increasing the weight of the distance consistency loss will lead to a further reduction in the Jacobian matrix, which indicates that the topological structure of the anatomical structure is better preserved, where the bold numbers indicate the best performance.

[0147] See also Fig.12 , is the second ablation experiment provided in the embodiment of the present application. When the window size is 7×7×7, the Dice similarity coefficient is the best. The deviation of the window size (whether increasing or decreasing) will cause a slight decrease in the Dice similarity coefficient, where the bold numbers indicate the best performance.

[0148] See also Fig.13 This is the ablation experiment 3 provided in the embodiment of the present application. Although the changes in the Dice similarity coefficient and the Jacobian matrix are small, after using target sampling, the prediction time is significantly reduced from 391.42 seconds to 250.00 seconds, where the bold numbers represent the best performance.

[0149] In this application, we introduce DisIR, an INR-based framework that combines distance consistency loss and target sampling, aiming to establish spatial correspondences between two images while maintaining topological integrity. We propose a novel regularizer derived from homeomorphism, called distance consistency, which is specifically designed to preserve the topological structure of anatomical structures in the transformed images and facilitate the learning of homeomorphic deformable transformations. This regularizer provides enhanced flexibility and can be seamlessly integrated into both INR-based and learning-based methods. Comprehensive experimental evaluations show that our method not only achieves superior registration accuracy but also preserves the anatomical topology, significantly outperforming existing baselines on three brain MRI datasets.

[0150] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0151] Corresponding to the image processing method of the above embodiment, Fig.11 This is a structural block diagram of the image processing device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0152] Reference Fig.14 , the device comprises:

[0153] The first acquisition module 141 is used to acquire a first image, where the first image is an image representing a target body tissue of a patient.

[0154] The second acquisition module 142 is used to acquire first displacement distances in multiple directions corresponding to multiple first target points of the target body tissue in the first image based on the second image, where the second image is a reference image corresponding to the target body tissue, and the multiple directions are multiple directions of the sampling space.

[0155] Optionally, the second acquisition module 142 is further used for:

[0156] Inputting a plurality of first target points in the first image into the trained registration model;

[0157] The first displacement distances in multiple directions corresponding to the multiple first target points are generated according to the trained registration model.

[0158] Optionally, the second acquisition module 142 is further used for:

[0159] Acquire a first sample image, where the first sample image is an image representing a target body tissue of a patient;

[0160] Obtaining a registration model to be trained, where the registration model to be trained is used to obtain a second displacement distance that a first sample point in a first sample image needs to move, wherein the second displacement distance is a distance that the first sample point needs to move when mapping the first sample point of the first sample image to a corresponding position of a second sample image, and the second sample image is a reference image corresponding to the target body tissue in the first sample image;

[0161] Based on the first sample image, the registration model to be trained is adjusted to obtain a trained registration model.

[0162] Optionally, the second acquisition module 142 is further used for:

[0163] Setting a first function, the first function being used to represent the consistency between a first distance between a plurality of first sample points and a second distance between a plurality of second sample points, wherein the second sample points are sample points in the sample image when the first sample image is mapped to the same position of the second sample image in the sampling space;

[0164] Obtaining, by means of a first function, a first output result obtained after inputting a plurality of first sample points of a first sample image into the registration model to be trained, wherein the first output result indicates the second displacement distance that the first sample point needs to move when mapping the first sample point to the second sample image;

[0165] Acquire a first registration result between the first sample image and the second sample image according to the first output result;

[0166] The registration model parameters are adjusted according to the first function result until the first registration result obtained is within a preset threshold, then it is determined that the registration model training is completed, and the trained registration model is obtained.

[0167] Optionally, the second acquisition module 142 is further used for:

[0168] Obtaining, by means of a second function, a second output result obtained by inputting the first sample image into the registration model to be trained;

[0169] Acquire a second registration result between the first sample image and the second sample image according to the second output result;

[0170] The registration model parameters are adjusted according to the result of the second function until the second registration result obtained is within a preset threshold, then it is determined that the registration model training is completed, and a trained registration model is obtained.

[0171] The mapping module 143 is used to map multiple first target points to the same position of the second image in the sampling space for each first target point based on the first displacement distance corresponding to the first target point, so as to obtain a registration result between the first image and the second image.

[0172] Optionally, the mapping module 143 is further configured to:

[0173] Move the first target point by a corresponding first displacement distance to obtain a corresponding second target point;

[0174] The plurality of second target points are mapped to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image.

[0175] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0176] in addition, Fig.14 The image processing device shown may be a software unit, a hardware unit, or a unit combining software and hardware built into an existing terminal device, or may be integrated into the terminal device as an independent accessory, or may exist as an independent terminal device.

[0177] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0178] Fig.15 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Fig.15 As shown, the terminal device 15 of this embodiment includes: at least one processor 150 ( Fig.15 Only one is shown in the figure) a processor, a memory 151, and a computer program 152 stored in the memory 151 and executable on at least one processor 150, and when the processor 150 executes the computer program 152, the steps in any of the above-mentioned image processing method embodiments are implemented.

[0179] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Fig.15 It is only an example of the terminal device 15 and does not constitute a limitation on the terminal device 15. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0180] The processor 150 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0181] In some embodiments, the memory 151 may be an internal storage unit of the terminal device 15, such as a hard disk or memory of the terminal device 15. In other embodiments, the memory 151 may also be an external storage device of the terminal device 15, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the terminal device 15. Further, the memory 151 may also include both an internal storage unit of the terminal device 15 and an external storage device. The memory 151 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program, etc. The memory 151 may also be used to temporarily store data that has been output or is to be output.

[0182] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0183] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0184] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0185] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0186] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0187] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0188] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first image, wherein the first image is an image representing a target body tissue of a patient; Based on the second image, obtaining first displacement distances in multiple directions corresponding to multiple first target points of the target body tissue in the first image, wherein the second image is a reference image corresponding to the target body tissue in the first image, and the multiple directions are multiple directions of the sampling space; For each first target point, based on a first displacement distance corresponding to the first target point, the multiple first target points are mapped to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image.

2. The image processing method according to claim 1, characterized in that: The acquiring, based on the second image, first displacement distances in multiple directions corresponding to each of multiple first target points of the target body tissue in the first image comprises: Inputting a plurality of first target points in the first image into the trained registration model; The first displacement distances in multiple directions corresponding to each of the multiple first target points are generated according to the trained registration model.

3. The image processing method according to claim 2, characterized in that: The method further comprises: Acquire a first sample image, where the first sample image is an image representing a target body tissue of a patient; Acquire a registration model to be trained, wherein the registration model to be trained is used to acquire a second displacement distance that a first sample point in the first sample image needs to move, wherein the second displacement distance is a distance that the first sample point needs to move when mapping the first sample point of the first sample image to a corresponding position of a second sample image, and the second sample image is a reference image corresponding to a target body tissue in the first sample image; Based on the first sample image, the registration model to be trained is adjusted to obtain the trained registration model.

4. The image processing method according to claim 3, characterized in that: The step of adjusting the registration model to be trained to obtain the trained registration model includes: Setting a first function, wherein the first function is used to represent the consistency between the first distances between the plurality of first sample points and the second distances between the plurality of second sample points, wherein the second sample points are sample points in the second sample image when the first sample image is mapped to the same position of the second sample image in the sampling space; Obtaining, by means of the first function, a first output result obtained after inputting a plurality of first sample points of the first sample image into the registration model to be trained, wherein the first output result indicates the second displacement distance that the first sample point needs to move when mapping the first sample point to the second sample image; Acquire a first registration result between the first sample image and the second sample image according to the first output result; The registration model parameters are adjusted according to the first function result until the first registration result is within a preset threshold, and then it is determined that the registration model training is completed, and the trained registration model is obtained.

5. The image processing method according to claim 3 or 4, characterized in that: The step of adjusting the registration model to be trained to obtain the trained registration model includes: Setting a second function, wherein the second function is used to represent the similarity of pixel intensities of the first sample point and the second sample point; Obtaining, by means of the second function, a second output result obtained after inputting the first sample image into the registration model to be trained; Acquire a second registration result between the first sample image and the second sample image according to the second output result; The registration model parameters are adjusted according to the result of the second function until the second registration result is within a preset threshold, and then it is determined that the training of the registration model is completed, and the trained registration model is obtained.

6. The image processing method according to claim 1, characterized in that: The mapping of the plurality of first target points to the same position of the second image in the sampling space based on the first displacement distance corresponding to the first target point to obtain a registration result between the first image and the second image includes: Moving the first target point by a corresponding first displacement distance to obtain a corresponding second target point; The plurality of second target points are mapped to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image.

7. An image processing device, characterized in that: include: A first acquisition module, configured to acquire a first image, wherein the first image is an image representing a target body tissue of a patient; A second acquisition module is used to acquire, based on a second image, first displacement distances in multiple directions corresponding to each of multiple first target points of the target body tissue in the first image, wherein the second image is a reference image corresponding to the target body tissue in the first image, and the multiple directions are multiple directions of a sampling space; A mapping module is used to map the multiple first target points to the same position of the second image in the sampling space based on the first displacement distance corresponding to each first target point, so as to obtain a registration result between the first image and the second image.

8. The image processing device according to claim 7, characterized in that: The mapping module comprises: A moving module, used for moving the first target point by a corresponding first displacement distance to obtain a corresponding second target point; The corresponding module maps the multiple second target points to the same position of the second image in the sampling space to obtain a registration result between the first image and the second image.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images