Breast image registration method, computer device and storage medium

Abstract

The present application relates to a breast image registration method, computer equipment and storage medium. The method comprises: acquiring a first image and a second image; inputting the first image and the second image into a preset registration network, and obtaining a deformation field corresponding to the second image through the registration network; wherein the registration network obtains a registered sample breast image by inputting the first sample image and the second sample image into a preset initial registration network, and obtains a registered sample gland segmentation image according to the registered sample breast image, and obtains a first loss according to the first sample image and the registered sample breast image, obtains a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and trains the initial registration network according to the first loss and the second loss; and obtains a registered breast image according to the second image and the deformation field. The use of this method can reduce the time consumption of breast DCE‑MRI image registration.

Description

Technical Field

[0001] The present application relates to the technical field of image registration, and in particular to a breast image registration method, a computer device and a storage medium. Background Art

[0002] During the breast dynamic enhanced magnetic resonance imaging (DCE-MRI) scanning process, the breast is imaged before, during, and after the contrast agent injection, and multi-layer and multi-phase scanning images can be obtained. Usually, due to the diffusion of contrast agents over time, there are grayscale differences between breast DCE-MRI images at different phases on the same layer, and this difference is most obvious in the lesion area, which will cause inaccurate diagnosis of tissue lesions; in addition, the breast DCE-MRI scanning process lasts for 2 to 3 minutes, and the patient's breathing movement will cause deformation and displacement of breast tissue, resulting in misalignment of internal tissue areas of images at different phases on the same layer, which makes it difficult to analyze and process breast DCE-MRI images. Therefore, before analyzing breast DCE-MRI, breast DCE-MRI images at different phases need to be registered to eliminate the deformation and displacement of breast tissue and the grayscale differences between breast DCE-MRI images at different phases on the same layer.

[0003] In traditional technology, the registration of breast DCE-MRI images is mainly achieved by iteratively optimizing the breast DCE-MRI images. For example, the breast DCE-MRI images are iteratively optimized using models such as the Free From Deformation (FFD), the HS optical flow model, and the Markov random field with a similarity evaluation model to achieve the registration of breast DCE-MRI images.

[0004] However, the traditional breast DCE-MRI image registration method is time-consuming. Summary of the invention

[0005] Based on this, it is necessary to provide a breast image registration method, device, computer equipment and storage medium that can reduce the time consumption of breast DCE-MRI image registration in response to the above technical problems.

[0006] A breast image registration method, the method comprising:

[0007] Acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland;

[0008] Input the first image and the second image into a preset registration network, and obtain the deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: input the first sample image and the second sample image into a preset initial registration network to obtain a registered sample breast image, obtain a registered sample gland segmentation image according to the registered sample breast image, obtain a first loss according to the first sample image and the registered sample breast image, obtain a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and train the initial registration network according to the first loss and the second loss;

[0009] A registered breast image is obtained according to the second image and the deformation field.

[0010] In one embodiment, the first image and the second image are dynamic enhanced magnetic resonance images of the breast at different scanning phases in the same body position of the subject; before inputting the first image and the second image into a preset registration network and obtaining the deformation field corresponding to the second image through the registration network, the method further includes:

[0011] Pre-registering the second image and the first image through three-dimensional affine transformation to obtain a first registered image;

[0012] The step of inputting the first image and the second image into a preset registration network and obtaining a deformation field corresponding to the second image through the registration network includes:

[0013] The first image and the first registered image are input into the registration network, and the deformation field is obtained through the registration network.

[0014] In one embodiment, the method further comprises:

[0015] Acquire a first gland mask image and a second gland mask image; wherein the first gland mask image is a gland mask image corresponding to the first image, and the second gland mask image is a gland mask image corresponding to the second image;

[0016] Pre-registering the first gland mask image and the second gland mask image through three-dimensional affine transformation to obtain a second registered image;

[0017] A registered gland image is obtained according to the second registered image and the deformation field.

[0018] In one embodiment, obtaining a registered breast image according to the second image and the deformation field includes:

[0019] The second image and the deformation field are input into a space transformer model, and the registered breast image is obtained through the space transformer model.

[0020] In one embodiment, the space transformer model includes a grid generator and a sampler, and the inputting the second image and the deformation field into the space transformer model to obtain the registered breast image through the space transformer model includes:

[0021] Inputting the deformation field into the grid generator to obtain a first mapping relationship between each voxel of the second image and each voxel of the registered breast image;

[0022] The first mapping relationship and the second image are input into the sampler, and the registered breast image is obtained through the sampler.

[0023] In one embodiment, obtaining a registered gland image according to the second registered image and the deformation field includes:

[0024] The second registered image and the deformation field are input into a space transformer model, and the registered gland image is obtained through the space transformer model.

[0025] In one embodiment, the space transformer model includes a grid generator and a sampler, and the inputting the second registered image and the deformation field into the space transformer model to obtain the registered gland image through the space transformer model includes:

[0026] Inputting the deformation field into the grid generator to obtain a second mapping relationship between each voxel of the second registered image and each voxel of the registered gland image;

[0027] The second mapping relationship and the second registered image are input into the sampler, and the registered gland image is obtained through the sampler.

[0028] In one embodiment, the loss of the initial registration network further includes a smoothing loss, and the training process of the registration network includes:

[0029] Pre-registering the first sample image and the second sample image through three-dimensional affine transformation to obtain a first sample registered image;

[0030] Pre-registering the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image through three-dimensional affine transformation to obtain a second sample registered image;

[0031] Inputting the first sample image and the first sample registered image into the initial registration network, and obtaining the sample deformation field and the smoothing loss corresponding to the first sample registered image through the initial registration network;

[0032] Inputting the first sample registered image and the sample deformation field into the space transformer model, and obtaining a registered sample breast image through the space transformer model;

[0033] Inputting the second sample registered image and the sample deformation field into the space transformer model, and obtaining the registered sample gland segmentation image through the space transformer model;

[0034] Obtaining the first loss according to the registered sample breast image and the first sample image;

[0035] Obtaining the second loss according to the registered sample gland segmentation image and the segmentation image corresponding to the first sample image;

[0036] The initial registration network is trained according to the smoothing loss, the first loss and the second loss to obtain the registration network.

[0037] A breast image registration device, comprising:

[0038] A first acquisition module is used to acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland;

[0039] A second acquisition module is used to input the first image and the second image into a preset registration network, and obtain the deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: input the first sample image and the second sample image into a preset initial registration network to obtain a registered sample breast image, obtain a registered sample gland segmentation image according to the registered sample breast image, obtain a first loss according to the first sample image and the registered sample breast image, obtain a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and train the initial registration network according to the first loss and the second loss;

[0040] The first registration module is used to obtain a registered breast image according to the second image and the deformation field.

[0041] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0042] Acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland;

[0043] Input the first image and the second image into a preset registration network, and obtain the deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: input the first sample image and the second sample image into a preset initial registration network to obtain a registered sample breast image, obtain a registered sample gland segmentation image according to the registered sample breast image, obtain a first loss according to the first sample image and the registered sample breast image, obtain a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and train the initial registration network according to the first loss and the second loss;

[0044] A registered breast image is obtained according to the second image and the deformation field.

[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0046] Acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland;

[0047] Input the first image and the second image into a preset registration network, and obtain the deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: input the first sample image and the second sample image into a preset initial registration network to obtain a registered sample breast image, obtain a registered sample gland segmentation image according to the registered sample breast image, obtain a first loss according to the first sample image and the registered sample breast image, obtain a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and train the initial registration network according to the first loss and the second loss;

[0048] A registered breast image is obtained according to the second image and the deformation field.

[0049] The above-mentioned breast image registration method, device, computer equipment and storage medium can obtain a registered sample breast image by inputting the first sample image and the second sample image into a preset initial registration network, so that a registered sample gland segmentation image can be obtained according to the registered sample breast image, and a first loss is obtained according to the first sample image and the registered sample breast image, and a second loss is obtained according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and then the initial registration network is trained according to the first loss and the second loss to obtain a registration network, so that the acquired first image and the second image are input into the trained registration network, and the deformation field corresponding to the second image can be quickly obtained through the registration network. Compared with the traditional method of obtaining a registered breast image by iteratively optimizing a breast dynamic enhanced magnetic resonance image, the efficiency of obtaining the deformation field corresponding to the second image is improved through the registration network. Since the efficiency of obtaining the deformation field corresponding to the second image is improved, the efficiency of obtaining the registered breast image according to the second image and the deformation field is improved, and the time for obtaining the registered breast image is shortened. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A diagram of an application environment of a breast image registration method in one embodiment;

[0051] Figure 2 A schematic diagram of a flow chart of a breast image registration method in one embodiment;

[0052] Figure 2a A schematic diagram of the structure of a registration network in one embodiment;

[0053] Figure 3 A schematic diagram of a flow chart of a breast image registration method in one embodiment;

[0054] Figure 4 A schematic diagram of the structure of a space converter in one embodiment;

[0055] Figure 5 A schematic diagram of a flow chart of a breast image registration method in one embodiment;

[0056] Figure 6 A schematic diagram of a registration result obtained by using the breast image registration method of the present application in one embodiment;

[0057] Figure 7 A schematic diagram of a registration result obtained by using the breast image registration method of the present application in one embodiment;

[0058] Figure 8 A schematic diagram showing comparison of registration results with and without the gland mask-assisted optimization loss function in one embodiment;

[0059] Fig. 9 FIG. 4 is a structural block diagram of a breast image registration device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] The breast image registration method provided in the embodiment of the present application can be applied to Figure 1 The computer device shown. The computer device includes a processor and a memory connected by a system bus, and a computer program is stored in the memory. When the processor executes the computer program, the steps of the following method embodiment can be executed. Optionally, the computer device may also include a network interface, a display screen and an input device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory, and the non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. Optionally, the computer device can be a server, a personal computer, a personal digital assistant, or other terminal devices, such as a tablet computer, a mobile phone, etc., or a cloud or remote server. The embodiment of the present application does not limit the specific form of the computer device.

[0062] In one embodiment, Figure 2 As shown, a breast image registration method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0063] S201, acquiring a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland.

[0064] Among them, the breast dynamic scanning image is obtained by dynamically scanning the breast of the tester. Optionally, the breast dynamic scanning image can be a breast dynamic enhanced magnetic resonance image. Usually, during the dynamic enhanced magnetic resonance scanning process, the diffusion of the contrast agent will affect the nuclear magnetic signal and cause changes in the brightness of the dynamic enhanced magnetic resonance image. The breast is imaged before, during and after the contrast agent injection, and multi-layer and multi-phase scanning images can be obtained. Specifically, the computer device obtains a first image and a second image, wherein the first image and the second image are both breast dynamic scanning images, and the breast dynamic scanning image includes a breast and a gland. Optionally, the computer device can obtain the above-mentioned first image and second image from a magnetic resonance device, and can also obtain the above-mentioned first image and second image from a PACS (Picture Archiving and Communication Systems) server.

[0065] S202, inputting the first image and the second image into a preset registration network, and obtaining a deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: inputting the first sample image and the second sample image into a preset initial registration network to obtain a registered sample breast image, obtaining a registered sample gland segmentation image based on the registered sample breast image, and obtaining a first loss based on the first sample image and the registered sample breast image, and obtaining a second loss based on the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and training the initial registration network based on the first loss and the second loss.

[0066] Specifically, the computer device inputs the acquired first image and second image into a preset registration network, and obtains the deformation field corresponding to the second image through the registration network. The registration network is obtained by the following training method: the first sample image and the second sample image are input into a preset initial registration network to obtain a registered sample breast image, a registered sample gland segmentation image is obtained according to the registered sample breast image, and a first loss is obtained according to the first sample image and the registered sample breast image, and a second loss is obtained according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and the initial registration network is trained according to the first loss and the second loss. Optionally, the registered sample gland segmentation image can be a gland mask image corresponding to the registered sample breast image, or it can be a gland segmentation image in the registered sample breast image. Optionally, the network structure diagram of the registration network can be as shown below. Figure 2aAs shown, usually, the input of the registration network is a dual-channel 3D image formed by splicing the first image and the second image. The size of the input image is generally 2×256×256×96. The encoding path contains several blocks, each of which consists of two 3×3×3 convolutional layers and a 2×2×2 maximum pooling layer. Each convolutional layer is connected to a normalization layer and a leakage-corrected linear unit (slope is 0.2). The convolution step is 2, and the size of the feature map of the smallest layer in each direction is 1 / 16 of the size of the input image in each direction. The decoding path also contains several blocks, each of which is similar to the block composition of the same level in the encoding stage. The upsampling is done by 2×2 ×2 deconvolution is implemented, the jump connection between the encoding path and the decoding path represents the concatenation of the upsampled feature map with the corresponding downsampled feature map, the features learned in the upsampling stage are directly propagated to the layer used to generate the deformation field, and the hierarchical features of the input image pair are obtained in the convolution layer to estimate the corresponding deformation field. The last layer of the decoding path maps the feature map to the output layer of the entire network through a 1×1×1 convolution, and generates a displacement field with the same size as the input image in the three directions of x, y, and z. The displacement field represents the displacement of each voxel in each direction (x, y, z), and the size of the output displacement field is 3×256×256×96.

[0067] S203: Obtain a registered breast image according to the second image and the deformation field.

[0068] Specifically, the computer device obtains a registered breast image based on the second image and the obtained deformation field. Optionally, the computer device may apply the deformation field to the second image to obtain a registered breast image, or may input the deformation field and the second image into a preset spatial transformation network to obtain a registered breast image.

[0069] In the above-mentioned breast image registration method, by inputting the first sample image and the second sample image into a preset initial registration network, a registered sample breast image can be obtained, so that a registered sample gland segmentation image can be obtained according to the registered sample breast image, and a first loss is obtained according to the first sample image and the registered sample breast image, and a second loss is obtained according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and then the initial registration network can be trained according to the first loss and the second loss to obtain a registration network, so that the acquired first image and the second image are input into the trained registration network, and the deformation field corresponding to the second image can be quickly obtained through the registration network. Compared with the traditional method of obtaining a registered breast image by iteratively optimizing a breast dynamic enhanced magnetic resonance image, the efficiency of obtaining the deformation field corresponding to the second image is improved through the registration network. Since the efficiency of obtaining the deformation field corresponding to the second image is improved, the efficiency of obtaining the registered breast image according to the second image and the deformation field is improved, and the time for obtaining the registered breast image is shortened.

[0070] In the above scenario of acquiring the first image and the second image, the first image and the second image are dynamic enhanced magnetic resonance images of the breast at different scanning phases in the same body position of the subject. In one embodiment, before S202, the method further includes: pre-registering the second image and the first image by three-dimensional affine transformation to obtain a first registered image.

[0071] Specifically, the computer device pre-registers the second image and the first image through a three-dimensional affine transformation to obtain a first registered image, and the above S202 includes: inputting the first image and the obtained first registered image into the above registration network, and obtaining the above deformation field through the registration network. It should be noted that pre-registering the second image and the first image through a three-dimensional affine transformation can eliminate the displacement change caused by the breathing movement of the tester, thereby reducing the impact of registering the second image and the first image.

[0072] In this embodiment, the computer device pre-registers the acquired second image and the first image through three-dimensional affine transformation, and the obtained first registered image can eliminate the displacement changes of the second image and the first image caused by the respiratory movement of the tester, thereby reducing the impact of the displacement changes on the registration of the second image and the first image.

[0073] In some scenarios, after the computer device obtains the registered breast image through the second image and the first image, it is also necessary to register the glands in the breast image to obtain the registered gland image. Figure 3 As shown, the above method also includes:

[0074] S301, obtaining a first gland mask image and a second gland mask image; wherein the first gland mask image is a gland mask image corresponding to the first image, and the second gland mask image is a gland mask image corresponding to the original second image.

[0075] Specifically, the computer device obtains a first gland mask image and a second gland mask image, wherein the first gland mask image is a gland mask image corresponding to the first image, and the second gland mask image is a gland mask image corresponding to the original second image. Optionally, the computer device may input the first image into a preset segmentation network, and segment the first image through the segmentation network to obtain the first gland mask image, or annotate the gland mask in the first image, and segment the annotated gland mask image to obtain the first gland mask image. Optionally, the computer device may input the second image into the preset segmentation network, and segment the second image through the segmentation network to obtain the second gland mask image, or annotate the gland mask in the second image, and segment the annotated gland mask image to obtain the second gland mask image.

[0076] S302: pre-register the first gland mask image and the second gland mask image through three-dimensional affine transformation to obtain a second registered image.

[0077] Specifically, the computer device pre-registers the first gland mask image and the second gland mask image through three-dimensional affine transformation to obtain a second registered image. It should be noted that pre-registering the first gland mask image and the second gland mask image through three-dimensional affine transformation can eliminate the displacement change caused by the respiratory movement of the tester, thereby reducing the impact of registering the first gland mask image and the second gland mask image.

[0078] S303: Obtain a registered gland image according to the second registered image and the deformation field.

[0079] Specifically, the computer device obtains the registered gland image according to the second registered image obtained above and the deformation field obtained above. Optionally, the computer device may apply the deformation field to the second registered image to obtain the registered gland image, or may input the deformation field and the second registered image into a preset spatial transformation network to obtain the registered gland image.

[0080] In this embodiment, the computer device pre-registers the first gland mask image corresponding to the first image and the second gland mask image corresponding to the second image through three-dimensional affine transformation, which can eliminate the displacement changes caused by the respiratory movement of the tester, thereby reducing the impact of the registration of the first gland mask image and the second gland mask image, and improving the accuracy of the obtained second registered image. Therefore, the second registered image can be accurately registered according to the second registered image and the deformation field, thereby improving the accuracy of the obtained registered gland image.

[0081] In the above scenario where the registered breast image is obtained based on the second image and the deformation field, in one embodiment, the above S203 includes: inputting the second image and the deformation field into a space transformer model, and obtaining the registered breast image through the space transformer model.

[0082] Specifically, the computer device inputs the second image and the deformation field obtained above into the space transformer model, and obtains the registered breast image through the space transformer model. Optionally, the space transformer model includes a grid generator and a sampler. Optionally, as Figure 4 As shown, the computer device can input the deformation field into the grid generator of the space transformer model to obtain the first mapping relationship between each voxel of the second image and each voxel of the registered breast image. Further, the computer device inputs the first mapping relationship and the second image into the sampler of the space transformer model to obtain the registered breast image through the sampler. It should be noted that the grid generator obtains a mapping relationship T θ (G), assuming that the coordinates of each voxel in the second image are The coordinates of each voxel in the registered breast image are Spatial transformation function T θ (G) is the three-dimensional affine transformation function, and The corresponding relationship can be expressed as:

[0083]

[0084] Optionally, the computer device inputs the first mapping relationship and the second image into a sampler, and the registered breast image obtained by the sampler may be sampled by using an 8-neighborhood linear interpolation method, as shown in the following formula: Among them, for each voxel p, p′=p+u(p), Z(p′) represents the neighborhood voxel of p′, and d represents the spatial dimension, which is convenient for calculating the gradient of the loss function when optimizing the loss function during the back propagation process.

[0085] In this embodiment, the computer device inputs the second image and the deformation field corresponding to the second image into the space transformer model, and the registered breast image can be quickly obtained through the space transformer model, thereby improving the efficiency of obtaining the registered breast image.

[0086] In the above scenario where the registered gland image is obtained based on the second registered image and the deformation field, in one embodiment, the above S303 includes: inputting the second registered image and the deformation field into the space transformer model, and obtaining the registered gland image through the space transformer model.

[0087] Specifically, the computer device inputs the above-mentioned second registered image and the above-mentioned deformation field into the space transformer model, and obtains the registered gland image through the space transformer model. Optionally, the space transformer model includes a grid generator and a sampler. Optionally, the computer device inputs the above-mentioned deformation field into the above-mentioned grid generator, obtains the second mapping relationship between each voxel of the above-mentioned second registered image and each voxel of the registered gland image, and further inputs the obtained second mapping relationship and the above-mentioned second registered image into the sampler, and obtains the registered gland image through the sampler. It should be noted that the detailed process and principle of the grid generator obtaining the above-mentioned second mapping relationship and obtaining the registered gland image through the sampler are the same as the process and principle of obtaining the above-mentioned first mapping relationship and obtaining the registered breast image through the sampler, and this embodiment will not be repeated here.

[0088] In this embodiment, the computer device inputs the second registered image and the deformation field corresponding to the second registered image into the space transformer model, and the registered gland image can be quickly obtained through the space transformer model, thereby improving the efficiency of obtaining the registered gland image.

[0089] In the above scenario where the first image and the second image are input into a preset registration network and the deformation field corresponding to the second image is obtained through the registration network, the registration network is a pre-trained network. In one embodiment, Figure 5 As shown, the loss of the above initial registration network also includes smoothing loss, and the training process of the above registration network includes:

[0090] S501 , pre-registering a first sample image and a second sample image through three-dimensional affine transformation to obtain a first sample registered image.

[0091] Specifically, the computer device pre-registers the acquired first sample image and the second sample image through a three-dimensional affine transformation to obtain a first sample registered image. It should be noted that pre-registering the first sample image and the second sample image through a three-dimensional affine transformation can eliminate the displacement change caused by the breathing movement of the tester, thereby reducing the impact of registering the first sample image and the second sample image.

[0092] S502 , pre-registering the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image through three-dimensional affine transformation to obtain a second sample registered image.

[0093] Specifically, the computer device pre-registers the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image through three-dimensional affine transformation to obtain a second sample registered image. It should be noted that pre-registering the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image through three-dimensional affine transformation can eliminate the displacement change caused by the breathing movement of the tester, thereby reducing the impact of registering the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image.

[0094] S503: Input the first sample image and the first sample registered image into an initial registration network, and obtain a sample deformation field and a smoothing loss corresponding to the first sample registered image through the initial registration network.

[0095] Specifically, the computer device inputs the first sample image and the obtained first sample registration image into the initial registration network, and obtains the sample deformation field and smoothness loss corresponding to the first sample registration image through the initial registration network. It should be noted that the smoothness loss is the smoothness loss L of the sample deformation field itself. smooth (φ). It can be understood that the network structure of the initial registration network is the same as the network structure of the above registration network. For detailed description of the network structure of the initial registration network, please refer to the description of the above embodiment. This embodiment will not repeat the network structure of the initial registration network.

[0096] S504: Input the first sample registered image and the sample deformation field into the space transformer model, and obtain the registered sample breast image through the space transformer model.

[0097] Specifically, the computer device inputs the first sample registration image and the sample deformation field obtained above into the space transformer model, and obtains the registered sample breast image through the space transformer model. Optionally, the space transformer model includes a grid generator and a sampler. Optionally, the computer device inputs the sample deformation field obtained above into the grid generator, obtains the mapping relationship between each voxel of the first sample registration image and each voxel of the registered sample breast image, and further inputs the obtained mapping relationship and the first sample registration image into the sampler, and obtains the registered sample breast image through the sampler. It should be noted that the detailed process and principle of the grid generator obtaining the mapping relationship and obtaining the registered sample breast image through the sampler are the same as the process and principle of obtaining the first mapping relationship and obtaining the registered breast image through the sampler, and this embodiment will not be repeated here.

[0098] S505 , inputting the second sample registered image and the sample deformation field into the space transformer model, and obtaining the registered sample gland mask image through the space transformer model.

[0099] Specifically, the computer device inputs the second sample registration image and the sample deformation field into the space transformer model, and obtains the registered sample gland mask image through the space transformer model. Optionally, the space transformer model includes a grid generator and a sampler. Optionally, the computer device inputs the sample deformation field obtained above into the grid generator, obtains the mapping relationship between each voxel of the second sample registration image and each voxel of the registered sample gland mask image, and further inputs the obtained mapping relationship and the second sample registration image into the sampler, and obtains the registered sample gland mask image through the sampler. It should be noted that the detailed process and principle of the grid generator obtaining the mapping relationship and obtaining the registered sample gland mask image through the sampler are the same as the process and principle of obtaining the first mapping relationship and obtaining the registered breast image through the sampler, and this embodiment will not be repeated here.

[0100] S506: Obtain a first loss according to the registered sample breast image and the first sample image.

[0101] Specifically, the computer device uses the registered sample breast image And the first sample image f, we get the first loss It should be noted that, due to the obvious change in the grayscale value of the breast dynamic enhanced MRI image before and after the injection of contrast agent, the grayscale value distribution of the first sample image f and the second sample image m is no longer similar. Therefore, the first sample image f and the registered sample breast image m are used. The local correlation between them is used as the similarity measure, where the local correlation calculation formula is: In the formula, and Respectively represent the first sample image f and the registered sample breast image The larger the CC value, the stronger the correlation between the registered image and the first sample image, and the better the registration effect. Therefore, the first loss is: When the similarity loss is minimized, the registered image is closer to the first sample image. However, the deformation field may be discontinuous and inconsistent with the actual deformation. Therefore, a diffusion constraint (Diffusion Regularizer) is applied to the spatial gradient of the displacement field u to smooth the deformation field φ, as shown in the following formula: Where Ω is the gland voxel set. It should be noted that the first loss The value of may not be limited to the first sample image f and the registered sample breast image The local correlation between them is obtained as a similarity measure.

[0102] S507 , obtaining a second loss according to the registered sample gland segmentation image and the segmentation image corresponding to the first sample image.

[0103] Specifically, the computer device segments the image according to the registered sample glands. The segmented image s corresponding to the first sample image f , and get the second loss Optionally, the second loss can be defined as follows: In the formula, Dice is used to measure the gland segmentation image s corresponding to the first sample image f And the registered sample gland segmentation image The overlap of the mid-glandular area is calculated as follows:

[0104]

[0105] It should be noted that in the calculation Before, the gland binary mask image needs to be linearly interpolated to make Differentiable during back-propagation.

[0106] S508, training the initial registration network according to the smoothing loss, the first loss and the second loss to obtain the registration network.

[0107] Specifically, the computer device trains the initial registration network according to the smoothing loss, the first loss, and the second loss obtained above to obtain the registration network. Optionally, the computer device may train the initial registration network according to the smoothing loss L smooth (φ) value, first loss The value and the second loss The value of , and the loss function of the initial registration network is obtained Wherein, λ and γ are regularization parameters. It should be noted that the regularization of the smoothing loss and the second loss is because optimizing the similarity loss of the gland mask image will cause discontinuity in the deformation field, so regularization is required. Optionally, the computer device can determine the initial registration network corresponding to the minimum or stable value of the loss function of the initial registration network as the above-mentioned registration network. It should be noted that during the training of the initial registration network, the batch size can be set to 1, and each epoch is defined as iterative optimization on 100 batches, and the maximum value is set to 700. Optionally, the adaptive moment estimation method (Adaptive Moment Estimation, Adam) can be used to optimize the loss function, using the initialization parameters of the normal distribution random initial registration network, the initial learning rate is set to 1e-4, and the regularization coefficients λ and γ can be set to 1.5 and 1.8 respectively after grid search.

[0108] In this embodiment, the computer device pre-registers the first sample image and the second sample image through three-dimensional affine transformation to obtain a first sample registered image, and pre-registers the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image through three-dimensional affine transformation to obtain a second sample registered image, which can reduce the displacement change caused by the respiratory movement of the tester, so that the first sample image and the first sample registered image can be input into the initial registration network, and the sample deformation field and smooth loss corresponding to the first sample registered image can be accurately obtained through the initial registration network, and then the first sample registered image and the sample deformation field can be input into the space transformer model, and the registered sample breast image can be accurately obtained through the space transformer model, and the second sample registered image and the sample deformation field can be input into the space transformer model, and the registered sample gland segmentation image can be accurately obtained through the space transformer model. Further, the first loss can be obtained according to the registered sample breast image and the first sample image, and the second loss can be obtained according to the registered sample gland segmentation image and the segmentation image corresponding to the first sample image, and then the initial registration network can be accurately trained according to the smooth loss, the first loss and the second loss to obtain the registration network, thereby improving the accuracy of the obtained registration network.

[0109] Exemplarily, in one embodiment, the above registration model is tested and the average Dice similarity coefficient on the test data set is 0.831±0.019. Figure 6 , Figure 7The following are schematic diagrams of the image registration results of breast dynamic enhanced magnetic resonance images of mass-type cases and microcalcification-type cases. By visualizing the deformation field grid, it can be observed that the distorted part of the deformation field is concentrated in the glandular area in the breast, and the glandular area basically overlaps after registration. By comparing the difference image of the second image and the first image before and after registration, it can be seen that the contours of tissues and lesions in the difference image before registration are relatively blurred, while the image after registration basically overlaps with the first image, and the contours of the lesions become clear. Figure 7 Shown is a schematic diagram of tiny calcification foci. The lesion in the right breast is almost invisible in the difference image before registration, while the lesion is highlighted in the difference image after registration, which indirectly proves the effectiveness of the registration and realizes the alignment of the physical spatial positions of voxels within the ROI (gland) with diagnostic significance, achieving the expected goal.

[0110] Furthermore, by training the registration models with and without the gland mask assisted optimization loss function (Auxiliary Fibroglandular Tissues Mask Optimization, AFMO) respectively, and evaluating the registration performance on the same test set, the average Dice of the registration results with and without the AFMO model were 0.831 ± 0.019 and 0.793 ± 0.036 (p-value = 0.016 < 0.05), respectively, indicating that the registration results using AFMO have higher gland similarity. Figure 8 The registration results of four representative DCE-MR image samples with and without AFMO are compared. Each column in the figure represents a different DCE-MR image sample. The first column is a sample with no lesions and low breast density, and the second to fourth columns are samples with lesions. It can be seen that for cases without lesions and low breast density ( Figure 8 The Dice of the registration results with and without AFMO are 0.812 and 0.825, respectively, which is not much different. For cases with lesions, the overlap of the glandular area in the registration results with AFMO is higher than that without AFMO, and the lesion area is highlighted in the registration results with AFMO. In the registration results without AFMO, although the breast area overlaps better, the edge contour of the lesion is weakened ( Figure 8The second column), or the entire lesion is blurred or even disappears (the third and fourth columns of Figure 8), which will bring difficulties to the analysis of lesions and glands. Therefore, using AFMO in the registration process better retains the information of the glandular area and BPE tissue in the image, and improves the effectiveness of glandular area registration. In addition, the performance of image registration by affine alignment, SimpleElastix toolkit, ANTs SyN toolkit, AFMO-free model and the registration model in this application is compared, and the average Dice value of the registration model of this application on the same registration test set is optimal, and a good balance is achieved between registration effectiveness and real-time performance.

[0111] It should be understood that although Figure 2-5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0112] In one embodiment, Fig. 9 As shown, a breast image registration device is provided, comprising: a first acquisition module, a second acquisition module and a first registration module, wherein:

[0113] The first acquisition module is used to acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; and the breast dynamic scanning image includes a breast and a gland.

[0114] The second acquisition module is used to input the first image and the second image into a preset registration network, and obtain the deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: the first sample image and the second sample image are input into a preset initial registration network to obtain a registered sample breast image, a registered sample gland segmentation image is obtained according to the registered sample breast image, a first loss is obtained according to the first sample image and the registered sample breast image, a second loss is obtained according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and the initial registration network is trained according to the first loss and the second loss.

[0115] The first registration module is used to obtain a registered breast image according to the second image and the deformation field.

[0116] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0117] On the basis of the above embodiment, optionally, the first image and the second image are dynamic enhanced magnetic resonance images of the breast of the examinee in the same position but different scanning phases, and the above device further includes: a second registration module, wherein:

[0118] The second registration module is used to pre-register the second image and the first image through three-dimensional affine transformation to obtain a first registered image.

[0119] The second acquisition module is used to input the first image and the first registered image into the registration network, and obtain the deformation field through the registration network.

[0120] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0121] On the basis of the above embodiment, optionally, the above device further includes: a third acquisition module, a third registration module and a fourth registration module, wherein:

[0122] The third acquisition module is used to acquire a first gland mask image and a second gland mask image; wherein the first gland mask image is a gland mask image corresponding to the first image, and the second gland mask image is a gland mask image corresponding to the second image.

[0123] The third registration module is used to pre-register the first gland mask image and the second gland mask image through three-dimensional affine transformation to obtain a second registered image.

[0124] The fourth registration module is used to obtain a registered gland image according to the second registered image and the deformation field.

[0125] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0126] Based on the above embodiment, optionally, the above first registration module includes: a first registration unit, wherein:

[0127] The first registration unit is used to input the second image and the deformation field into the space transformer model, and obtain the registered breast image through the space transformer model.

[0128] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0129] Based on the above embodiment, optionally, the space transformer model includes a grid generator and a sampler, and the above-mentioned first registration unit is used to input the deformation field into the grid generator to obtain a first mapping relationship between each voxel of the second image and each voxel of the registered breast image; input the first mapping relationship and the second image into the sampler to obtain the registered breast image through the sampler.

[0130] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0131] Based on the above embodiment, optionally, the fourth registration module includes: a second registration unit, wherein:

[0132] The second registration unit is used to input the second registered image and the deformation field into the space transformer model, and obtain the registered gland image through the space transformer model.

[0133] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0134] Based on the above embodiment, optionally, the space transformer model includes a grid generator and a sampler, and the above-mentioned second registration unit is used to input the deformation field into the grid generator to obtain a second mapping relationship between each voxel of the second registration image and each voxel of the registered gland image; the second mapping relationship and the second registration image are input into the sampler to obtain the registered gland image through the sampler.

[0135] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0136] On the basis of the above embodiment, optionally, the loss of the initial registration network further includes a smoothing loss, and the above device further includes: a fifth registration module, a sixth registration module, a fourth acquisition module, a fifth acquisition module, a sixth acquisition module, a seventh acquisition module, an eighth acquisition module and a training module, wherein:

[0137] The fifth registration module is used to pre-register the first sample image and the second sample image through three-dimensional affine transformation to obtain a first sample registered image.

[0138] The sixth registration module is used to pre-register the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image through three-dimensional affine transformation to obtain a second sample registered image.

[0139] The fourth acquisition module is used to input the first sample image and the first sample registration image into the initial registration network, and obtain the sample deformation field and smoothing loss corresponding to the first sample registration image through the initial registration network.

[0140] The fifth acquisition module is used to input the first sample registration image and the sample deformation field into the space transformer model, and obtain the registered sample breast image through the space transformer model.

[0141] The sixth acquisition module is used to input the second sample registration image and the sample deformation field into the space transformer model, and obtain the registered sample gland segmentation image through the space transformer model.

[0142] The seventh acquisition module is used to obtain the first loss according to the registered sample breast image and the first sample image.

[0143] The eighth acquisition module is used to obtain the second loss according to the registered sample gland segmentation image and the segmentation image corresponding to the first sample image.

[0144] The training module is used to train the initial registration network according to the smoothing loss, the first loss and the second loss to obtain the registration network.

[0145] The breast image registration device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0146] For the specific definition of the breast image registration device, please refer to the definition of the breast image registration method above, which will not be repeated here. The various modules in the above-mentioned breast image registration device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0148] Acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland;

[0149] Input the first image and the second image into a preset registration network, and obtain a deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: input the first sample image and the second sample image into a preset initial registration network to obtain a registered sample breast image, obtain a registered sample gland segmentation image according to the registered sample breast image, obtain a first loss according to the first sample image and the registered sample breast image, obtain a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and train the initial registration network according to the first loss and the second loss;

[0150] A registered breast image is obtained according to the second image and the deformation field.

[0151] The implementation principle and technical effect of the computer device provided in the above embodiment are similar to those of the above method embodiment, and will not be repeated here.

[0152] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0153] Acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland;

[0154] Input the first image and the second image into a preset registration network, and obtain a deformation field corresponding to the second image through the registration network; wherein the registration network is obtained by the following training method: input the first sample image and the second sample image into a preset initial registration network to obtain a registered sample breast image, obtain a registered sample gland segmentation image according to the registered sample breast image, obtain a first loss according to the first sample image and the registered sample breast image, obtain a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, and train the initial registration network according to the first loss and the second loss;

[0155] A registered breast image is obtained according to the second image and the deformation field.

[0156] The computer-readable storage medium provided in the above embodiment has similar implementation principles and technical effects as those of the above method embodiment, and will not be described in detail here.

[0157] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0158] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A breast image registration method, characterized in that: The method comprises: Acquire a first image and a second image; wherein the first image and the second image are both breast dynamic scanning images; the breast dynamic scanning image includes a breast and a gland; The first image and the second image are input into a preset registration network, and the deformation field corresponding to the second image is obtained through the registration network; wherein the registration network is obtained through the following training method: the first sample image and the second sample image are pre-registered through three-dimensional affine transformation to obtain a first sample registered image; the gland mask image corresponding to the first sample image and the gland mask image corresponding to the second sample image are pre-registered through three-dimensional affine transformation to obtain a second sample registered image; the first sample image and the first sample registered image are input into a preset initial registration network, and the sample deformation field corresponding to the first sample registered image is obtained through the initial registration network and smoothing loss; inputting the first sample registration image and the sample deformation field into the space transformer model, obtaining the registered sample breast image through the space transformer model; inputting the second sample registration image and the sample deformation field into the space transformer model, obtaining the registered sample gland segmentation image through the space transformer model, and obtaining a first loss according to the first sample image and the registered sample breast image, obtaining a second loss according to the segmentation image corresponding to the first sample image and the registered sample gland segmentation image, training the initial registration network according to the smoothing loss, the first loss and the second loss to obtain the registration network; A registered breast image is obtained according to the second image and the deformation field.

2. The method according to claim 1, characterized in that The first image and the second image are dynamic enhanced magnetic resonance images of the breast at different scanning phases in the same body position of the examinee; before inputting the first image and the second image into a preset registration network and obtaining the deformation field corresponding to the second image through the registration network, the method further includes: Pre-registering the second image and the first image through three-dimensional affine transformation to obtain a first registered image; The step of inputting the first image and the second image into a preset registration network and obtaining a deformation field corresponding to the second image through the registration network includes: The first image and the first registered image are input into the registration network, and the deformation field is obtained through the registration network.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquire a first gland mask image and a second gland mask image; wherein the first gland mask image is a gland mask image corresponding to the first image, and the second gland mask image is a gland mask image corresponding to the second image; Pre-registering the first gland mask image and the second gland mask image through three-dimensional affine transformation to obtain a second registered image; A registered gland image is obtained according to the second registered image and the deformation field.

4. The method according to claim 1, characterized in that: The step of obtaining a registered breast image according to the second image and the deformation field comprises: The second image and the deformation field are input into a space transformer model, and the registered breast image is obtained through the space transformer model.

5. The method according to claim 4, characterized in that The space transformer model includes a grid generator and a sampler, and the second image and the deformation field are input into the space transformer model to obtain the registered breast image through the space transformer model, including: Inputting the deformation field into the grid generator to obtain a first mapping relationship between each voxel of the second image and each voxel of the registered breast image; The first mapping relationship and the second image are input into the sampler, and the registered breast image is obtained through the sampler.

6. The method according to claim 3, characterized in that The step of obtaining a registered gland image according to the second registered image and the deformation field comprises: The second registered image and the deformation field are input into a space transformer model, and the registered gland image is obtained through the space transformer model.

7. The method according to claim 6, characterized in that The space transformer model includes a grid generator and a sampler, and the second registered image and the deformation field are input into the space transformer model to obtain the registered gland image through the space transformer model, including: Inputting the deformation field into the grid generator to obtain a second mapping relationship between each voxel of the second registered image and each voxel of the registered gland image; The second mapping relationship and the second registered image are input into the sampler, and the registered gland image is obtained through the sampler.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images