A method and apparatus for bone micro-CT image reconstruction

Abstract

The application discloses a bone micro-CT image reconstruction method and device, and belongs to the field of micro-imaging. Through steps of time domain visual basic model and frequency domain visual basic model pre-training, high-dimensional feature coding coordinate sampling, training continuity representation function, and super-resolution reasoning and image reconstruction on the multiple bone sparse projection images, high resolution and fast imaging of the bone micro-CT can be realized simultaneously in the sparse sampling mode, and the radiation dose damage of more than 80% of patients is reduced.

Description

Technical Field

[0001] This invention relates to the field of microscopic imaging, and in particular to a method and apparatus for reconstructing bone micro-CT images. Background Technology

[0002] Bone micro-CT is an in vivo microscopic imaging system specifically designed for joints, bones, and their internal microstructures. It enables precise diagnosis of osteoporosis and assessment of fracture risk, as well as evaluation of osteoarthritis and sports injuries.

[0003] Because the scanned object is a large, high-density bone structure, the scanning imaging speed is greatly limited by the power and focal size of the X-ray source, as well as the requirement for an extremely high sampling rate. Excessive scanning time increases the radiation dose received by the patient or subject and may also increase the likelihood of patient movement, leading to motion artifacts in the reconstructed images. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, one of the objectives of this invention is to provide a bone micro-CT image reconstruction method that can simultaneously achieve high resolution and fast imaging in bone micro-CT under sparse sampling mode.

[0005] In order to overcome the shortcomings of the prior art, the second objective of this invention is to provide a bone micro-CT image reconstruction device that can simultaneously achieve high resolution and fast imaging in bone micro-CT under sparse sampling mode.

[0006] One of the objectives of this invention is achieved through the following technical solution:

[0007] A method for bone micro-CT image reconstruction includes the following steps:

[0008] Pre-training of the temporal-domain visual foundation model and the frequency-domain visual foundation model: acquire multiple sparse bone projection images, and use the multiple sparse bone projection images to perform transfer training on the temporal-domain visual foundation model; perform Fourier transform on the multiple sparse bone projection images to obtain multiple sparse bone frequency-domain projection images, and use the multiple sparse bone frequency-domain projection images to perform transfer training on the frequency-domain visual foundation model.

[0009] Coordinate sampling for high-dimensional feature encoding: Obtain the encoders of the pre-trained temporal and frequency domain visual base models, freeze the network parameters, and extract the temporal representation t from sparse bone CT acquisition data. S and frequency domain characterization t F Using the fusion module to accept t S and t F To obtain complementary feature representations t C , for t C Perform high-dimensional feature meshing, with a sampling grid of c.S , obtain t G ;

[0010] Feature grid and coordinate grid fusion: using a sinusoidal position encoding function to map the defined coordinate domain to a high-dimensional function space, γ k (c)=[cos(2 k πu),sin(2 k πu),cos(2 k πv),sin(2 k πv)], where γ k (·) indicates the SPE function encoding K. th Frequency components, where c = (u, v) is the defined coordinate domain;

[0011] Training a continuous representation function: A super-resolution network architecture is constructed using a high-dimensional function representing the continuous representation of t. G and γ k (c S They are concatenated and then fed into a multilayer perceptron (MLP) super-resolution network architecture to predict density values.

[0012] Super-resolution inference and image reconstruction: using 2D spatial coordinates c of a full-view sine curve F =(u F ,v F Replace the 2D spatial coordinates c of the sparse sine graph S =(u S ,v S ), will c F =(u F ,v F ) is converted to γ ​​via the SPE function k (c F After that, γ k (c F ) and t G The data are combined and fed into a super-resolution network to predict the full-view sine curve. right The FBP reconstruction algorithm is applied to obtain a full-view CT image x, where FBP represents the reconstruction operation from bone CT projection to the image domain.

[0013] Furthermore, in the pre-training steps of the time-domain visual basic model and the frequency-domain visual basic model, the acquired multiple bone sparse projection images have a sparsity of 15%-55% of the field of view.

[0014] Furthermore, in the pre-training steps of the temporal-domain visual base model and the frequency-domain visual base model, the transfer training of the temporal-domain visual base model and the frequency-domain visual base model is optimized using a self-supervised method, specifically as follows:

[0015] L S =||SDVFM(y),y||1

[0016] L F =||FDVFM(y),y FFT ||1

[0017] Where y represents the projection observation data obtained in the sparse sampling mode of bone micro-CT, FFT represents the fast Fourier transform of the projection observation data, SDVFM is the basic visual model in the time domain, and FDVFM is the basic visual model in the frequency domain.

[0018] Furthermore, in the coordinate sampling step of the high-dimensional feature encoding, a fusion module is used to accept t S and t F To obtain complementary feature representations t C Specifically: t C =FM(Cat(t) S ,t F )), where FM is the feature fusion module, which contains a three-layer convolutional network.

[0019] Furthermore, in the coordinate sampling step of the high-dimensional feature encoding, for t C The specific process of performing high-dimensional feature meshing is as follows: Utilizing the cell decoding operation G, this operation uses t... C and sampling grid c S As input, and then by using the position information of each pixel in the coordinates, from t C Extract the corresponding eigenvalues ​​and apply bilinear interpolation to calculate the output matrix at time t. G The feature value at the corresponding position.

[0020] Furthermore, in the feature grid and coordinate grid fusion step, in order to make the trained super-resolution network applicable to different complex imaging scenarios, the generalized feature information learned by the visual base model from multi-source imaging scenarios is processed into coordinates and then fused with the SPE function to serve as the input of the super-resolution network.

[0021] Furthermore, in the training continuity representation function step, a residual connection is introduced between the input of the MLP and the output of the 10th Linear+ReLU layer to accelerate gradient backpropagation and suppress training instability.

[0022] Furthermore, in the step of training the continuity representation function, the performance of the super-resolution network is improved by using a composite self-supervised loss function, which is:

[0023]

[0024] In the training of the super-resolution network, the hyperparameter γ is set to 1e-3.

[0025] Furthermore, in the super-resolution inference and image reconstruction steps,

[0026] The second objective of this invention is achieved by the following technical solution:

[0027] A bone micro-CT image reconstruction device, used to implement any of the above-mentioned bone micro-CT image reconstruction methods, including...

[0028] CT scanner to acquire multiple images of sparse bone projection;

[0029] The data processing device performs pre-training of the temporal and frequency domain visual basic models, coordinate sampling of high-dimensional feature encoding, training of continuous representation functions, and super-resolution inference and image reconstruction on the multiple bone sparse projection images.

[0030] Compared with existing technologies, the bone micro-CT image reconstruction method of the present invention, through steps such as pre-training of the time-domain visual basic model and frequency-domain visual basic model, coordinate sampling of high-dimensional feature encoding, training of continuity representation function, and super-resolution inference and image reconstruction of multiple sparse projection bone images, can simultaneously achieve high resolution and fast imaging of bone micro-CT in sparse sampling mode, reducing the radiation dose damage to patients by more than 80%. Attached Figure Description

[0031] Figure 1 This is a flowchart of the bone micro-CT image reconstruction method of the present invention;

[0032] Figure 2 This is a network architecture diagram of the bone micro-CT image reconstruction method of the present invention;

[0033] Figure 3 This is a schematic diagram of high-dimensional feature meshing processing;

[0034] Figure 4 This is a full-view reconstruction result image based on existing technology;

[0035] Figure 5 The reconstruction result is shown at a 15% viewpoint.

[0036] Figure 6 The reconstruction result is shown at a 20% viewpoint.

[0037] Figure 7 The super-resolution output is for a sparsity of 15%.

[0038] Figure 8 This is the super-resolution output with a sparsity of 20%. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or it can be fixed through another intermediate component. When a component is said to be "connected to" another component, it can be directly connected to the other component or it may be fixed through another intermediate component. When a component is said to be "set on" another component, it can be set directly on the other component or it may be set through another intermediate component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0042] Please see Figure 1 The present invention provides a method for bone microscopic CT image reconstruction, comprising the following steps:

[0043] Pre-training of the temporal-domain visual foundation model and the frequency-domain visual foundation model: acquire multiple sparse bone projection images, and use the multiple sparse bone projection images to perform transfer training on the temporal-domain visual foundation model; perform Fourier transform on the multiple sparse bone projection images to obtain multiple sparse bone frequency-domain projection images, and use the multiple sparse bone frequency-domain projection images to perform transfer training on the frequency-domain visual foundation model.

[0044] Coordinate sampling for high-dimensional feature encoding: Obtain the encoders of the pre-trained temporal and frequency domain visual base models, freeze the network parameters, and extract the temporal representation t from sparse bone CT acquisition data. S and frequency domain characterization t F Using the fusion module to accept t S and t F To obtain complementary feature representations t C , for t C Perform high-dimensional feature meshing, with a sampling grid of c. S , obtain tG ;

[0045] Feature grid and coordinate grid fusion: using a sinusoidal position encoding function to map the defined coordinate domain to a high-dimensional function space, γ k (c)=[cos(2 k πu),sin(2 k πu),cos(2 k πv),sin(2 k πv)], where γ k (·) indicates the SPE function encoding K. th Frequency components, where c = (u, v) is the defined coordinate domain;

[0046] Training a continuous representation function: A super-resolution network architecture is constructed using a high-dimensional function representing the continuous representation of t. G and γ k (c S They are concatenated and then fed into a multilayer perceptron (MLP) super-resolution network architecture to predict density values.

[0047] Super-resolution inference and image reconstruction: using 2D spatial coordinates c of a full-view sine curve F =(u F ,v F Replace the 2D spatial coordinates c of the sparse sine graph S =(u S ,v S ), will c F =(u F ,v F ) is converted to γ ​​via the SPE function k (c F After that, γ k (c F ) and t G The data are combined and fed into a super-resolution network to predict the full-view sine curve. right The FBP reconstruction algorithm is applied to obtain a full-view CT image x, where FBP represents the reconstruction operation from bone CT projection to the image domain.

[0048] This invention presents a bone micro-CT image reconstruction method that combines the powerful data representation capabilities of a visual foundation model with the continuous representation modeling of bone projection data, effectively addressing the challenges of high resolution and rapid imaging in bone micro-CT. Specifically, this invention employs a pre-trained visual foundation model to extract rich feature representations from the temporal and frequency domains of the projection data, and after coordinate processing, incorporates them into a super-resolution framework based on implicit representation learning, thereby improving the imaging quality of bone micro-CT under extremely sparse acquisition conditions.

[0049] Please continue reading. Figure 2 The specific pre-training steps for the temporal-domain visual fundamental model and the frequency-domain visual fundamental model are as follows:

[0050] The invention employs a sinusoidal domain visual foundation model (SDVFM) and a frequency domain visual foundation model (FDVFM). Features extracted from different domains using pre-trained visual foundation models (VFMs) are used to optimize downstream image super-resolution tasks. On one hand, the rich features extracted by VFMs enhance the generalization ability of downstream super-resolution models; on the other hand, their rich feature representation enhances the imaging performance of bone micro-CT. In this invention, 20,000 multi-bone sparse projection images (in this embodiment, 15%-55% spectral density) and corresponding multiple frequency domain projection images are selected for transfer training of SDVFM and FDVFM. Training employs a self-supervised optimization method, as detailed in the following expression:

[0051] L S =||SDVFM(y),y||1,

[0052] L F =||FDVFM(y),y FFT ||1,

[0053] Where y represents the projected observation data obtained in the sparse sampling mode of bone micro-CT, and FFT represents the Fast Fourier Transform of the projected observation data. For the pre-trained SDVFM and FDVFM, only their encoders are retained and used for the extraction of time-domain and frequency-domain features in downstream imaging tasks.

[0054] The coordinate sampling steps for high-dimensional feature encoding are as follows:

[0055] Using encoders pre-trained with SDVFM and FDVFM, network parameters were frozen, and temporal and frequency domain features were extracted from sparse bone CT acquisition data, respectively. To obtain complementary feature representations... C The feature fusion module FM is used to accept the sinusoidal (time domain) representation t. S and frequency domain characterization t F Here, the FM module consists of three convolutional layers, and the fusion process is specifically defined as follows:

[0056] t C =FM(Cat(t) S ,t F ))

[0057] Using the above formula, complementary feature information in the time and frequency domains can be obtained. However, a new problem arises: t CA mismatch between the coordinate grid information and the downstream super-resolution input can hinder the execution of downstream super-resolution processes. Therefore, this invention proposes a gridding operation for high-dimensional features, specifically as follows: Figure 3 As shown. The cell decoding operation G is used, which is t C and sampling grid c S As input, and then by using the position information of each pixel in the coordinates, from t C Extract the corresponding eigenvalues ​​and apply bilinear interpolation to calculate the output matrix (feature grid) at time t. G The feature value at the corresponding position.

[0058] The specific steps for fusing feature meshes and coordinate meshes are as follows:

[0059] Considering that super-resolution networks tend to learn from low-frequency functions to achieve continuous modeling of high-dimensional functions, for the defined coordinate domain c = (u, v), we use the Sine Position Encoding (SPE) function to map c = (u, v) to a high-dimensional function space, specifically defined as follows:

[0060] γ k (c)=[cos(2 k πu),sin(2 k πu),cos(2 k πv),sin(2 k πv)]

[0061] In the above formula, γ k (·) indicates the SPE function encoding K. th Frequency components. Furthermore, to ensure the trained super-resolution network is applicable to various complex imaging scenarios, the super-resolution input cannot solely rely on the high-dimensional coordinate function input in the above equation. Therefore, we coordinate-process the generalized feature information learned by the visual base model from multi-source imaging scenarios and fuse it with the SPE function, using both as inputs to the super-resolution network, such as... Figure 2 As shown.

[0062] The training of the continuous representation function is specifically as follows:

[0063] This application employs the continuous representation of high-dimensional functions to construct a super-resolution network architecture. Specifically, it includes a 20-layer multilayer perceptron (MLP), such as... Figure 2 The diagram shows that each layer has 320 channels. To accelerate gradient backpropagation and suppress training instability, a residual connection is introduced between the input of the MLP and the output of the 10th Linear+ReLU layer. Subsequently, for t... G and γ k (c S Perform a concatenation operation and input them into an MLP to predict density values.

[0064]

[0065] To improve the performance of super-resolution networks, we designed a carefully constructed composite self-supervised loss function, which consists of an L1-based L1 loss and a frequency loss, aiming to improve the fidelity of reconstructed image content and structure. The specific expression of the composite self-supervised loss function is as follows:

[0066]

[0067] In the training of the super-resolution network, the hyperparameter γ is set to 1e-3.

[0068] The specific steps of super-resolution inference and image reconstruction are as follows:

[0069] The network proposed in this application is trained in a self-supervised manner, therefore its predictions during the training phase are sparse (SV) sine waves. To reconstruct the full-view CT (FVCT) image x, a reprojection technique is used during the inference phase to display, as shown in the image. Figure 2 The workflow in (B) involves learning complementary representations t from the SV sine wave and the FFT image. C The 2D spatial coordinates c of the FV sine curve F =(u F ,v F This is combined with other methods to predict FVCT. Specifically, firstly, c... F =(u F ,v F Replace the 2D spatial coordinates c of the SV sine graph. S =(u S ,v S At the same time, c F =(u F ,v F ) is converted to γ ​​via the SPE function k (c F ), then, γ k (c F ) and t G The data are combined and fed into a super-resolution network to predict the FV sine curve. Finally, we The FBP reconstruction algorithm is applied to obtain the FVCT image x:

[0070]

[0071] FBP represents the reconstruction operation that transforms the bone CT projection image into the image domain.

[0072] The images reconstructed using the bone micro-CT image reconstruction method of this application are attached. Figure 7 as well as Figure 8 As shown, Figure 7 This is the super-resolution output result with a sparsity of 15%. Figure 8 This is the super-resolution output result with a sparsity of 20%. In existing technologies, the full-view reconstruction result is as follows: Figure 4 As shown, this requires an extremely high sampling rate and results in a large radiation dose. When the bone micro-CT image reconstruction method of this application is not used, and image reconstruction is performed directly using 15% and 20% viewing angles, the reconstructed images are as follows: Figure 5 as well as Figure 6 As shown, the reconstruction results exhibit significant stripe artifacts, and the microstructures within the bones are severely blurred. In contrast, the super-resolution reconstruction results of this invention show visual consistency with the full-view reconstruction results of existing technologies.

[0073] This application also discloses a bone micro-CT image reconstruction device for implementing any of the above-mentioned bone micro-CT image reconstruction methods, including...

[0074] CT scanner to acquire multiple images of sparse bone projection;

[0075] The data processing device performs pre-training of the temporal and frequency domain visual basic models, coordinate sampling of high-dimensional feature encoding, training of continuous representation functions, and super-resolution inference and image reconstruction on the multiple bone sparse projection images.

[0076] Compared with existing technologies, the bone micro-CT image reconstruction method of the present invention, through steps such as pre-training of the time-domain visual basic model and frequency-domain visual basic model, coordinate sampling of high-dimensional feature encoding, training of continuity representation function, and super-resolution inference and image reconstruction of multiple sparse projection bone images, can simultaneously achieve high resolution and fast imaging of bone micro-CT in sparse sampling mode, reducing the radiation dose damage to patients by more than 80%.

[0077] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These are all equivalent modifications and improvements made to the above embodiments based on the essential technology of the present invention, and all of these fall within the protection scope of the present invention.

Claims

1. A method for bone micro-CT image reconstruction, characterized in that, Includes the following steps: Pre-training of the temporal-domain visual basic model and the frequency-domain visual basic model: acquire multiple bone sparse projection images, and use the multiple bone sparse projection images to perform transfer training on the temporal-domain visual basic model; Multiple sparse bone projection images are subjected to Fourier transform to obtain multiple sparse bone frequency domain projection images, and the frequency domain visual basic model is transferred and trained using the multiple sparse bone frequency domain projection images. Coordinate sampling for high-dimensional feature encoding: Obtain the encoders of the pre-trained temporal and frequency domain visual base models, freeze the network parameters, and extract the temporal representation t from sparse bone CT acquisition data. S and frequency domain characterization t F Use the fusion module to accept t S and t F To obtain complementary feature representations t C , for t C Perform high-dimensional feature meshing, with a sampling grid of c. S , obtain t G ; Feature grid and coordinate grid fusion: using a sinusoidal position encoding function to map the defined coordinate domain to a high-dimensional function space, γ k (c)=[cos(2 k πu),sin(2 k πu),cos(2 k πv),sin(2 k πv)], where γ k (·) indicates the SPE function encoding K. th Frequency components, where c = (u, v) is the defined coordinate domain; Training a continuous representation function: A super-resolution network architecture is constructed using a high-dimensional function representing the continuous representation of t. G and γ k (c S They are concatenated and then fed into a multilayer perceptron (MLP) super-resolution network architecture to predict density values. Super-resolution inference and image reconstruction: using 2D spatial coordinates c of a full-view sine curve F =(u F ,v F Replace the 2D spatial coordinates c of the sparse sine graph S =(u S ,v S ), will c F =(u F ,v F ) is converted to γ ​​via the SPE function k (c F After that, γ k (c F ) and t G The data are combined and fed into a super-resolution network to predict the full-view sine curve. right The FBP reconstruction algorithm is applied to obtain a full-view CT image x, where FBP represents the reconstruction operation from bone CT projection to the image domain.

2. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the pre-training steps of the time-domain visual basic model and the frequency-domain visual basic model, the acquired multiple bone sparse projection images have a sparsity of 15%-55% of the field of view.

3. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the pre-training steps of the temporal and frequency domain visual base models, the transfer training of the temporal and frequency domain visual base models is optimized using a self-supervised method, specifically as follows: L S =||SDVFM(y),y||1 L F =||FDVFM(y),y FFT ||1 Where y represents the projection observation data obtained in the sparse sampling mode of bone micro-CT, FFT represents the fast Fourier transform of the projection observation data, SDVFM is the basic visual model in the time domain, and FDVFM is the basic visual model in the frequency domain.

4. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the coordinate sampling step of the high-dimensional feature encoding, a fusion module is used to accept t. S and t F To obtain complementary feature representations t C Specifically: t C =FM(Cat(t) S ,t F )), where FM is the feature fusion module, which contains a three-layer convolutional network.

5. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the coordinate sampling step of the high-dimensional feature encoding, for t C The specific process of performing high-dimensional feature meshing is as follows: Utilizing the cell decoding operation G, this operation uses t... C and sampling grid c S As input, and then by using the position information of each pixel in the coordinates, from t C Extract the corresponding eigenvalues ​​and apply bilinear interpolation to calculate the output matrix at time t. G The feature value at the corresponding position.

6. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the feature grid and coordinate grid fusion step, in order to make the trained super-resolution network applicable to different complex imaging scenarios, the generalized feature information learned by the visual base model from multi-source imaging scenarios is processed into coordinates and then fused with the SPE function to serve as the input of the super-resolution network.

7. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the training continuity representation function step, a residual connection is introduced between the input of the MLP and the output of the 10th Linear+ReLU layer to accelerate gradient backpropagation and suppress training instability.

8. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the step of training the continuous representation function, the performance of the super-resolution network is improved by using a composite self-supervised loss function, which is: In the training of the super-resolution network, the hyperparameter γ is set to 1e-3.

9. The bone micro-CT image reconstruction method according to claim 1, characterized in that: In the super-resolution inference and image reconstruction steps, 10. A bone micro-CT image reconstruction device, used to implement the bone micro-CT image reconstruction method as described in any one of claims 1-9, characterized in that; include CT scanner to acquire multiple images of sparse bone projection; The data processing device performs pre-training of the temporal and frequency domain visual basic models, coordinate sampling of high-dimensional feature encoding, training of continuous representation functions, and super-resolution inference and image reconstruction on the multiple bone sparse projection images.

Images