Method of processing rotational positioning map data, computer program, computer readable storage medium and computing device
By combining transformer and CNN methods, the data loss problem caused by sparse sampling in rotary projection positioning imaging is solved, and high-quality positioning map reconstruction and computing power optimization are achieved.
Patent Information
- Application Number
- CN202410178070.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-08
- Publication Date
- 2025-08-08
AI Technical Summary
The existing rotary projection positioning imaging technology has data loss caused by sparse sampling in high-step situations, which affects image quality, and the existing methods require high computing power when image recovery.
Using a combined architecture of transformer and CNN, the missing information is predicted through transformer and reconstructed using a multi-head attention structure, combined with CNN for upsampling and detail recovery, reducing computing power requirements.
High-quality positioning map reconstruction is realized, reducing the limitations of reconstructing positioning maps in the prior art, improving image quality and optimizing computing power requirements.
Smart Images
Figure CN120451293A_ABST
Abstract
Description
[0001] Nouns and pronouns referring to persons in this patent application are not limited to a specific gender. Technical Field
[0002] The present invention relates to the processing of rotational topogram data in computed tomography. Background Art
[0003] As a standardized procedure for most commercial CT scans, the localizer radiography (scouting image, or topogram) provides an important reference for scanning range planning, attenuation distribution for automatic exposure control, and important clinical guidance for technicians (such as metal accessories or implants). Figure 1 This is the positioning map obtained by the traditional method, which includes three positioning maps arranged one above the other.
[0004] The following papers propose a new method, Rotating Projectionbase Localizer Radiograph (RPLR), which can calculate more accurate attenuation, provide patient- or organ-specific localization, and improve clinical workflow by avoiding frequent gantry acceleration and deceleration:
[0005] [1]Cai, X., Yang, Y., Wen,
[0006] [2]Tian, Y., Wen,
[0007] In rotational projection scintigraphy, the CT tube rotates while the examination table moves in the Z direction. The tube is exposed at the 12 o'clock position. Because the examination table moves a certain distance between exposures, the scintigraphy (LR) data is incomplete. Figure 2 The positioning map obtained by RPLR in the prior art includes three positioning maps arranged up and down. Figure 1 Positioning Figure 1 One-to-one correspondence ( Figure 5-6 Same).
[0008] However, RPLR still has scanning speed limitations due to image quality considerations. While high pitch can reduce data acquisition time, projections from sparse exposure angles result in a sparse sampling pattern for a given anatomical structure or attenuation distribution, resulting in data loss. This compromises image quality compared to conventional scout imaging or small-pitch RPLR.
[0009] The following paper proposes a transformer+CNN architecture for image completion tasks of multi-images.
[0010] Wan, Z., Zhang, J., Chen, D., & Liao, J-High-fidelity pluralistic image completion with transformers. In Proceedings of the IEEE / CVF International Conference on Computer Vision (pp.4692-4701). (2021) Summary of the Invention
[0011] In view of this, the present invention provides a method for processing rotation positioning map data, a computer program, a computer-readable storage medium, and a computing device.
[0012] According to a first aspect of the present invention, there is provided a method for processing rotational localization map data of a computed tomography scan, comprising:
[0013] Rearranging the rotation positioning map data into one-dimensional data;
[0014] Processing the one-dimensional data using a transformer;
[0015] Rearranging the one-dimensional data processed by the transformer into two-dimensional data to obtain a restored positioning map;
[0016] The transformer is trained according to the following steps:
[0017] Providing a plurality of pairs of positioning map training data, each pair of training data includes complete positioning map training data and its corresponding rotated positioning map training data;
[0018] Rearranging the rotation positioning map training data into one-dimensional data;
[0019] Processing the one-dimensional data using a transformer;
[0020] The one-dimensional data processed by the transformer is rearranged into two-dimensional data so that the two-dimensional data corresponds to the complete positioning map training data.
[0021] In one embodiment, the method further comprises, before rearranging the rotation positioning map data:
[0022] Compressing the rotation positioning map data so that it has fewer pixels and a narrower grayscale distribution;
[0023] After rearranging the one-dimensional data processed by the transformer:
[0024] The two-dimensional data obtained in the previous step is upsampled so that its image size is equal to the corresponding complete localization map training data.
[0025] In one embodiment, the method further comprises:
[0026] Using CNN to process the rotation positioning map data and the two-dimensional data obtained in the above steps or the two-dimensional data obtained in step S108 after upsampling;
[0027] The CNN is trained according to the following steps:
[0028] Provide multiple pairs of positioning map training data, each pair of training data includes the complete positioning map training data, the corresponding rotated positioning map training data and the rough prior positioning map generated by the transformer;
[0029] The CNN is used to process the rotated positioning map training data and the coarse prior positioning map generated by the transformer or the coarse prior positioning map generated by the transformer after upsampling, so that the two-dimensional data is equal to the corresponding complete positioning map training data.
[0030] In one embodiment, the method further comprises:
[0031] The original data in the rotated positioning image is used to replace the data at the corresponding position in the restored positioning image.
[0032] In one embodiment, the rotation positioning map data includes projection data of first direction exposure and projection data of second direction exposure, or only includes projection data of first direction exposure.
[0033] In one embodiment, the first direction is the 12 o'clock direction.
[0034] In one embodiment, the second direction is the opposite direction of the first direction.
[0035] According to a second aspect of the present invention, a computer program is provided, which can implement the steps of the above method when executed by a processor.
[0036] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program can implement the steps of the above-described method when executed by a processor.
[0037] According to a fourth aspect of the present invention, there is provided a computing device comprising the computer-readable storage medium described above.
[0038] This invention uses a transformer to predict missing information in the rotational localization map data. The resulting reconstructed localization map has high image quality, alleviating the limitations of existing localization map reconstruction techniques (e.g., interpolation processing in high-step scenarios). To reduce computing power requirements, the rotational localization map data can be compressed, and then the lost information can be restored through a CNN. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that those skilled in the art will understand the above and other features and advantages of the present invention more clearly. In the accompanying drawings:
[0040] Figure 1 This is the positioning map obtained by the traditional method.
[0041] Figure 2 This is the positioning map obtained by RPLR in the prior art.
[0042] Figure 3 The figure is a schematic flow chart of a method for processing rotation positioning map data according to an embodiment of the present invention.
[0043] Figure 4 FIG. 4 is a schematic diagram of a method for processing rotation positioning map data according to an embodiment of the present invention.
[0044] Figure 5 Schematic diagram of a rough priori positioning map obtained by the method for processing rotation positioning map data according to an embodiment of the present invention.
[0045] Figure 6 It is a schematic diagram of obtaining a restored positioning map by a method for processing rotated positioning map data according to an embodiment of the present invention.
[0046] Figure 7 2 is a schematic flowchart of training a transformer according to an embodiment of the present invention.
[0047] Figure 8 Schematic flowchart of training CNN according to an embodiment of the present invention.
[0048] Figure 9 Schematic diagram of a CNN according to an embodiment of the present invention.
[0049] Figure 10 Schematic diagram of a discriminator used for training a CNN network according to an embodiment of the present invention.
[0050] In the above drawings, the reference numerals used are as follows:
[0051] 100 Method for processing rotation positioning map data S101, S102, S102', S103, S104, S105, S106,
[0052] step
[0053] S107, S107', S108, S11O, S11O', S112 DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail with reference to the following examples.
[0055] The present invention proposes a method for forming a positioning map based on deep learning, combining CNN (convolutional neural network) and transformer. The CNN and transformer combination has demonstrated excellent ability to process images and complete tasks in the field of computer vision.
[0056] To solve the data missing caused by sparse sampling in RPLR, the following scheme is used to predict and recover the missing data:
[0057] Recover the missing attenuated information from the reduced-resolution image set through the transformer;
[0058] Through upsampling CNN, the projection context detail is modified to restore the original LR image quality.
[0059] Specifically, when the transformer predicts missing pixels, it considers all available pixels, unlike autoregressive methods that only use previous context information. Therefore, this method can also be called a bidirectional transformer.
[0060] The context recovery in RPLR aims to obtain the conditional distribution p(I|I m ):
[0061] p(I|I m )=p(I|I m )·p(I t |I,I m )=p(I t , I|I m )=p(I t |I m )·p(I|I t , I m ) (1)
[0062] Among them, I represents the gold standard (ground truth) of traditional LR, I m represents sparsely sampled RPLR with missing data.
[0063] The recovery process can be divided into two steps:
[0064] First, for a given I m , a rough priori positioning map I can be predicted t , expressed as p(I t |I m ), which can be obtained through transformer.
[0065] Then, we have a rough prior positioning map I t and sparse sampling localization map I m , (can restore the complete image I, expressed as p(I|I t , I m ), which can adopt CNN. In this embodiment, the sparse sampling positioning map Im Including projection data from the positive direction (e.g., 12 o'clock direction) and the reverse direction (e.g., 6 o'clock direction). m Including projection data from the positive direction and other directions, or only including projection data from directly above. In fact, the above situation can be further generalized: sparse sampling positioning map I m The projection data may include projection data exposed in a first direction, or projection data exposed in a first direction and a second direction. Preferably, the second direction is the opposite direction of the first direction.
[0066] In the present invention, the terms "data" and "graph" may be used interchangeably. m Sometimes also called rotation positioning map data or rotation positioning map.
[0067] Figure 3 FIG. 1 is a schematic flow chart of a method 100 for processing rotation positioning map data according to an embodiment of the present invention. Figure 4 FIG. 1 is a schematic diagram of a method 100 for processing rotation positioning map data according to an embodiment of the present invention. Figure 3 As shown, the method 100 for processing rotational localization map data of a computed tomography scan includes:
[0068] Step S102 compresses (rescaling) the rotation positioning map data to make it have fewer pixels and a narrower grayscale distribution. This operation will lose some information, which needs to be restored through step S110. Considering the multi-head attention structure in the transformer structure, in order to optimize the computing power, the method 100 in this embodiment includes step S102. If computing power is not considered, step S102 can also be omitted. Figure 4 As shown in FIG. 1 , the original rotated positioning map with a dimension of 512×512 and a grayscale distribution range of 0 to 255 is compressed into a positioning map with a dimension of 64×64 and a grayscale distribution range of 0 to 63.
[0069] Step S104: rearrange the rotation positioning map data into one-dimensional data. The rotation positioning map data can be represented as I m , it loses some pixel information. Figure 4 As shown, the 64×64 two-dimensional data is transformed into 4096×1 one-dimensional data, which includes missing pixels.
[0070] Step S106, using transformer to process the one-dimensional data. The transformer can generate a probability distribution of missing pixel values, and reconstruct the localization map (LR) with the pixel values with the maximum probability. In this embodiment, the transformer includes layer normalization, multi-head self-attention and FC layer. Unlike the transformer in the autoregressive model (elements are predicted by a single direction of attention, that is, only the elements before the target element (token) participate in the prediction), all valid elements in the prediction of this model will participate in the prediction of the target element to capture all available information, so it can also be called a bidirectional transformer. In order to train this transformer model, the loss function is the cross entropy between the prediction result and the gold standard. The training of the model will be described in detail below. Figure 4 As shown in the figure, after the transformer processing, the original missing pixels in the 4096×1 one-dimensional data are replaced by the predicted values.
[0071] Step S108: Rearrange the one-dimensional data processed by the transformer into two-dimensional data. Figure 4 As shown, the 4096×1 one-dimensional data is transformed into 64×64 two-dimensional data, which corresponds to the coarse prior positioning map I t The number of pixels and grayscale range of this two-dimensional data are the same as before the rearrangement in step S104. Figure 5 The rough priori positioning map I is obtained according to the method 100 for processing the rotation positioning map data according to the embodiment of the present invention. t Schematic diagram of .
[0072] If there is no step S102, that is, no information loss due to compression, then I t This is the a priori positioning map, and there is no need to perform upsampling in step S110 to restore the image size.
[0073] Step S110 upsamples the two-dimensional data obtained in step S108 to an image size equal to the corresponding complete localization map training data. In this embodiment, the image size is upsampled from 64×64 to 512×512. While this upsampling algorithm (bflinear interpolation) can increase the image size, it does not effectively restore image details. Therefore, a CNN network can be used to restore image details.
[0074] Step S112, using CNN to process the data of the rotated positioning map and the two-dimensional data obtained by upsampling in step S110 to obtain a restored positioning map. The characteristic of CNN is that it has a strong ability to extract image detail features. It is used as an upsampling network to recover the original positioning map I with missing areas. m Under the guidance of , the real texture information of the prior localization map is reconstructed. The guided upsampling network can be expressed as:
[0075]
[0076] in, Represents the rough prior positioning map I t The result of bilinear interpolation (S110) is shown in FIG1 , where ∩ represents a concatenation operation in the channel dimension, and F represents an upsampling network with a residual block. These residual blocks are used for feature extraction in the latent space.
[0077] The guided upsampling network is optimized by a loss function that includes not only I pred The mean absolute error (MAE) l1 between the corresponding gold standard I, and also includes the adversarial loss L adv , perceptual loss L p and style loss L s , to produce more realistic and precise details:
[0078] L l1 =||I pred -I||1 (3)
[0079]
[0080]
[0081] Among them, L l1 represents the loss measured in l1, ||I pred -I||1 means I pred The absolute error of the pixel corresponding to I, the subscript 1 represents the arithmetic difference (in other embodiments, I can also be used pred The mean square error (mS2) between the two values and the corresponding gold standard I.
[0082] In this embodiment, after experiments, the loss weights are set as follows: α1=1,0, α2==0.1, α3=1, α4=250.
[0083] Figure 6 FIG. 1 is a schematic diagram of obtaining a restored positioning map I according to the method 100 for processing rotated positioning map data according to an embodiment of the present invention. Figure 6 As shown, the missing information is predicted in the restored positioning map I. In order to improve the authenticity of the image, the method 100 for processing the rotation positioning map data may further include:
[0084] Step S114: replacing the data at the corresponding position in the restored positioning map I with the original data in the rotated positioning map.
[0085] Figure 7 2 is a schematic flowchart of training a transformer according to an embodiment of the present invention. Figure 7 Step S103, step S105, and step S107 are similar to Figure 3 Steps S104, S106, and S108 in FIG. 1 are briefly described below. Figure 7 As shown, the transformer is trained according to the following steps:
[0086] Step S101: providing a plurality of pairs of positioning map training data, each pair of training data including complete positioning map training data and its corresponding rotated positioning map training data. These complete positioning map training data are used as a gold standard to make the algorithm converge.
[0087] Step S103: rearrange the rotation positioning map training data into one-dimensional data.
[0088] Step S105: Process the one-dimensional data using a transformer.
[0089] Step S107: Rearrange the one-dimensional data processed by the transformer into two-dimensional data so that the two-dimensional data is equal to the corresponding complete positioning map training data (if the image data is compressed before the transformer, the compressed complete positioning map training data is the corresponding complete positioning map training data). In this application, "complete" means that the scanned data is complete, but after subsequent processing, some information may be lost due to compression.
[0090] Figure 8 is a schematic flow chart of training CNN according to an embodiment of the present invention, Figure 9 Schematic diagram of a CNN according to an embodiment of the present invention. Figure 8 Steps S102', S107', S110', and S112' are similar to Figure 3 Step S102 in Figure 7 Step S107, Figure 3 Steps S110 and S112 in the following will be briefly described. Figure 8 and 9 As shown in the figure, CNN is trained according to the following steps:
[0091] Step S109: Provide multiple pairs of positioning map training data. Each pair of training data includes the complete positioning map training data, the corresponding rotated positioning map training data, and the rough prior positioning map generated by the transformer. These complete positioning map training data are used as the gold standard to make the algorithm converge.
[0092] Step S110 ′: upsampling (Bilinear interpolation) the two-dimensional data of the rough priori positioning map so that the pixel size of the two-dimensional data is equal to the corresponding complete positioning map training data.
[0093] In step S112 ′, the rotated positioning map training data and the two-dimensional data obtained in step S110 ′ are processed using CNN so that the processed two-dimensional data is equal to the corresponding complete positioning map training data.
[0094] If the image size of the rotated localization map training data and the coarse prior localization map generated by the transformer are the same, then upsampling in step S110 ′ is not required.
[0095] During training, the discriminator can be used to determine whether the positioning map is obtained by the upsampling network to help CNN network training. Figure 10 is a schematic diagram of an identifier according to an embodiment of the present invention.
[0096] The present invention further provides a computer program, which can implement the steps of the method 100 for processing rotation positioning map data when executed by a processor.
[0097] The present invention further provides a computer-readable storage medium having a computer program stored thereon, which can implement the steps of the method 100 for processing rotation positioning map data when the program is executed by a processor.
[0098] The present invention also provides a computing device, which includes the computer-readable storage medium described above.
[0099] This invention uses a transformer to predict missing information in the rotational localization map data. The resulting reconstructed localization map has high image quality, alleviating the limitations of existing localization map reconstruction techniques (e.g., interpolation at high step sizes). To reduce computing power requirements, the rotational localization map data can be compressed, and then the lost information can be restored through a CNN.
[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for processing rotational localization map data of a computed tomography scan, comprising: Step S104, rearranging the rotation positioning map data into one-dimensional data; Step S106, using a transformer to process the one-dimensional data; Step S108, rearranging the one-dimensional data processed by the transformer into two-dimensional data to obtain a restored positioning map; The transformer is trained according to the following steps: Step S101, providing a plurality of pairs of positioning map training data, each pair of training data including complete positioning map training data and its corresponding rotated positioning map training data; Step S103, rearranging the rotation positioning map training data into one-dimensional data; Step S105, using a transformer to process the one-dimensional data; Step S107: rearrange the one-dimensional data processed by the transformer into two-dimensional data, so that the two-dimensional data corresponds to complete positioning map training data.
2. The method according to claim 1, wherein: Also included before step S104: Step S102, compressing the rotation positioning map data to make it have fewer pixels and a narrower grayscale distribution; After step S108: Step S110 , upsampling the two-dimensional data obtained in step S108 so that its image size is equal to the corresponding complete positioning map training data.
3. The method according to claim 1, wherein include: Step S112, using CNN to process the rotation positioning map data and the two-dimensional data obtained in step S108 or the up-sampled two-dimensional data obtained in step S108; The CNN is trained according to the following steps: Step S109, providing a plurality of pairs of positioning map training data, each pair of training data including complete positioning map training data, corresponding rotated positioning map training data and a rough prior positioning map generated by a transformer; Step S112 ′: using the CNN to process the rotated positioning map training data and the coarse prior positioning map generated by the transformer or the coarse prior positioning map generated by the transformer after upsampling, so that the processed two-dimensional data is equal to the corresponding complete positioning map training data.
4. The method according to claim 1, wherein include: Step S114: replacing the data at the corresponding position in the restored positioning map with the original data in the rotated positioning map.
5. The method according to claim 1, wherein: The rotation positioning map data includes projection data of first direction exposure and projection data of second direction exposure, or only includes projection data of first direction exposure.
6. The method according to claim 5, wherein: The first direction is the 12 o'clock direction.
7. The method according to claim 5, wherein: The second direction is the opposite direction of the first direction.
8. A computer program, which, when executed by a processor, can implement the steps of the method according to any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 can be implemented.
10. A computing device comprising the computer-readable storage medium of claim 9.