Method for suppressing x-ray rib images based on joint prediction filtering generation network

Abstract

The application discloses a rib image suppression method for X-ray chest radiographs based on a generative network of joint prediction filtering, and specifically comprises the following steps: step 1, pre-processing training set and test set images in a data set; step 2, constructing an X-ray chest radiograph rib image suppression network model based on a generative network of joint prediction filtering; step 3, training the model constructed in step 2 by using the training set data pre-processed in step 1, to obtain an X-ray chest radiograph rib image suppression network model based on a generative network of joint prediction filtering after training; and step 4, putting the test set images pre-processed in step 1 into the model trained in step 3, and finally obtaining soft tissue images after bone suppression. The method can reduce the negative influence of the overlapping structure of ribs on reading X-ray chest radiographs.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a method for suppressing rib images on chest X-rays based on a generative network of joint predictive filtering. Background Technology

[0002] Chest X-rays are a commonly used screening tool for lung diseases in employee physical examinations and small and medium-sized hospitals. However, the overlapping rib structures in ordinary chest X-rays can affect doctors' interpretation and increase the difficulty of computer-aided analysis. When X-rays pass through different tissues in the body, they are absorbed to varying degrees, resulting in differences in the amount of X-rays reaching the screen or film. This creates images with varying brightness or black and white contrast on the screen or X-ray film. The high density of ribs in chest X-rays obstructs X-ray penetration, causing ribs and other anatomical structures and soft tissues to overlap in the image, obscuring some soft tissues. If a lesion happens to fall within these overlapping areas, the lesion image will be obscured by the ribs in the chest X-ray, severely affecting the detection results of lung diseases. Suppressing the rib images in chest X-rays can significantly reduce the negative impact of overlapping rib structures on doctors' interpretation. Summary of the Invention

[0003] The purpose of this invention is to provide a method for suppressing rib images on chest X-rays based on a generative network of joint predictive filtering. By using this method to process rib images on chest X-rays, the negative impact of overlapping rib structures on reading chest X-rays can be reduced.

[0004] The technical solution adopted in this invention is a method for suppressing rib images on chest X-rays based on a generator network with joint predictive filtering, specifically including the following steps:

[0005] Step 1: Preprocess the images in the training and test sets of the dataset;

[0006] Step 2: Construct a network model for suppressing rib images on chest X-rays based on a generative network of joint predictive filtering.

[0007] Step 3: Use the training set data preprocessed in Step 1 to train the model constructed in Step 2, and obtain the trained X-ray rib image suppression network model based on the joint prediction filter generator network.

[0008] Step 4: Input the preprocessed test set images from Step 1 into the model trained in Step 3 to finally obtain soft tissue images after bone suppression.

[0009] The invention is further characterized by:

[0010] The preprocessing process in step 1 is to perform rotation and translation operations on the images in the dataset to achieve data augmentation.

[0011] In step 2, the X-ray rib image suppression network model based on the joint prediction filter generation network includes a bone suppression generator module (Generator), a bone suppression kernel prediction filter module (KPN), and a bone suppression discriminator module (Discriminator).

[0012] The bone suppression kernel prediction filter module KPN adaptively predicts the pixel kernel of the rib region based on the filter reconstruction according to the input image, and feeds the prediction result to the bone suppression generator module Generator to realize the intelligent combination of filtering and generating bone suppression results, thereby generating a more realistic chest X-ray rib suppression image. Finally, the discriminator distinguishes between real and fake bone suppression images.

[0013] The process of suppressing X-ray rib images using a generative network model based on joint predictive filtering is as follows:

[0014] Step A) uses the image X in the dataset as input to the generator, and the encoder sequentially extracts image features F at multiple different scales. i As shown in formula (1) below; the bone suppression kernel prediction filter module for F i Perform feature-level filtering kernel K l As shown in formula (2) below:

[0015] F i =Encoder(X) (1);

[0016] K l =Conv(ε l (…ε i+1 ([E i F i ])) (2);

[0017] Step B) involves filtering the features F at the feature level. a The internal and external attention modules of the fusion attention mechanism are fused for correction. The internal correction adopts the SKnet attention mechanism principle to complete the feature correction of the channel dimension as shown in formula (3). A spatial attention block Gate is added outside SKnet to calculate the spatial coefficient g, as shown in formula (4). The input features are then processed by a dilated convolution r=1 and then the feature F is calculated by a sigmoid operation. a Spatial variation threshold g, this value is compared with F b and input features F a The final aggregated feature F is obtained by weighted summation using the formula, as shown in formula (5):

[0018] F b =Sknet(Fa (3);

[0019] g = Gate(F) a (4);

[0020] F = F a *g+F b *(1-g) (5);

[0021] Step C), based on the aggregated features F obtained in step B), after passing through the decoder of the generative adversarial network, it is filtered at the image level with the image-level prediction kernel K0 from the bone suppression kernel prediction filtering module, as shown in Equation (6), to further enhance the rib detail information. Finally, the rib attention map R, which is adaptively learned by the network, is output, as shown in Equation (7). The rib attention map R obtained from the generator and the residual rib attention map are then processed. Figure 1 -R is fused with the input chest X-ray to obtain a soft tissue image S with bone suppression. o The output formula is shown in (8).

[0022] K0=ε L (…ε i+1 ([E i F i ])) (6);

[0023]

[0024] S o =R+X(1-R) ​​(8).

[0025] The specific process of step 3 is as follows: constrain the network model obtained in step 2 using the adversarial loss function, reconstruction loss function, attention loss function and feature matching loss, and then back-broadcast to update the parameters to obtain the trained attention-based generative adversarial network model for suppressing X-ray rib images.

[0026] In step 3, the adversarial loss function The following formula (9) is used: where D and G represent the discriminator and generator, respectively, G is a function of the standard chest X-ray, generating a virtual bone suppression image for the discriminator to distinguish, and E represents the error; P data Represents the data distribution, x ~ P data(x) and s~P data(s) This indicates the selection of s and x from the data distributions on chest X-ray and rib-suppressed images, respectively:

[0027]

[0028] Reconstruction loss function L rec(G) is represented by the following formula (10): where S represents the ground truth value of the bone-suppressed image, X is the input regular chest X-ray, and L1 loss||.||1 is used to calculate the difference between pixels. The generator G will learn the mapping between the bone-suppressed image and the regular chest X-ray:

[0029]

[0030] Attention loss function L R The following formula (11) is used to represent this: where W, H, and C are the width, height, and number of channels of the attention map R, respectively:

[0031]

[0032] The feature matching loss function is expressed by the following formula (12): where D i The function representing the discriminator's handling of the feature map information obtained from the i-th layer of the input image:

[0033]

[0034] The total loss function L is:

[0035]

[0036] Among them, λ1, λ2, λ3 and λ4 are hyperparameters that control the relative importance of each loss term.

[0037] The beneficial effects of this invention are as follows: This invention proposes an X-ray rib image suppression algorithm based on a generative network using joint predictive filtering. This invention addresses the limitations of previous rib image suppression methods in extracting feature information and the resulting lack of sufficient feature information, leading to poor suppression performance. By introducing deep predictive filtering technology, the generative network performs feature-level filtering after the encoder and image-level filtering after the decoder, thereby enhancing the contextual relevance between information and adding more detailed information. Ultimately, this further enhances network performance and achieves more robust information extraction. Simultaneously, a feature matching loss function is added to further constrain the bone suppression image obtained by the generative network at the feature level, further accelerating the convergence speed and improving the precision of the network towards the target of X-ray rib suppression. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the method for suppressing rib images on chest X-rays based on a generator network using joint predictive filtering, as described in this invention.

[0039] Figure 2 This is a schematic diagram of the overall structure of the X-ray rib image suppression method based on a generator network with joint predictive filtering according to the present invention.

[0040] Figure 3 This is the fusion attention module of the X-ray rib image suppression method based on a generative network with joint predictive filtering in this invention;

[0041] Figure 4 This diagram illustrates the bone suppression image fusion generation process of a chest X-ray rib image suppression method based on a joint predictive filtering generative network.

[0042] Figure 5 This is a comparison of the results of the X-ray rib image suppression method based on the generator network of joint predictive filtering according to the present invention. Detailed Implementation

[0043] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0044] This invention relates to a method for suppressing rib images on chest X-rays based on a generative network with joint predictive filtering, such as... Figure 1 As shown, the specific steps include the following:

[0045] Step 1: Preprocess the images in the training and test sets of the dataset; specifically:

[0046] This invention enhances the data through operations such as rotation and translation. It uses the dataset from the Japanese Society for Radiological Technology (JSRT), which, after augmentation, comprises 4080 paired images, each a PNG file with a resolution of 1024*1024 and a bit depth of 8. The dataset is trained using a pre-defined training dataset, and tested using a test set.

[0047] Step 2, Figure 2 This is a schematic diagram of the overall structure of a chest X-ray rib image suppression method based on a joint predictive filtering generative network. The model consists of a bone suppression generator module, a bone suppression kernel prediction filter module (KPN module), and a bone suppression discriminator module. The bone suppression kernel prediction module adaptively predicts the pixel kernels of the rib region based on the filter reconstruction according to the input image and feeds them to the bone suppression generator module to achieve intelligent combination of filtering and generated bone suppression results, thereby generating more realistic chest X-ray rib suppression images. Finally, the discriminator distinguishes between real and fake bone suppression images.

[0048] The specific steps are as follows:

[0049] Step 2.1: The image X in the dataset is used as input to both the bone suppression generator and the bone suppression kernel prediction filter module. The generator's encoder extracts image features F at multiple different scales sequentially through convolution operations. i(i represents the i-th layer feature), (as in formula (1)), whose dimensions are 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the input image, respectively. Predictive filtering adopts deep CNN adaptive prediction, which makes the prediction kernel highly matched to each image. Removing rib images from chest X-rays by predictive filtering fusion generation network is an effective method to improve the quality of bone suppression images. The kernel prediction filtering module performs two filtering operations in the generator: feature-level filtering in the encoder stage of the generator and image-level filtering in the decoder stage of the generator. The prediction kernel for the first feature-level filtering is K. l As shown in formula (2).

[0050] F i =Encoder(X) (1);

[0051] K l =Conv(ε l (…ε i+1 ([E i F i ])) (2);

[0052] in, and E i (E i =ε i (…ε1(X))) represent the i-th layer features of the generator network and the i-th layer of the kernel prediction filter network, respectively.

[0053] Step 2.2, perform feature-level filtering on feature F a Channel-space correction is performed using a fusion attention module. The fusion attention module, for example... Figure 3 As shown, the internal correction adopts the SKnet attention mechanism principle to complete the feature correction of the channel dimension as shown in formula (3). In order to maintain the consistency of the global structure distribution of features after pixel detail information correction, this invention adds a spatial attention block outside SKnet to calculate the spatial coefficient g, as shown in formula (4). The feature aggregation combining spatial and channel changes accelerates the convergence speed of the network to generate target images while ensuring the overall structure layout of the feature attention map. The stacked fusion attention block greatly enriches the network depth of the generator, allowing the generator to capture as much rib region information as possible and form a more complete rib attention map.

[0054] Specifically, feature F aConvolutional operations with kernel size of 3x3 and dilated convolutions of r=1, 2, 4, and 8 are performed respectively. Different dilation convolutions allow the sub-kernels to focus on different receptive fields, fully considering both the overall and detailed information of image features. Feature information at different scales is fused by element-wise addition. Then, global average pooling (GAP) is used to obtain global information for each channel. Two fully connected operations (FC) complete the information exchange across the channel dimensions. Softmax normalization yields weight values ​​corresponding to different scales, which are multiplied by the corresponding features to be corrected for feature correction. Finally, the corrected feature information is fused by element-wise addition to obtain the final fused feature C5, thus completing the internal channel correction. To maintain the consistency of the global structure distribution of features after pixel detail information correction, this invention adds a spatial attention block outside SKNet to calculate the spatial coefficient g. The input features are then subjected to a dilated convolution of r=1 followed by a sigmoid operation to calculate the feature F. a Spatial variation threshold g, this value is compared with F b and input features F a The final aggregated feature F is obtained by weighted summation using the formula as shown in formula (5):

[0055] F b =Sknet(F a (3);

[0056] g = Gate(F) a (4);

[0057] F = F a *g+F b *(1-g) (5);

[0058] Step 2.3: Based on the aggregated features F obtained in Step 2.2, after passing through the decoder of the generative adversarial network, image-level filtering is performed with the image-level prediction kernel K0 from the bone suppression kernel prediction filtering module (as shown in Equation (6)) to further enhance the rib detail information. The final output is the rib attention map learned adaptively by the network. As shown in Equation (7), the rib attention map (rib attention R) obtained from the generator and the residual rib attention map (residual ribattention 1-R) are fused with the input chest X-ray to obtain the bone-suppressed soft tissue image S. o (output), the formula is shown in (8), and the image fusion visualization process is as follows: Figure 4 As shown.

[0059] K0=ε L (…ε i+1 ([E i F i ])) (6);

[0060]

[0061] S o =R+X(1-R) ​​(8).

[0062] Step 3: Train the model using the preprocessed training set data from Step 1. Constrain the network model obtained in Step 2 using adversarial loss, reconstruction loss, feature matching loss, and attention loss functions. Then, perform backpropagation to update the parameters. After 200 training rounds (one round refers to training all the preprocessed images once), a trained X-ray rib image suppression network model based on a joint prediction filter is obtained. The specific loss functions used are shown below:

[0063] Adversarial loss function: D and G represent the discriminator and generator, respectively. G is a function of the standard chest X-ray, generating a virtual bone-suppressed image for the discriminator to distinguish. E represents the error; P data Represents the data distribution; and x ~ P data(x) and s~P data(s) This indicates that s and x are selected from the data distributions on the chest X-ray and rib suppressed images, respectively. An adversarial discriminator D is used to distinguish whether the generated image is a real soft tissue image.

[0064]

[0065] Reconstruction loss function: where S represents the ground truth value of the bone-suppressed image, and X is the input regular chest X-ray. L1 loss ||.||1 is used to calculate the difference between pixels. The generator G will learn the mapping between the bone-suppressed image and the regular chest X-ray.

[0066]

[0067] Attention Loss Function: The mask automatically learned by the network can easily saturate to 1, preventing the generator from achieving ideal results. To prevent this, total variation regularization is applied to the mask R as a constraint. W, H, and C are the width, height, and number of channels of the attention map R, respectively. To generate the target soft tissue image So:

[0068]

[0069] Feature matching loss function: In the discriminator comparison stage, each layer feature map in the bone suppression image obtained by the generator network needs to be compared with the feature map information of the label soft tissue image. Further constraints are applied from the overall structure to detailed information, thereby further constraining the network to generate a bone suppression image that is almost identical to the label. The formula is as follows:

[0070]

[0071] Among them, D i This represents a function that describes the feature map information obtained by the discriminator from the i-th layer of the input image.

[0072] Therefore, the total loss of this invention is:

[0073]

[0074] Among them, λ1, λ2, λ3 and λ4 are hyperparameters that control the relative importance of each loss term.

[0075] Step 4: Input the test set images processed in Step 1 into the model trained in Step 3 to finally obtain soft tissue images after bone suppression.

[0076] Example 1

[0077] The method for suppressing rib images on chest X-rays based on a generative network using joint predictive filtering includes the following steps:

[0078] Step 1: Preprocess the images in the training and test sets of the dataset;

[0079] Step 2: Construct a network model for suppressing rib images on chest X-rays based on a generative network of joint predictive filtering.

[0080] Step 3: Use the training set data preprocessed in Step 1 to train the model constructed in Step 2, and obtain the trained X-ray rib image suppression network model based on the joint prediction filter generator network.

[0081] Step 4: Input the preprocessed test set images from Step 1 into the model trained in Step 3 to finally obtain soft tissue images after bone suppression.

[0082] Example 2

[0083] Based on Example 1, in step 2, the X-ray rib image suppression network model based on the joint prediction filtering generation network includes a bone suppression generator module (Generator), a bone suppression kernel prediction filtering module (KPN), and a bone suppression discriminator module (Discriminator). The bone suppression kernel prediction filtering module (KPN) adaptively predicts the pixel kernel of the rib region based on the filter reconstruction according to the input image, and feeds the prediction result to the bone suppression generator module (Generator) to realize the combination of filtering and generating bone suppression results, thereby generating an X-ray rib suppression image. Finally, the discriminator distinguishes between true and false bone suppression images.

[0084] Example 3

[0085] Based on Example 2, the process of suppressing X-ray rib images using the X-ray rib image suppression network model based on the joint prediction filtering generator network is as follows:

[0086] Step A) uses the image X in the dataset as input to the generator, and the encoder sequentially extracts image features F at multiple different scales. i As shown in formula (1) below; the bone suppression kernel prediction filter module for F i Perform feature-level filtering kernel K l As shown in formula (2) below:

[0087] F i =Encoder(X) (1);

[0088] K l =Conv(ε l (…ε i+1 ([E i F i ])) (2);

[0089] Step B) involves filtering the features F at the feature level. a The internal and external attention modules of the fusion attention mechanism are fused for correction. The internal correction adopts the SKnet attention mechanism principle to complete the feature correction of the channel dimension as shown in formula (3). A spatial attention block Gate is added outside SKnet to calculate the spatial coefficient g, as shown in formula (4). The input features are then processed by a dilated convolution r=1 and then the feature F is calculated by a sigmoid operation. a Spatial variation threshold g, this value is compared with F b and input features F a The final aggregated feature F is obtained by weighted summation using the formula, as shown in formula (5):

[0090] F b =Sknet(F a (3);

[0091] g = Gate(F) a (4);

[0092] F = F a *g+F b *(1-g) (5);

[0093] Step C), based on the aggregated features F obtained in step B), after passing through the decoder of the generative adversarial network, it is filtered at the image level with the image-level prediction kernel K0 from the bone suppression kernel prediction filtering module, as shown in Equation (6), to further enhance the rib detail information. Finally, the rib attention map R, which is adaptively learned by the network, is output, as shown in Equation (7). The rib attention map R obtained from the generator and the residual rib attention map are then processed. Figure 1 -R is fused with the input chest X-ray to obtain a soft tissue image S with bone suppression. o The output formula is shown in (8).

[0094] K0=ε L (…ε i+1 ([E i F i ])) (6);

[0095]

[0096] S o =R+X(1-R) ​​(8).

[0097] The method of this invention is compared with existing benchmark methods, and the comparison results are as follows: Figure 5 As shown. Figure 5 Column I is the input image. Figure 5 Columns II through V are bone suppression images obtained using existing benchmark methods. Figure 5 Column VI shows the bone suppression image obtained by this method, and column VII shows the labeled bone suppression image. From Figure 5 The comparison results show that the present invention, while completing the bone suppression task, not only ensures image clarity but also fully restores detailed texture information. Table 1 presents the quantitative results of the present method and the comparative methods, demonstrating that the present method performs best in all evaluation metrics.

[0098] Table 1

[0099] Group Method PSNR↑ SSIM↑ RMSE↓ Suzuki et al. MTANN 32.451 0.955 0.023 Gusarev et al. AELike Model 32.590 0.963 0.015 Rajaraman et al. Res-BS 33.067 0.949 0.015 Rajaraman et al. DeBoNet 33.865 0.981 0.013 Ours Method of the present invention 42.661 0.990 0.006

Claims

1. A method for suppressing rib images on chest X-rays based on a generative network using joint predictive filtering, characterized in that: Specifically, the steps include the following: Step 1: Preprocess the training and test set images in the dataset; the preprocessing process in Step 1 is as follows: rotate and translate the images in the dataset to achieve data augmentation; Step 2: Construct an X-ray rib image suppression network model based on a joint predictive filtering generator network; the X-ray rib image suppression network model based on a joint predictive filtering generator network in Step 2 includes a bone suppression generator module Generator, a bone suppression kernel predictive filtering module KPN, and a bone suppression discriminator module Discriminator. The bone suppression kernel prediction filter module KPN adaptively predicts the pixel kernel of the rib region based on the filter reconstruction according to the input image, and feeds the prediction result to the bone suppression generator module Generator to realize the combination of filtering and generating bone suppression results, thereby generating a chest X-ray rib suppression image. Finally, the discriminator distinguishes between real and fake bone suppression images. The process of the X-ray rib image suppression network model based on the joint prediction filtering generator network for suppressing X-ray rib images is as follows: Step A) uses the image X in the dataset as input to the generator, and the encoder sequentially extracts image features at multiple different scales. As shown in formula (1) below; the bone suppression kernel prediction filter module for... Perform feature-level filtering kernels As shown in formula (2) below: Step B), the features after feature-level filtering The internal and external attention modules of the fusion attention mechanism are fused for correction. The internal correction adopts the SKnet attention mechanism principle to complete the feature correction of the channel dimension as shown in formula (3). A spatial attention block Gate is added outside SKnet to calculate the spatial coefficient g, as shown in formula (4). The input features are then processed by a dilated convolution r=1 and then the features are calculated by a sigmoid operation. Spatial variation threshold g, this value is compared with and input features The final aggregated feature F is obtained by weighted summation using the formula, as shown in formula (5): （3） （4） （5） Step C), based on the aggregated features F obtained in step B), after passing through the decoder of the generative adversarial network, is compared with the image-level prediction kernel from the bone suppression kernel prediction filter module. As shown in Equation (6), image-level filtering is performed to further enhance the rib detail information, and the final output is the rib attention map R learned by network adaptive learning, as shown in Equation (7). The rib attention map R obtained from the generator and the residual rib attention map 1-R are fused with the input chest X-ray to obtain a bone-suppressed soft tissue image. The output formula is shown in (8). Step 3: Use the training set data preprocessed in Step 1 to train the model constructed in Step 2, and obtain the trained X-ray rib image suppression network model based on the joint prediction filter generator network. Step 4: Input the preprocessed test set images from Step 1 into the model trained in Step 3 to finally obtain soft tissue images after bone suppression.

2. The method for suppressing rib images on chest X-rays based on a generative network using joint predictive filtering as described in claim 1, characterized in that: The specific process of step 3 is as follows: constrain the network model obtained in step 2 using the adversarial loss function, reconstruction loss function, attention loss function and feature matching loss, and then back-broadcast to update the parameters to obtain the trained attention-based generative adversarial network model for suppressing X-ray rib images.

3. The method for suppressing rib images on chest X-rays based on a generative network using joint predictive filtering as described in claim 2, characterized in that: In step 3, the adversarial loss function The following formula (9) is used to represent: where D and G represent the discriminator and generator respectively, G is a function of the standard chest X-ray to generate a virtual bone suppression image for the discriminator to distinguish, and E represents the error; Indicates data distribution, and This indicates the selection of s and x from the data distributions on chest X-ray and rib-suppressed images, respectively: （9） Reconstruction loss function The following formula (10) is used: where S represents the ground truth value of the bone-suppressed image, X is the input regular chest X-ray, and L1 loss ||.||1 is used to calculate the difference between pixels. The generator G will learn the mapping between the bone-suppressed image and the regular chest X-ray: （10） Attention loss function The following formula (11) is used to represent this: where W, H, and C are the width, height, and number of channels of the attention map R, respectively: （11） The feature matching loss function is expressed by the following formula (12): where The function representing the discriminator's handling of the feature map information obtained from the i-th layer of the input image: （12） The total loss function L is: （13） in, , and It is a hyperparameter that controls the relative importance of each loss term.

Images