A chest radiograph rib extraction method based on a generative adversarial network

Abstract

The application discloses a chest radiograph rib extraction method based on an adversarial generation network, and comprises the following steps: obtaining a chest radiograph image pair through dual-energy subtraction to construct a data set, wherein the chest radiograph image pair comprises one frame of standard chest radiograph I C , one frame of soft tissue image I S and one frame of bone image I B ; performing gray histogram equalization processing on the standard chest radiograph I C , the soft tissue image I S and the bone image I B respectively to obtain I CB and I CS ; using pairs of I C , I CB and I CS from the same individual to construct a rib generation data set, constructing rib boundary supervision data sets according to I C , I B and I CS of each individual in the data set according to a construction rule; inputting the rib boundary supervision data set into a rib supervision network, calculating the similarity loss between three feature maps of the image pair, inputting the constructed gray image into the rib supervision network for training, and completing the training of the rib supervision network; constructing a rib image generation network and training the rib image generation network; and inputting the collected chest radiograph image into the trained rib image generation network to obtain the rib image in the chest radiograph image.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, specifically a method for extracting ribs from chest X-rays based on generative adversarial networks. Background Technology

[0002] In the medical field, chest radiographs (CXR) and computed tomography (CT) are the two most commonly used imaging techniques in medical diagnosis. Both can be used to identify various diseases that may occur in the ribs, thoracic vertebrae, organs, and soft tissues contained in the thoracic cavity. Both techniques utilize the principle that the different densities of human anatomical tissues such as muscles, soft tissues, organs, and bones result in different absorption capacities of X-rays. Chest radiographs use X-rays to irradiate the chest cavity and create images on the film. The soft tissue images contained in the radiographs are often used to diagnose lung diseases such as tuberculosis, pneumonia, masses, and lung cancer, as well as gastrointestinal diseases such as intestinal obstruction and foreign bodies in the gastrointestinal tract. The skeletal images contained in the radiographs are often used to diagnose bone tumors, rib and clavicle fractures, intercostal abnormalities, scoliosis, rickets, or neurofibromatosis. Chest CT scans use precisely collimated X-rays and highly sensitive detectors to perform cross-sectional scans around the chest cavity, producing cross-sectional or axial images. These images are then reconstructed to obtain chest CT images, which can be used to diagnose diseases such as bronchial abnormalities, lymph node tuberculosis, lung cancer, and skeletal abnormalities.

[0003] While both chest X-rays and chest CT scans offer distinct advantages and can aid in medical diagnosis, they also have their limitations. For instance, the overlapping of multiple anatomical structures (clavicle, ribs, organs, and muscles) in a chest X-ray increases the difficulty for doctors to interpret the images and may even obscure lesions, making diagnosis highly dependent on clinical experience. Studies have shown that suppressing bone structures in chest X-rays can improve the detection performance of lung diseases. By locating the edges of skeletal structures, including the ribs and clavicle, effective label priors can be constructed, allowing for better optimization of bone suppression in chest X-rays and providing better diagnostic results for chest diseases. Furthermore, extracting more complete skeletal information also helps doctors diagnose rib and clavicle disorders such as fractures, intercostal abnormalities, scoliosis, rickets, and neurofibromatosis. Although chest CT scans can also clearly provide skeletal information for disease assessment, the radiation emitted by a single chest CT scan is tens to hundreds of times greater than that of a chest X-ray, with the radiation dose depending primarily on the CT equipment. Some studies have shown that the radiation from a chest CT scan can be up to 150 times that of a conventional chest X-ray. Besides the radiation dose to the patient, the cost of a chest CT scan is tens or even hundreds of times higher than that of a regular chest X-ray. A regular chest X-ray typically costs tens of yuan, while a chest CT scan costs hundreds or even thousands of yuan. In remote, economically underdeveloped areas or community hospitals, many medical institutions do not even have dedicated CT equipment due to the higher cost of CT equipment compared to regular chest X-ray equipment, and can only take regular chest X-rays. In such cases, extracting useful bone information from regular chest X-rays for early diagnosis of bone diseases becomes particularly important.

[0004] In conclusion, the ability to significantly extract and depict skeletal information from chest X-rays, especially the ribs and clavicle, has important research significance and application value. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for rib extraction from chest X-rays based on generative adversarial networks, comprising the following steps:

[0006] Step 1: Obtain chest radiograph image pairs through dual-energy subtraction to construct a dataset. The chest radiograph image pairs include one standard chest radiograph I. C 1 frame soft tissue image I S and 1 frame of skeletal image I B ;

[0007] Step 2, collect the standard chest X-rays I in pairs from the dataset. C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each image and adjust the image size to the set value. Then, use the standard chest X-ray image I. C Compared with soft tissue image IS and skeletal images I B I is obtained by performing a saturated image subtraction operation. CB and I CS ;I CB Generate datasets and I for ribs CS This is a supervised dataset for rib boundaries;

[0008] Step 3, using paired I from the same individual C I CB and I CS Construct a rib generation dataset, and analyze the I of each individual in the dataset. C I B and I CS A rib boundary supervision dataset is constructed based on the construction rules;

[0009] Step four: Construct the rib-based supervised network. During the training phase, the rib-based supervised network employs a weight-sharing ternary network architecture, while during the application phase, it uses a weight-sharing Siamese network architecture. The input to the rib-based supervised network is image pairs from the rib boundary supervision dataset, and the output is the feature map of the corresponding image pair. The similarity loss between the three feature maps of the image pair is calculated.

[0010] Step 5: Train the rib supervision network, and then construct the (I) C I B I CS The grayscale image pairs were resized to... The tensor is then input into the rib supervision network for training, thus completing the training of the rib supervision network;

[0011] Step 6: Construct a rib image generation network. Based on the principle of generative adversarial network model, the rib image generation network consists of a generator, a discriminator, and a supervisor. The generator adopts a U-net structure with a long jump connection, the discriminator adopts a multi-layer convolutional neural network structure, and the supervisor is a trained rib supervision network. The constructed rib image generation network is then trained to obtain the trained rib image generation network.

[0012] Step 7: Train the rib image generation network. Input the image pairs from the constructed rib generation dataset into the trained rib image generation network to obtain the rib images in the chest X-ray image.

[0013] Furthermore, the aforementioned method of centralizing standard chest X-rays in pairs... C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each image and adjust the image size to the set value. Then, use the standard chest X-ray image I. CCompared with soft tissue image I S and skeletal images I B I is obtained by performing a saturated image subtraction operation. CB and I CS ;I CB Generate datasets and I for ribs CS The rib boundary supervised dataset includes:

[0014] Standard chest X-rays I from the dataset C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each sample and adjust their size to the set dimensions. Use a standard chest X-ray I. C Compared with soft tissue image I S and skeletal images I B Perform saturated image subtraction Get I CB and I CS ,in The subtraction operation The following formula is used:

[0015]

[0016] Furthermore, the use of paired I from the same individual C I CB and I CS Construct a rib generation dataset, and analyze the I of each individual in the dataset. C I B and I CS The rib boundary supervision dataset is constructed according to the construction rules, including the following process:

[0017] Using paired I from the same individual C I CB and I CS A rib generation dataset was constructed for use in a model to reconstruct skeleton images. Based on contrastive learning, the I of each individual in the dataset was analyzed. C I B and I CS A rib boundary supervision dataset is constructed according to the established rules to train a neural network model highly sensitive to rib information and supervise the generation process of the generative network. The specific construction rules are as follows: As a positive sample, For negative samples:

[0018] ①I C I B and I CS All images are from the same individual, and the constructed image pairs are

[0019] ②I C I B They all come from the same individual, I CS The image pairs constructed from different individuals are

[0020] ③I C I CS They all come from the same individual, I B The image pairs constructed from different individuals are

[0021] ④I B I CS They all come from the same individual, I C The image pairs constructed from different individuals are

[0022] ⑤I C I B and I CS All images are from different individuals, and the constructed image pairs are

[0023] Furthermore, the aforementioned rib-supervised network employs a weight-sharing ternary network architecture during the training phase and a weight-sharing Siamese network architecture during the application phase. The input to the rib-boundary supervision dataset consists of image pairs, and the output is the feature map of the corresponding image pair. The similarity loss between the three feature maps of the image pair is calculated. The process includes the following:

[0024] Input constructed (I C I B I S Grayscale image pairs are processed by an octave-convolutional residual neural network to extract image pairs (I). C I B I CS For each image in the dataset, feature maps (f1, f2, f3) are used, and the similarity loss between the three feature maps is calculated.

[0025]

[0026]

[0027] Where Cos_Sim(f i ,f j ,l ij ) is the feature map f i With feature map f j The distance, l ij For feature map fi With feature map f j The label indicates whether the feature map comes from the same individual. If it does, the value is 1; otherwise, it is -1. The margin is a hyperparameter that represents the distance boundary between feature map fi and feature map fj, and its value ranges from -1 to 1.

[0028] Furthermore, the generator employs a U-net structure with added long jump connections, including:

[0029] The U-net structure of the generator contains four encoding modules in the encoder part. Each encoding module contains two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. There is a downsampling layer with a 2×2 convolutional kernel between each encoding module. The decoder part contains four decoding modules. Each decoding module contains two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. There is also a deconvolution layer with a 2×2 convolutional kernel between each decoding module. Finally, the generated skeleton image is obtained through a 1×1 deconvolution layer.

[0030] Furthermore, the discriminator is a multi-layer convolutional neural network structure, wherein each layer of the network includes a convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer.

[0031] The beneficial effects of this invention are: this method does not require any annotation of rib information during practice, saving a lot of time and cost; unlike the problem of poor adaptability in rigid model strong fitting of rib shape, this method uses neural network to extract rib information from image, which can effectively adapt to deformed ribs or spine, and has strong adaptability. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating a method for rib extraction from chest X-rays based on generative adversarial networks.

[0033] Figure 2 Standard chest X-ray I C Soft tissue images I S Skeletal Image I B Image pairs;

[0034] Figure 3 I is obtained by subtracting the saturated image. CB with I CS image;

[0035] Figure 4 This is a diagram of the rib-based supervisory network architecture.

[0036] Figure 5 To generate a network structure diagram;

[0037] Figure 6This is a schematic diagram comparing the original chest X-ray image (left) with the rib image obtained using this method (right). Detailed Implementation

[0038] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, and not all of them. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0040] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention. It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0041] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0042] The features and performance of the present invention will be further described in detail below with reference to embodiments. For example... Figure 1 As shown, a method for rib extraction from chest X-rays based on generative adversarial networks includes the following steps:

[0043] Step 1: Obtain chest radiograph image pairs through dual-energy subtraction to construct a dataset. The chest radiograph image pairs include one standard chest radiograph I. C 1 frame soft tissue image I S and 1 frame of skeletal image I B ;

[0044] Step 2, collect the standard chest X-rays I in pairs from the dataset. CSoft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each image and adjust the image size to the set value. Then, use the standard chest X-ray image I. C Compared with soft tissue image I S and skeletal images I B I is obtained by performing a saturated image subtraction operation. CB and I CS ;I CB Generate datasets and I for ribs CS This is a supervised dataset for rib boundaries;

[0045] Step 3, using paired I from the same individual C I CB and I CS Construct a rib generation dataset, and analyze the I of each individual in the dataset. C I B and I CS A rib boundary supervision dataset is constructed based on the construction rules;

[0046] Step four: Construct the rib-based supervised network. During the training phase, the rib-based supervised network employs a weight-sharing ternary network architecture, while during the application phase, it uses a weight-sharing Siamese network architecture. The input to the rib-based supervised network is image pairs from the rib boundary supervision dataset, and the output is the feature map of the corresponding image pair. The similarity loss between the three feature maps of the image pair is calculated.

[0047] Step 5: Train the rib supervision network, and then construct the (I) C I B I CS The grayscale image pairs were resized to... The tensor is then input into the rib supervision network for training, thus completing the training of the rib supervision network;

[0048] Step 6: Construct a rib image generation network. Based on the principle of generative adversarial network model, the rib image generation network consists of a generator, a discriminator, and a supervisor. The generator adopts a U-net structure with a long jump connection, the discriminator adopts a multi-layer convolutional neural network structure, and the supervisor is a trained rib supervision network. The constructed rib image generation network is then trained to obtain the trained rib image generation network.

[0049] Step 7: Train the rib image generation network. Input the image pairs from the constructed rib generation dataset into the trained rib image generation network to obtain the rib images in the chest X-ray image.

[0050] Furthermore, the aforementioned method of centralizing standard chest X-rays in pairs...C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each image and adjust the image size to the set value. Then, use the standard chest X-ray image I. C Compared with soft tissue image I S and skeletal images I B I is obtained by performing a saturated image subtraction operation. CB and I CS ;I CB Generate datasets and I for ribs CS The rib boundary supervised dataset includes:

[0051] Standard chest X-rays I from the dataset C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each sample and adjust their size to the set dimensions. Use a standard chest X-ray I. C Compared with soft tissue image I S and skeletal images I B Perform saturated image subtraction Get I CB and I CS ,in The subtraction operation The following formula is used:

[0052]

[0053] The use of paired I from the same individual C I CB and I CS Construct a rib generation dataset, and analyze the I of each individual in the dataset. C I B and I CS The rib boundary supervision dataset is constructed according to the construction rules, including the following process:

[0054] Using paired I from the same individual C I CB and I CS A rib generation dataset was constructed for use in a model to reconstruct skeleton images. Based on contrastive learning, the I of each individual in the dataset was analyzed. C I B and I CS A rib boundary supervision dataset is constructed according to the established rules to train a neural network model highly sensitive to rib information and supervise the generation process of the generative network. The specific construction rules are as follows: As a positive sample, For negative samples:

[0055] ⑥I C I B and I CS All images are from the same individual, and the constructed image pairs are

[0056] ⑦I C I B They all come from the same individual, I CS The image pairs constructed from different individuals are

[0057] ⑧I C I CS They all come from the same individual, I B The image pairs constructed from different individuals are

[0058] ⑨I B I CS They all come from the same individual, I C The image pairs constructed from different individuals are

[0059] ⑩I C I B and I CS All images are from different individuals, and the constructed image pairs are

[0060] A rib-based supervised network is constructed. During the training phase, a weight-sharing ternary network architecture is used, while during the application phase, a weight-sharing Siamese network architecture is employed. The input to the rib boundary supervised network consists of image pairs from the rib boundary supervised dataset, and the output is the feature map of the corresponding image pair. The similarity loss between the three feature maps of the image pair is calculated. The process includes the following:

[0061] Input constructed (I C I B I S Grayscale image pairs are processed by an octave-convolutional residual neural network to extract image pairs (I). C I B I CS For each image in the dataset, feature maps (f1, f2, f3) are used, and the similarity loss between the three feature maps is calculated.

[0062]

[0063]

[0064] Where Cos_Sim(f i ,f j,l ij ) is the feature map f i With feature map f j The distance, l ij For feature map f i With feature map f j The label indicates whether the feature map comes from the same individual. If it does, the value is 1; otherwise, it is -1. The margin is a hyperparameter that represents the distance boundary between feature map fi and feature map fj, and its value ranges from -1 to 1.

[0065] The generator employs a U-net structure with added long jump connections, including:

[0066] The U-net structure of the generator contains four encoding modules in the encoder part. Each encoding module contains two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. There is a downsampling layer with a 2×2 convolutional kernel between each encoding module. The decoder part contains four decoding modules. Each decoding module contains two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. There is also a deconvolution layer with a 2×2 convolutional kernel between each decoding module. Finally, the generated skeleton image is obtained through a 1×1 deconvolution layer.

[0067] The discriminator is a multi-layer convolutional neural network structure, wherein each layer of the network includes a convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer.

[0068] Specifically, the rib segmentation network model designed in this method consists of a generative model and a supervised model. The generative model is a generative adversarial network (GAN) structure responsible for generating skeleton images, while the supervised model is a triple network using octave convolutions, responsible for supervising and completing the generated skeleton information. Utilizing the idea of ​​contrastive learning, a dataset is reconstructed from images with complete skeletal information, standard chest X-ray images, and skeletal images. This reconstructed dataset is then fed into the triple network using octave convolutions for training, resulting in a neural network model highly sensitive to rib information. This model is then fed into the GAN to supervise the generation process.

[0069] This invention constructs a dataset by collecting paired image pairs obtained through dual-energy subtraction angiography, wherein each image pair includes one standard chest radiograph I. C (Chest Radiograph, CXR), 1-frame soft tissue image I S (Soft-tissue Image) and 1 frame of skeletal image I B (Bone Image), such as Figure 2 As shown.

[0070] Standard chest X-rays I from the dataset C Soft tissue images I Sand skeletal images I B Perform grayscale histogram equalization on each image and adjust its size to 720*720. Use a standard chest X-ray I. C Compared with soft tissue image I S and skeletal images I B Perform saturated image subtraction Get I CB and I CS ,like Figure 3 As shown, where

[0071]

[0072] This method utilizes a standard chest X-ray (I) C Soft tissue images I S Skeletal Image I B I obtained by subtracting the saturated image CB and I CS Two datasets are constructed: a rib generation dataset and a rib boundary supervision dataset.

[0073] Using paired I from the same individual C I CB and I CS A rib generation dataset was constructed for use in a model to reconstruct skeleton images. Based on the idea of ​​contrastive learning, the I of each individual in the dataset was analyzed. C I B and I CS A rib boundary supervision dataset is constructed according to the construction rules to train a neural network model that is highly sensitive to rib information and to supervise the generation process of the generative network. The specific construction rules are as follows, where P represents positive samples and N represents negative samples:

[0074] I C I B and I CS All images are from the same individual, and the constructed image pairs are (I C I B_P I CS_P );

[0075] I C I B They all come from the same individual, I CS The image pairs constructed from different individuals are (I C I B_P I CS_N );

[0076] I C ICS They all come from the same individual, I B The image pairs constructed from different individuals are (I C I B_N I CS_P );

[0077] I B I CS They all come from the same individual, I C The image pairs constructed from different individuals are (I C I B_N I CS_N );

[0078] I C I B and I CS All images are from different individuals, and the constructed image pairs are (I C_N I B_N I CS_N );

[0079] For each individual in the source dataset, this construction rule can construct at least 5 pairs of images for supervised model learning.

[0080] Establish a rib monitoring network

[0081] The rib-supervised network adopts a weight-sharing ternary network architecture, such as... Figure 4 As shown, its input is the rib boundary supervised dataset constructed in the previous step, and the input for each step is the constructed (I) dataset. C I B I S Grayscale image pairs are extracted using an octave-convolutional residual neural network, OctConv-ResNet. C I B I CS For each image, feature maps (f1, f2, f3) are generated, and then the similarity loss between the three feature maps is calculated. Where Cos_Sim(f i ,f j ,l ij ) is the feature map f i With feature map f j The distance, l ij For feature map f i With feature map f j The label indicates whether the feature map comes from the same individual. If it does, the value is 1; otherwise, it is -1. The margin is a hyperparameter that represents the distance boundary between feature maps fi and fj, and its value ranges from -1 to 1.

[0082]

[0083]

[0084] The OctConv-ResNet residual neural network architecture with octave convolution consists of one preprocessing module, four multi-layer octave convolutional residual blocks, and one postprocessing module. The preprocessing module contains one 3×3 convolutional layer, one batch normalization (BN) layer, one ReLU activation layer, and one max-pooling layer. Each multi-layer octave convolutional residual block contains multiple residual connected convolutional layers using octave convolution, and each octave convolutional residual block is connected by a downsampling layer using octave convolution.

[0085] Training Rib Supervision Network

[0086] The constructed (I) C I B I CS The grayscale image pairs were resized to... The tensor is then input into the neural network for training, which is performed for a total of 100 rounds. The hyperparameter 'a' of the first octave convolution is... in With a out Set to a in =0,a out =a=0.5, the hyperparameter a of the last octave convolution. in With a out Set to a in =a=0.5,a out =0, the hyperparameter a in the remaining octave convolutions in With a out Set to a in =a out =a=0.5. During training, an SGD optimizer with a learning rate of 0.1 and momentum of 0.9 is used to optimize the training process. The learning rate is updated with gamma of 0.3 at rounds 50, 80, and 90 respectively. The loss function is... The hyperparameter margin is set to 0.5.

[0087] Building a rib image generation network

[0088] The rib image generation network, based on the principles of generative adversarial networks (GANs), consists of three parts: a generator, a discriminator, and a supervisor. The generator employs a U-Net structure with skip connections, the discriminator uses a multi-layer convolutional neural network, and the supervisor is a pre-trained rib supervision network. The overall network structure is shown in the diagram below. Figure 5 As shown.

[0089] The generator's U-net structure comprises four encoding modules in its encoder section. Each module contains two 3×3 convolutional layers, a batch normalization (BN) layer, and a ReLU activation layer. Each encoding module is also connected to a downsampling layer with a 2×2 convolutional kernel. The decoder section contains four decoding modules. Each decoding module contains two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. Each decoding module is also connected to a deconvolutional layer with a 2×2 convolutional kernel. Finally, a 1×1 deconvolutional layer is used to obtain the generated skeletal image. A jump connection is added between the i-th and ni-th layers if the total number of layers in the network is n. Each jump connection connects all channels of the encoder layer (i-th layer) to the channels of the decoder's mirror layer (ni-th layer).

[0090] The discriminator is a multi-layer convolutional neural network structure, where each layer of the network includes a convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer.

[0091] The supervisor is a residual neural network OctConv-ResNet structure with weight-shared twin octave convolutions, which has two image inputs and two feature map outputs.

[0092] Training a rib image generation network

[0093] The trained OctConv-ResNet octave convolutional residual neural network was used as a supervisor to connect the generator and discriminator, with its weights frozen throughout training. The generator and discriminator were trained jointly, generating image pairs (I1, I2, I3) from the same individual in the rib generation dataset. C I CB I CS After size adjustment After obtaining the tensor, it is input into the rib image generation network to obtain the I generated by the generator. CB_Fake Then [I] C ,I CB ]、[I C ,I CB_Fake The data are fed into the discriminator for training, and the loss is calculated. and The generator and discriminator are trained alternately to continuously improve the generator's generation ability and the discriminator's judgment ability. The supervisor's input is I. CS and the I generated by the generator CB_Fake The output is I CS and I CB_Fake Feature map and Calculate feature loss using two feature maps

[0094] Loss function G *Includes combat losses and feature loss Among the losses in combat for:

[0095]

[0096] The loss is:

[0097]

[0098] The loss is:

[0099]

[0100] Feature loss function for:

[0101]

[0102] Loss function G * for:

[0103]

[0104] Where G is the generator network, D is the discriminator network, and I... C For the input standard chest X-ray image, I CB Standard chest X-ray I C With skeletal image I B Obtained by subtracting saturated images I CS Standard chest X-ray I C With soft tissue image I S Obtained by subtracting saturated images I CB_Fake The image generated by the generator, I CB_Fake =G(I C (,z), where z is random input noise. and I CS and I CB_Fake Feature maps, λ1, λ2 and λ f These are the weighting coefficients.

[0105] In addition, both the generator and discriminator use the Adam optimizer for gradient updates, with a training cycle of 200 rounds and an initial learning rate of 0.02.

[0106] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for rib extraction from chest X-rays based on generative adversarial networks, characterized in that, Includes the following steps: Step 1: Obtain chest radiograph image pairs through dual-energy subtraction to construct a dataset. The chest radiograph image pairs include one standard chest radiograph I. C 1 frame soft tissue image I S and 1 frame of skeletal image I B ; Step 2, collect the standard chest X-rays I in pairs from the dataset. C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each image and adjust the image size to the set value. Then, use the standard chest X-ray image I. C Compared with soft tissue image I S and skeletal images I B I is obtained by performing a saturated image subtraction operation. CB and I CS ;I CB Generate datasets and I for ribs CS This is a supervised dataset for rib boundaries; Step 3, using paired I from the same individual C I CB and I CS Construct a rib generation dataset, and analyze the I of each individual in the dataset. C I B and I CS A rib boundary supervision dataset is constructed based on the construction rules; Step 4: Input the rib boundary supervision dataset into the rib supervision network, extract the feature map of each image pair in the rib boundary supervision dataset, obtain three feature maps for the corresponding image pair, and calculate the similarity loss between the three feature maps of the image pair. Step 5, construct (I) C I B I CS The grayscale image pairs were resized to... The tensor is then input into the rib supervision network for training, thus completing the training of the rib supervision network; Step 6: Construct a rib image generation network. Based on the principle of generative adversarial network model, the rib image generation network consists of a generator, a discriminator, and a supervisor. The generator adopts a U-net structure with a long jump connection, the discriminator adopts a multi-layer convolutional neural network structure, and the supervisor is a trained rib supervision network. The constructed rib image generation network is then trained to obtain the trained rib image generation network. Step 7: Input the acquired chest X-ray images into the trained rib image generation network to obtain rib images from the chest X-ray images; The use of paired I from the same individual C I CB and I CS Construct a rib generation dataset, and analyze the I of each individual in the dataset. C I B and I CS The rib boundary supervision dataset is constructed according to the construction rules, including the following process: Using paired I from the same individual C I CB and I CS A rib generation dataset was constructed for use in a model to reconstruct skeleton images. Based on contrastive learning, the I of each individual in the dataset was analyzed. C I B and I CS A rib boundary supervision dataset is constructed according to the construction rules to train a neural network model that is highly sensitive to rib information and to supervise the generation process of the generative network. The specific construction rules are as follows, where P represents positive samples and N represents negative samples: ①I C I B and I CS All images are from the same individual, and the constructed image pairs are (I C I B_P I CS_P ); ②I C I B They all come from the same individual, I CS The image pairs constructed from different individuals are (I C I B_P I CS_N ); ③I C I CS They all come from the same individual, I B The image pairs constructed from different individuals are (I C I B_N I CS_P ); ④I B I CS They all come from the same individual, I C The image pairs constructed from different individuals are (I C I B_N I CS_N ); I C I B and I CS All images are from different individuals, and the constructed image pairs are (I C_N I B_N I CS_N ).

2. The method for rib extraction from chest X-rays based on generative adversarial networks according to claim 1, characterized in that, The aforementioned standard chest X-rays in pairs, which are centrally paired in the dataset C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each image and adjust the image size to the set value. Then, use the standard chest X-ray image I. C Compared with soft tissue image I S and skeletal images I B I is obtained by performing a saturated image subtraction operation. CB and I CS ;I CB Generate datasets and I for ribs CS The rib boundary supervised dataset includes: Standard chest X-rays I from the dataset C Soft tissue images I S and skeletal images I B Perform grayscale histogram equalization on each sample and adjust their size to the set dimensions. Use a standard chest X-ray I. C Compared with soft tissue image I S and skeletal images I B Perform saturated image subtraction Get I CB and I CS ,in The subtraction operation The following formula is used:

3. The method for rib extraction from chest X-rays based on generative adversarial networks according to claim 1, characterized in that, The process involves inputting the rib boundary supervision dataset into the rib supervision network, extracting the feature map of each image in the image pairs from the rib boundary supervision dataset, obtaining three feature maps for the corresponding image pairs, and calculating the similarity loss between the three feature maps of the image pairs. The process includes the following: Input constructed (I C I B I S Grayscale image pairs are processed by an octave-convolutional residual neural network to extract image pairs (I). C I B I CS For each image in the dataset, feature maps (f1, f2, f3) are used, and the similarity loss between the three feature maps is calculated. Where Cos_Sim(f i ,f j ,l ij ) is the feature map f i With feature map f j The distance, l ij For feature map f i With feature map f j The label indicates whether the feature map comes from the same individual. If it does, the value is 1; otherwise, it is -1. The margin is a hyperparameter that represents the distance boundary between feature maps fi and fj, and its value ranges from -1 to 1.

4. The method for rib extraction from chest X-rays based on generative adversarial networks according to claim 1, characterized in that, The generator employs a U-net structure with added long jump connections, including: The U-net structure of the generator contains four encoding modules in the encoder part. Each encoding module contains two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. There is a downsampling layer with a 2×2 convolutional kernel between each encoding module. The decoder part contains four decoding modules. Each decoding module contains two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. There is also a deconvolution layer with a 2×2 convolutional kernel between each decoding module. Finally, the generated skeleton image is obtained through a 1×1 deconvolution layer.

5. The method for rib extraction from chest X-rays based on generative adversarial networks according to claim 1, characterized in that, The discriminator is a multi-layer convolutional neural network structure, wherein each layer of the network includes a convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer.

Images