A method for separating and extracting x-ray superimposed objects

Abstract

The application provides a method for separating and extracting X-ray overlapping objects, comprising the following steps: step S1: obtaining a separation and extraction model based on deep learning training; step S2: obtaining an X-ray overlapping object image, and inputting the X-ray overlapping object image into the separation and extraction model trained in step S1 to output separated object information; wherein, the VIT structure is used to replace the CNN structure in the traditional image generator, so that the network can better pay attention to the single object in the overlapping image, and on the other hand, more long-distance information of the overlapping image can be paid attention to. The method combines Mask R-CNN and the VIT-based generator to obtain a network capable of separating and extracting overlapping object X-ray images in an end-to-end manner, and good results are achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for separating and extracting X-ray overlapping objects. Background Art

[0002] Current developments in X-ray image processing focus primarily on image processing techniques such as denoising, enhancement, segmentation, and target recognition. These techniques, in addition to processing X-ray images, also have the same effects on ordinary grayscale or color images. Consequently, there are currently no unique X-ray image processing methods. Compared to ordinary grayscale or color images, X-ray images are most unique in that the penetrating nature of X-rays allows objects obscured by other objects to be seen in X-ray images, whereas in ordinary images, only the obstructing objects can be observed, while the obscured objects remain invisible. However, in X-ray images, the images of the obstructing objects and the obscured objects are aliased. Even if the obscured object is visible, it may be impossible to identify it due to image aliasing.

[0003] This method for separating X-ray images of overlapping objects has similar objectives to image inpainting. Image inpainting is the process of reconstructing lost or damaged portions of an image. Currently, image inpainting primarily involves traditional methods and deep learning. Traditional methods employ mathematical inference and primarily include structure-based and texture-based methods. Deep learning methods have evolved from autoencoder-based image inpainting to generative adversarial networks (GANs), with GANs currently the primary method. Traditional image inpainting can effectively repair scene images, but it cannot restore semantic information. Deep learning technology can capture image semantic information and predict missing semantic content, addressing the shortcomings of traditional inpainting methods. Therefore, deep learning technology is currently the primary development direction in the field of image inpainting.

[0004] However, when separating overlapping X-ray images, it's crucial to preserve the details of the overlapping areas, so the network must be able to take into account the pixel information in the aliased areas. This isn't something deep learning-based image restoration methods can achieve. Furthermore, there's no prior research in the field of X-ray image processing specifically addressing the separation of overlapping objects. Summary of the Invention

[0005] In view of the gap in the field of X-ray image processing in the direction of separating overlapping object images and the shortcomings of the existing technology, the present invention provides a method for separating and extracting overlapping X-ray objects, which uses a deep learning-based method to separate overlapping objects in X-ray images, combines the advantages of VIT with the advantages of GAN, and based on the mask images of overlapping and non-overlapping areas output by the Mask R-CNN network, can achieve better end-to-end separation effect for X-ray images containing overlapping objects.

[0006] In order to solve the technical problems existing in the prior art, the technical solutions of the present invention are as follows:

[0007] A method for separating and extracting overlapping X-ray objects comprises at least the following steps:

[0008] Step S1: Obtain a separation and extraction model based on deep learning training;

[0009] Step S2: Obtain X-ray overlapping object images and input them into the separation and extraction model trained in step S1 to output separated object information;

[0010] Wherein, the step S1 at least includes the following steps:

[0011] Step S11: X-ray images of common objects taken by existing security inspection devices are selected, cropped, and synthesized to obtain a dataset of X-ray overlapping object images. The dataset includes X-ray images of overlapping objects (hereinafter referred to as overlapping images), X-ray images of individual objects (hereinafter referred to as object images), and mask images of individual objects and overlapping portions (hereinafter referred to as mask images).

[0012] Step S12: Fine-tune a trained Mask R-CNN segmentation network using the dataset constructed in step S1;

[0013] Step S13: Building a separation network for overlapping object X-ray images. The network is a GAN structure, which includes a generator based on a VIT structure and a discriminator based on a convolutional network.

[0014] Step S14: using the data set constructed in step S1 to train the overlapping object X-ray image separation network constructed in step S3;

[0015] Step S15: Connect the Mask R-CNN fine-tuned in step S2 to the generator trained in step S4 to obtain the final separation model. Input the overlapping image processed in step S1 and not used for network training in step S4 to obtain the separated object image.

[0016] In step S2, the X-ray image containing overlapping objects is input into the Mask R-CNN network to obtain the corresponding mask image. The mask image and the X-ray image are input into the separation and extraction network together to finally obtain the X-ray image of the individual object.

[0017] As a further improvement, step S11 specifically includes the following steps:

[0018] Step S111: selecting images without overlapping objects from X-ray images of common objects taken by existing security inspection devices, that is, X-ray images of individual objects and background images;

[0019] Step S112: Using Photoshop software, the cluttered background in the X-ray image of the individual object obtained in step S11 is removed and changed to white. At the same time, the sizes of the X-ray image of the individual object and the background image are adjusted to obtain an object image and a background image.

[0020] Step S113: Divide the item images obtained in step S112 into two groups, group 1 and group 2. Randomly select the background image, the image of item 1 (selected from group 1), and the image of item 2 (selected from group 2) through the program to synthesize the overlapping image. The synthesis formula is expressed as follows:

[0021]

[0022] Where, I out is the pixel value of the final overlapping image, I1 is the pixel value of item 1 image, I2 is the pixel value of item 2 image, I b is the background image pixel value;

[0023] Step S114: Binarize the object image of the composite overlapping image in step S113, find the object outline, randomly select object 1 or object 2, set the value of the pixels within its outline to 255, and set the value of the pixels within both the outline of object 1 and the outline of object 2 to 128. The values ​​of the remaining pixels are set to 0 to obtain the final mask image.

[0024] Furthermore, in S112, the quick selection tool in the PS software is used to segment the individual items, separate them from the original background, add a white background, and adjust the image size to 224×224. The final number of object images and the number of background images obtained are 400 and 50, respectively.

[0025] Furthermore, in S113, group one contains 100 item images, group two contains 300 item images, and the number of channels of the synthesized overlapping images is 3.

[0026] Furthermore, in S114, the number of channels of the generated mask image is 1.

[0027] Furthermore, in S11, a total of 50,000 training set data and 400 test set data are generated.

[0028] Furthermore, in S12, the Mask R-CNN network is composed of Faster R-CNN and FCN, wherein Faster R-CNN performs classification and bounding box prediction, and FCN performs image segmentation in the screened ROI; fine-tuning is performed on the parameter model of the 91-category (including background) Mask R-CNN segmentation network trained on the coco dataset, and the categories to be segmented are modified to 3 categories, namely overlapping area, non-overlapping area and background;

[0029] Furthermore, in S13, building a separation network for overlapping object X-ray images specifically includes replacing a generator with a VIT structure instead of a traditional CNN structure and a discriminator based on a CNN structure.

[0030] Furthermore, the generator that replaces the traditional CNN structure with a VIT structure includes three parts: a data fusion layer, an encoder with a VIT structure, and a decoder with a VIT structure. The VIT structure includes three parts: Patch Embedding, Position Embedding, and a multi-layer Transformer Block. Patch Embedding converts the image from two-dimensional information into a series of one-dimensional information: Patch Embedding divides an image of dimension H×W×C into N pieces of size P. 2 The size of C's patch, N, is HW / P 2 , and reshape to N×(P 2 C) patches block; PositionEmbedding gives each patch block a unique code, namely position information. Since image features often depend on the information between adjacent pixels, adding position information to the one-dimensional image information after Patch Embedding can better improve training effects; Transformer Block contains Attention layer and MLP layer. The Attention layer performs a linear transformation on the image information with position information to obtain three feature vector groups Q:{q1,q2,…,qn}, K:{k1,k2,…,kn}, and V:{v1,v2,…,vn}. The attention weights are calculated through the multi-head self-attention mechanism and finally input into the MLP network to obtain the feature information of the image.

[0031] Furthermore, the data fusion layer in the generator is a convolutional layer with 4 input channels, 3 output channels, a convolution kernel size of 1, and a stride of 1. This convolutional layer performs a preliminary fusion of the input overlapping image and the mask image, concentrating the information of the overlapping image in the effective area, reducing the pressure on subsequent encoders and decoders.

[0032] Furthermore, the size of each patch in the Patch Embedding in the encoder is 16, so N is 196, and the dimension of the patches obtained for each image is 196×768; the specific process is completed by a convolutional layer with 3 input channels, 768 output channels, a convolution kernel size of 16, and a stride of 16; the Patch Embedding in the decoder only changes the dimension of the encoder output, and the dimension of the patches becomes 196×512.

[0033] Furthermore, the Position Embedding parameter uses the sin-cos position encoding method. The dimension of the position encoding of a patch in the encoder is 1×768, and the dimension of the position encoding of a patch in the decoder is 1×512. The encoding formula is as follows:

[0034]

[0035]

[0036] Where pos is the position of the current patch; d model is the length of the position code, which is 768 in the encoder and 512 in the decoder; i is the i-th digit in the position code. When i is an even number, the first formula is used, and when i is an odd number, the second formula is used.

[0037] Furthermore, in the encoder with the VIT structure, the number of Transformer Block layers is 6, where mlp_radio in the MLP layer is set to 4; in the decoder with the VIT structure, the number of Transformer Block layers is 4, where mlp_radio in the MLP layer is set to 4.

[0038] Furthermore, the discriminator based on the CNN structure consists of a six-layer convolutional network and a four-layer fully connected network. The convolutional layer extracts image features, and the fully connected layer performs feature analysis and determines whether the image is a real image through the sigmoid activation function.

[0039] Furthermore, in S4, the learning rate of the generator changes with the number of cycles, increasing from 0 generation by generation and decreasing generation by generation after reaching 1e-4. The Adam optimizer is used, the dropout is set to 0.3, and the loss function is the weighted sum of the absolute error loss between the decoder generated image and the real image and the discrimination loss of the discriminator for distinguishing the generated image from the real image. The specific formula is as follows:

[0040]

[0041] Where I pred For the item map generated by the generator, D(I pred ) is the judgment of the discriminator on the item image generated by the generator, and λ is the weight of the absolute error loss, which is set to 10.

[0042] Furthermore, the learning rate of the discriminator is set to 4e-4, and the Adam optimizer is used. The loss function consists of two parts: the cross entropy loss function between the discriminator output of the real image and the true value, and the cross entropy loss function between the discriminator output of the generator-generated image and the fake value. The specific formula is as follows:

[0043] L D =-[log D(I true )+log(1-D(I pred ))]

[0044] Where I true is the item graph in the dataset, D(I true ) is the judgment of the discriminator on the item graph; I pred For the item map generated by the generator, D(I pred ) is the judgment of the discriminator on the item graph generated by the generator.

[0045] Furthermore, the early stopping method is used for training the GAN network. The performance of the current model is observed during the training process. After the 25th cycle, the separation effect is the best.

[0046] Compared with the prior art, the method of the present invention has the following beneficial effects:

[0047] (1) The technical problem to be solved by the present invention is to separate overlapping objects in X-ray images, a field in which no corresponding research has been conducted. The research results of the present invention have opened up new directions in the field of X-ray image processing;

[0048] (2) The technical solution of the present invention constructs a VIT-based GAN model, using the VIT structure to replace the traditional CNN structure to create the encoder and decoder in the image generator. Compared with CNN, the VIT structure better preserves the spatial information of the input image and learns more global features of the input image; the GAN network can help generate more realistic images. This method combines the advantages of VIT and GAN, and based on the mask images of overlapping and non-overlapping areas output by the Mask R-CNN network, it can achieve a better end-to-end separation effect for X-ray images containing overlapping objects.

[0049] (3) Compared with the existing image restoration technology, this technical solution not only extracts the image features of the known area in the encoder to determine the hue of the output image, but also extracts the image features of the area to be restored to ensure the details of the output image. At the same time, the mask image is used to provide the encoder with prior knowledge of the positions of the two areas, so that the output image can achieve better results in both hue and details. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Flowchart of the training steps of the method for separating and extracting X-ray overlapping objects according to an embodiment of the present invention;

[0051] Figure 2 This is a flow chart of step S11 of the method for separating and extracting overlapping X-ray objects according to an embodiment of the present invention;

[0052] Figure 3 This is an example of a data set for the method for separating and extracting overlapping X-ray objects according to an embodiment of the present invention;

[0053] Figure 4 This is a diagram showing the Mask R-CNN output results of the X-ray overlapping object separation and extraction method according to an embodiment of the present invention;

[0054] Figure 5 This is a diagram of the VIT-based GAN network structure of the X-ray overlapping object separation and extraction method according to an embodiment of the present invention;

[0055] Figure 6 This is a Transformer Block structure diagram of the X-ray overlapping object separation and extraction method according to an embodiment of the present invention.

[0056] Figure 7 This is an example diagram of the end-to-end network process of the method for separating and extracting X-ray overlapping objects according to an embodiment of the present invention.

[0057] Figure 8 This is a result diagram of the method for separating and extracting X-ray overlapping objects according to an embodiment of the present invention.

[0058] Figure 9 This is the result of existing image restoration technology processing this problem.

[0059] Figure 10 This is a flow chart of a method for separating and extracting overlapping X-ray objects according to the present invention. DETAILED DESCRIPTION

[0060] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0061] The present invention takes the penetrability of X-rays as a starting point and finds that obscured objects can be observed in X-ray images, but the images of the obstructing objects and the obscured objects are overlapped, making it difficult to distinguish the obscured objects from the obscured objects.

[0062] To solve this technical problem, see Figure 10 , which is a flowchart of a method for separating and extracting overlapping X-ray objects according to the present invention, comprises at least the following steps:

[0063] Step S1: Obtain a separation and extraction model based on deep learning training;

[0064] Step S2: Obtain X-ray overlapping object images and input them into the separation and extraction model trained in step S1 to output separated object information;

[0065] Among them, see Figure 1 , which is a specific flow chart of step S1 of the present invention; step S1 further includes the following steps:

[0066] Step S11: X-ray images of common objects taken by existing security inspection devices are selected, cropped, and synthesized to obtain a dataset of overlapping X-ray images of objects. The dataset includes X-ray images of overlapping objects (hereinafter referred to as overlapping images), X-ray images of individual objects (hereinafter referred to as object images), and mask images of individual objects and overlapping portions (hereinafter referred to as mask images).

[0067] Step S12: Fine-tune a trained Mask R-CNN segmentation network using the dataset constructed in step S11;

[0068] Step S13: Building a separation network for overlapping object X-ray images. The network is a GAN structure, which includes a generator based on a VIT structure and a discriminator based on a convolutional network.

[0069] Step S14: using the data set constructed in step S11 to train the overlapping object X-ray image separation network constructed in step S3;

[0070] Step S15: Connect the Mask R-CNN fine-tuned in step S12 to the generator trained in step S14 to obtain the final separation model. Input the overlapping image processed in step S11 and not used for network training in step S14 to obtain the separated object image.

[0071] See also Figure 2 , S11 specifically includes the following steps:

[0072] Step S111: selecting images without overlapping objects from X-ray images of common objects taken by existing security inspection devices, that is, X-ray images of individual objects and background images;

[0073] Step S112: Using Photoshop software, the cluttered background in the X-ray image of the individual object obtained in step S111 is removed and changed to white. At the same time, the sizes of the X-ray image of the individual object and the background image are adjusted to obtain an object image and a background image.

[0074] Step S113: Divide the item images obtained in step S112 into two groups, group 1 and group 2. Randomly select the background image, the image of item 1 (selected from group 1), and the image of item 2 (selected from group 2) through the program to synthesize the overlapping image. The synthesis formula is expressed as follows:

[0075]

[0076] Where, I out is the pixel value of the final overlapping image, I1 is the pixel value of item 1 image, I2 is the pixel value of item 2 image, I b is the background image pixel value;

[0077] Step S114: Binarize the object image of the composite overlapping image in step S113, find the object outline, randomly select object 1 or object 2, set the value of the pixels within its outline to 255, and set the value of the pixels within both the outline of object 1 and the outline of object 2 to 128. The values ​​of the remaining pixels are set to 0 to obtain the final mask image.

[0078] In step S112, the quick selection tool in the PS software is used to segment the individual objects, separate them from the original background, add a white background, and adjust the image size to 224×224. The final number of object images and the number of background images obtained are 400 and 50, respectively.

[0079] In step S113, group one contains 100 item images, group two contains 300 item images, and the number of channels of the synthesized overlapping images is 3.

[0080] In step S114 , the number of channels of the generated mask image is 1.

[0081] After step S11, the obtained data set has 50,000 data in the training set and 400 data in the test set. For some data set examples, see Figure 3 .

[0082] In step S12, a trained Mask R-CNN segmentation network is fine-tuned using the dataset constructed in step S11. The input and output results of the fine-tuned network are shown in Figure 4 ;

[0083] In step S13, the separation network of overlapping object X-ray images is constructed, which specifically includes a generator using a VIT structure instead of a traditional CNN structure and a discriminator based on a CNN structure. For the specific network structure, see Figure 5 .

[0084] The generator includes three parts: a data fusion layer, an encoder with a VIT structure, and a decoder with a VIT structure. The VIT structure includes three parts: Patch Embedding, Position Embedding, and a multi-layer Transformer Block. Patch Embedding converts an image from two-dimensional information into a series of one-dimensional information: Patch Embedding divides an image of dimension H×W×C into N pieces of size P. 2 The size of C's patch, N, is HW / P 2 , and reshape to N×(P 2 C) patches block; Position Embedding gives each patch block a unique code, namely position information. Since image features often depend on the information between adjacent pixels, adding position information to the one-dimensional image information after Patch Embedding can better improve training effects; Transformer Block contains Attention layer and MLP layer. The Attention layer performs a linear transformation on the image information with position information to obtain three feature vector groups Q: {q1, q2, …, qn}, K: {k1, k2, …, kn}, and V: {v1, v2, …, vn}. The attention weights are calculated through the multi-head self-attention mechanism and finally input into the MLP network to obtain the image feature information.

[0085] The data fusion layer is a convolutional layer with 4 input channels, 3 output channels, a kernel size of 1, and a stride of 1. This convolutional layer performs a preliminary fusion of the input overlap image and the mask image, concentrating the information of the overlapping images within the valid area, reducing the pressure on the subsequent encoder and decoder. In the patch embedding, each patch is 16 in size, so N is 196, and the dimensions of the patches obtained for each image are 196×768. This is accomplished by a convolutional layer with 3 input channels, 768 output channels, a kernel size of 16, and a stride of 16. In the decoder, patch embedding only reshapes the encoder output, changing the dimensions of the patches to 196×512.

[0086] The Position Embedding parameters use the sin-cos position encoding method. The dimension of the position encoding of a patch in the encoder is 1×768, and the dimension of the position encoding of a patch in the decoder is 1×512. The encoding formula is as follows:

[0087]

[0088]

[0089] Where pos is the position of the current patch; d model is the length of the position code, which is 768 in the encoder and 512 in the decoder; i is the i-th digit in the position code. When i is an even number, the first formula is used, and when i is an odd number, the second formula is used.

[0090] The Transformer Block structure refers to Figure 6 ; In the encoder with the VIT structure, the number of TransformerBlock layers is 6, of which mlp_radio in the MLP layer is set to 4; in the decoder with the VIT structure, the number of TransformerBlock layers is 4, of which mlp_radio in the MLP layer is set to 4.

[0091] The discriminator consists of a six-layer convolutional network and a four-layer fully connected network. The convolutional layer extracts image features, and the fully connected layer performs feature analysis and determines whether the image is a real image through a sigmoid activation function.

[0092] In step S14, the network is trained. The learning rate of the generator changes with the number of cycles, increasing from 0 generation by generation and decreasing after reaching 1e-4 generation by generation. The Adam optimizer is used, and the dropout is set to 0.3. The loss function is the weighted sum of the absolute error loss between the decoder generated image and the real image and the discrimination loss of the discriminator for distinguishing the generated image from the real image. The specific formula is as follows:

[0093]

[0094] Where I pred For the item map generated by the generator, D(I pred ) is the discriminator's judgment on the item image generated by the generator, λ is the weight of the absolute error loss, which is set to 10; the learning rate of the discriminator is set to 4e-4, and the Adam optimizer is used. The loss function consists of two parts: the cross entropy loss function of the discriminator output of the real image and the true value, and the cross entropy loss function of the discriminator output of the generator-generated image and the fake value. The specific formula is as follows:

[0095] L D =-[log D(I true )+log(1-D(I pred ))]

[0096] Where I true is the item graph in the dataset, D(I true ) is the judgment of the discriminator on the item graph; I pred For the item map generated by the generator, D(I pred ) is the discriminator's judgment on the item image generated by the generator; the early stopping method is used to train the GAN network. The performance of the current model is observed during the training process. After the 25th cycle, the separation effect is the best.

[0097] In step S15, the Mask R-CNN fine-tuned in step S12 is connected to the generator trained in step S14 to obtain the final separation model and applied to the test set images. The final network structure of this method is shown in Figure 7 .

[0098] See also Figure 8 , which intuitively demonstrates the technical effect of the present invention. It can be seen that for X-ray images of different overlapping objects, this method can separate the individual object images end-to-end. However, the existing image restoration technology cannot achieve the technical effect of the present invention. Figure 9 It can be seen that image restoration technology can only fill in the defects in the image according to the original image, that is, the image containing overlapping objects, and cannot achieve the effect of separating overlapping objects.

[0099] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for separating and extracting overlapping X-ray objects, characterized in that: At least the following steps are included: Step S1: Obtain a separation and extraction model based on deep learning training; Step S2: Obtain X-ray overlapping object images and input them into the separation and extraction model trained in step S1 to output separated object information; Wherein, the step S1 at least includes the following steps: Step S11: Acquire a dataset of X-ray overlapping object images, wherein the dataset includes an overlap map, an object map, and a mask map, which respectively represent the X-ray image of the overlapping object, the X-ray image of a single object, and the mask image of the single object and the overlapping portion; Step S12: Optimize and train the trained Mask R-CNN segmentation network using the dataset constructed in step S11. Step S13: Building a separation network for overlapping object X-ray images. The network is a GAN structure, which includes a generator based on a VIT structure and a discriminator based on a convolutional network. Step S14: using the data set constructed in step S11 to train the overlapping object X-ray image separation network constructed in step S13; Step S15: Connect the Mask R-CNN trained in step S12 with the generator trained in step S14 to obtain the final separation and extraction model; In step S2, the X-ray image containing overlapping objects is input into the Mask R-CNN network to obtain the corresponding mask image. The mask image and the X-ray image are then input into the separation and extraction network to finally obtain the X-ray image of the individual object. In step S12, the Mask R-CNN network is composed of Faster R-CNN and FCN, wherein Faster R-CNN is used to complete classification and bounding box prediction, and FCN is used to perform image segmentation in the screened ROI; Fine-tune the parameter model of the 91-category Mask R-CNN segmentation network trained on the coco dataset, changing the categories to be segmented to 3 categories: overlapping areas, non-overlapping areas, and background; In step S13, the VIT structure is used to replace the generator of the traditional CNN structure. The generator of the VIT structure includes three parts: a data fusion layer, an encoder with a VIT structure, and a decoder with a VIT structure. The VIT structure includes three parts: PatchEmbedding, Position Embedding, and a multi-layer Transformer Block. Patch Embedding is used to convert an image from two-dimensional information into a series of one-dimensional information; that is, an image with a dimension of H×W×C is divided into N pieces of size P. 2 The size of C's patch, N, is HW / P 2 , and reshape to N×(P 2 C) patches block; PositionEmbedding is used to give each patch block a unique code, namely position information; Transformer Block contains Attention layer and MLP layer. The Attention layer performs a linear transformation on the image information with position information to obtain three feature vector groups Q: {q1, q2, …, qn}, K: {k1, k2, …, kn}, and V: {v1, v2, …, vn}. The attention weights are calculated through the multi-head self-attention mechanism and finally input into the MLP network to obtain the feature information of the image.

2. The X-ray overlapping object separation and extraction method according to claim 1, characterized in that: In step S11, constructing a dataset of overlapping object X-ray images includes the following steps: Step S111: selecting images without overlapping objects from X-ray images of common objects taken by existing security inspection devices, that is, X-ray images of individual objects and background images; Step S112: Using Photoshop software, the cluttered background in the X-ray image of the individual object obtained in step S111 is removed and changed to white. At the same time, the sizes of the X-ray image of the individual object and the background image are adjusted to obtain an object image and a background image. Step S113: Divide the item images obtained in step S112 into two groups, and randomly select the background image, the image of item 1, and the image of item 2 to synthesize an overlapping image. The synthesis formula is as follows: Where, I out is the pixel value of the final overlapping image, I1 is the pixel value of item 1 image, I2 is the pixel value of item 2 image, I b is the background image pixel value; Step S114: Binarize the object image of the composite overlapping image in step S113, find the object outline, randomly select object 1 or object 2, set the value of the pixels within its outline to 255, and set the value of the pixels within both the outline of object 1 and the outline of object 2 to 128. The values ​​of the remaining pixels are set to 0 to obtain the final mask image.

3. The X-ray overlapping object separation and extraction method according to claim 2, characterized in that: In step S112, the quick selection tool in the PS software is used to segment the individual objects, separate them from the original background, add a white background, and adjust the image size.

4. The X-ray overlapping object separation and extraction method according to claim 2, characterized in that: In step S113 , the number of channels of the synthesized overlapping image is 3.

5. The method for separating and extracting X-ray overlapping objects according to claim 2, characterized in that: In step S114 , the number of channels of the generated mask image is 1.

6. The X-ray overlapping object separation and extraction method according to claim 1, characterized in that: The data fusion layer in the generator is a convolutional layer with 4 input channels, 3 output channels, a convolution kernel size of 1, and a stride of 1. This convolutional layer performs a preliminary fusion of the input overlap image and the mask image, concentrating the information of the overlapped image in the effective area, reducing the pressure on the subsequent encoder and decoder; The size of each patch in the Patch Embedding in the encoder is 16, so N is 196, and the dimension of the patches obtained for each image is 196×768. The specific process is completed by a convolutional layer with 3 input channels, 768 output channels, a convolution kernel size of 16, and a stride of 16. In the decoder, Patch Embedding only changes the dimension of the encoder output, and the dimension of the patches becomes 196×512.

7. The X-ray overlapping object separation and extraction method according to claim 1, characterized in that: The PositionEmbedding parameter uses the sin-cos position encoding method. The dimension of the position encoding of a patch in the encoder is 1×768, and the dimension of the position encoding of a patch in the decoder is 1×512. The encoding formula is as follows: Where pos is the position of the current patch; d model is the length of the position code, which is 768 in the encoder and 512 in the decoder; i is the i-th digit in the position code. When i is an even number, the first formula is used, and when i is an odd number, the second formula is used; In the encoder with VIT structure, the number of Transformer Block layers is 6, among which mlp_radio is set to 4 in the MLP layer; In the decoder with VIT structure, the number of Transformer Block layers is 4, among which mlp_radio is set to 4 in the MLP layer; The discriminator based on CNN structure consists of a six-layer convolutional network and a four-layer fully connected network. The convolutional layer extracts image features, and the fully connected layer performs feature analysis and uses the sigmoid activation function to determine whether the image is a real image.

8. The X-ray overlapping object separation and extraction method according to claim 1, characterized in that: In step S14, the learning rate of the generator changes with the number of cycles, increasing from 0 generation by generation and decreasing generation by generation after reaching 1e-4. The Adam optimizer is used, and the dropout is set to 0.

3. The loss function is the weighted sum of the absolute error loss between the decoder generated image and the real image and the discrimination loss of the discriminator for distinguishing the generated image from the real image. The specific formula is as follows: Where I pred For the item map generated by the generator, D(I pred ) is the judgment of the discriminator on the item image generated by the generator, λ is the weight of the absolute error loss, which is set to 100; In step S14, the learning rate of the discriminator is set to 4e-4, and the Adam optimizer is used. The loss function includes two parts: the cross entropy loss function of the discriminator output of the real image and the true value, and the cross entropy loss function of the discriminator output of the generator image and the false value. The specific formula is as follows: L D =-[logD(I true )+log(1-D(I pred ))] Where I true is the item graph in the dataset, D(I true ) is the judgment of the discriminator on the item graph; I pred For the item map generated by the generator, D(I pred ) is the judgment of the discriminator on the item graph generated by the generator; In step S14, the early stopping method is used to train the GAN network, and the performance of the current model is observed during the training process to obtain the optimal model.

Images