A method for segmenting pre-trained lung medical images based on reconstruction

Abstract

A kind of lung medical image segmentation method based on reconstruction pre-training, comprising: 1, construct unlabeled lung medical image pre-training dataset, labeled lung medical image training sample set and test sample set;2, construct pre-training network model, the grid division of each image of unlabeled lung medical image pre-training dataset is carried out and the order is disturbed, and pre-training network model is trained using it;3, construct segmentation-reconstruction network model, the weight of pre-training network model after training is used to initialize the parameters of segmentation-reconstruction network model;4, the image enhancement processing of labeled lung medical image dataset is carried out;segmentation-reconstruction network model is optimized and trained;5, segmentation-reconstruction network model is tested.The present application improves the robustness and generalization of model by unsupervised jigsaw reconstruction pre-training learning feature representation, combines the idea of transfer learning, reduces the model training time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, in particular to a lung medical image segmentation method based on reconstruction pre-training. BACKGROUND

[0002] In order to reduce the workload of radiologists, computer-aided diagnosis is gradually applied to the review and analysis process of medical images, and the segmentation of the region of interest based on deep learning method is one of the research hotspots. However, due to the poor robustness and generalization performance of the segmentation method based on deep learning, it is difficult to be widely used in real-world clinical diagnosis. At the same time, the training data of medical images are relatively scarce due to the involvement of patient privacy and the need for professional knowledge for label making, and in this case, the above problems will become particularly serious. Therefore, it is of great significance to ensure the robustness and generalization of the medical image analysis system under limited training data.

[0003] Transfer learning is a technique for exploring the potential of limited labeled data, which generally follows the pre-training-fine-tuning framework, that is, pre-training on a large supervised or unsupervised but irrelevant to the target domain dataset, transferring the learned domain-independent representation to the specific task model downstream, and fine-tuning the model with a limited number of target domain dataset to improve the generalization and robustness of the network model. Cheplyginab first pre-trains on a large supervised natural image dataset such as ImageNet for classification, and then transfers the learned features to medical images for learning and analysis. In addition, the pre-training of unsupervised dataset focuses on how to learn meaningful representation without expert annotation. Zhang et al. convert color images to grayscale images and pre-train by coloring grayscale images to learn visual representation of convolutional neural network. Gidaris sets the pre-training task to obtain the angle of randomly rotated pictures to learn domain-independent features. However, unlike natural images, medical images are usually grayscale images, and different medical image data have similar spatial structures. Therefore, the above methods transfer the knowledge learned in natural images to medical images, and the robustness and generalization cannot meet the real demand. In addition, in the fine-tuning process of some models, such as the generative model commonly used in medical image segmentation, the diversity of the generator results is insufficient, causing feature collapse, leading to difficulty in training during the fine-tuning process, and limiting the segmentation accuracy of medical images. SUMMARY

[0004] The present application provides a lung medical image segmentation method based on reconstruction pre-training to solve the technical problem of long model training time caused by the inability of robustness and generalization to meet real demand in the prior art.

[0005] To achieve the above purpose, the technical scheme of the present application is as follows:

[0006] The application provides a lung medical image segmentation method based on reconstruction pre-training, comprising the following steps:

[0007] Step S1, constructing a no-label lung medical image pre-training data set, a labeled lung medical image training sample set and a test sample set;

[0008] Step S2, constructing a pre-training network model, performing grid division on each image of the no-label lung medical image pre-training data set and shuffling the order, and training the pre-training network model by using the no-label lung medical image pre-training data set;

[0009] Step S3, constructing a segmentation-reconstruction network model, and initializing parameters of the segmentation-reconstruction network model by using weights of the trained pre-training network model;

[0010] Step S4, performing image enhancement processing on the labeled lung medical image data set, and optimizing and training the segmentation-reconstruction network model after parameter initialization by using a stochastic gradient descent method;

[0011] Step S5, testing the segmentation-reconstruction network model after optimization and training by using the test sample set.

[0012] Preferably, the step S1 specifically comprises the following steps:

[0013] Step S11, collecting multiple no-label lung medical images and multiple labeled lung medical images;

[0014] Step S12, taking the collected multiple no-label lung medical images as the no-label lung medical image pre-training data set;

[0015] That is, D r ={x1, x2,... x i ,..., x M}, wherein D r is the no-label lung medical image pre-training data set, is D r in the Euclidean space manifold; x i represents the i-th image in D r , and M represents the number of images in the no-label lung medical image pre-training data set;

[0016] A part of the multiple labeled lung medical images is selected to construct the labeled lung medical image training sample set, and another part of the labeled lung medical images is selected to construct the test sample set.

[0017] Preferably, the step S2 specifically comprises the following steps:

[0018] Step S21, constructing a pre-training network model, the pre-training network model comprising an inference network modeling a posterior distribution q β (z|x) and a generative network for fitting a p δ (x|z) distribution, the inference network being a pre-training encoder f β (·), and the generative network being a pre-training decoder f δ (·); establishing a grid arrangement label prediction branch;

[0019] Step S22, inputting an unlabeled lung medical image dataset into the pre-training encoder f β (·) of the pre-training network model, and first adjusting each image in the unlabeled lung medical image dataset to a specified size, and then segmenting into regular n×n grid pictures;

[0020] That is x M represents a set of grid pictures, each grid picture being assigned a prior distribution p(z k ), wherein Z is a latent variable output by the pre-training encoder f β (·);

[0021] Step S23, generating a plurality of random arrangement labels S i , and randomly arranging the n×n grid pictures through the arrangement label S i ; establishing a network ranking optimization loss function, inputting the randomly arranged n 2 ×n grid pictures into the pre-training network model, and combining the network ranking optimization loss function and the grid arrangement label prediction branch to optimize and learn the pre-training network model, learning the latent encoding of the whole picture so that the arrangement label S i of each grid can be correctly predicted, to obtain a learned pre-training network model;

[0022] Step S24, establishing a pre-training total loss function, and optimizing and training the pre-training network model through the pre-training total loss function to obtain an optimized and trained pre-training network model;

[0023] Preferably, the network ranking optimization loss function in step S23 is specifically:

[0024]

[0025] wherein represents a cumulative grid feature set; represents the kth grid feature in the grid feature set; K(S i ) is a sign function, taking a value of 1 when the predicted grid arrangement label is the same as the real arrangement label S i , and otherwise taking a value of 0. Is the grid arrangement tag The predicted probability of ; N is the number of samples.

[0026] Preferably, step S24 specifically includes the following steps:

[0027] Step S241: Establish distance loss function The posterior distribution q is made by the distance loss function β (z k |x k ) approximates the prior distribution p(z k ), the distance loss function can be expressed as:

[0028]

[0029] where Θ is the d-dimensional unit sphere A collection of is a hyperplane, θ is the element sampled from Θ, p(z k ) is assigned a Gaussian distribution is the marginal distribution of random transformation, W p (·) is the p-Wasserstein distance, also known as the heap distance, which is expressed as:

[0030]

[0031] where ∏(p X , p Y ) represents the set of all joint distributions, and d is the metric function, such as the Euclidean distance d(x, y) = ||xy|| 2 ; p is the exponent of the metric function; W p Refers to the distance between the piles of soil; px represents the probability of X; p Y represents the probability of Y; γ represents the joint probability distribution; Indicates expectation; X and Y represent two different variables respectively;

[0032] Step S242: pre-trained encoder f β The output features of (·) are fed into the pre-trained decoder f δ (·) Perform image reconstruction, set the image reconstruction loss function, and construct the pre-training total loss function based on the network ranking optimization loss function, distance loss function, and image reconstruction loss function;

[0033] The image reconstruction loss function is specifically:

[0034]

[0035] The pre-training total loss function is specifically:

[0036]

[0037] wherein, refers to a pre-trained decoder f δ (·) the i-th generated reconstructed image;

[0038] Step S243, using a random gradient descent optimizer and combining a pre-training total loss function to optimize and train the pre-training network model, to obtain an optimized pre-training network model.

[0039] Preferably, the step S3 specifically comprises the following steps:

[0040] Step S31, building a segmentation-reconstruction network model, the segmentation-reconstruction network model comprising a segmentation branch network and a reconstruction branch network; the segmentation branch network comprising a segmentation encoder f θ (·) and a segmentation decoder f r (·); the reconstruction branch network comprising a reconstruction encoder f s (·) and a reconstruction decoder f θ (·).

[0041] Step S32, using the segmentation decoder f Pre (·) to output a lung prediction segmentation mask M r of the current image.

[0042] Step S33, initializing parameters of the reconstruction branch network using the trained pre-training network model, and sharing weights of the segmentation encoder f r (·) and the reconstruction encoder f β (·), to obtain a segmentation-reconstruction network model after parameter initialization.

[0043] Preferably, the reconstruction encoder f r (·) in the step S31 is consistent with the framework of the pre-training encoder f β (·), i.e., f s (·) = f δ (·), and the reconstruction decoder f s (·) is consistent with the framework of the pre-training decoder f δ (·), i.e., f GT (·) = f Pre (·).

[0044] Preferably, the step S4 specifically comprises the following steps:

[0045] Step S41, performing image enhancement processing on the labeled lung medical image dataset, and inputting the same into the second segmentation-reconstruction network model;

[0046] Step S42, a segmentation-reconstruction network total loss function is established, and the segmentation-reconstruction network total loss function is specifically:

[0047]

[0048] Wherein, The segmentation branch loss function is used for calculating the similarity between the segmentation true value M GT and the lung prediction segmentation mask M Pre , and can be expressed as:

[0049]

[0050] Wherein, |·| represents the sum of all element values, and epsilon is a preset constant, which is set to 1 to prevent the denominator from being 0;

[0051] Step S43, input the image enhancement processed labeled lung medical image dataset, and use the segmentation-reconstruction network total loss function to optimize and train the parameter initialized segmentation-reconstruction network model to obtain the optimized segmentation-reconstruction network model.

[0052] Preferably, the image enhancement processing in step S41 specifically includes noise processing, blur processing, contrast transformation, brightness change and saturation change.

[0053] Advantages of the present application:

[0054] 1、The present application pre-trains by unsupervised puzzle reconstruction, reconstructs the input lung medical image after grid segmentation and disorder, captures global and local information to learn important semantic feature representation of the image, combines the idea of transfer learning, improves the robustness and generalization of the segmentation branch network, and enhances the segmentation performance of the unseen image or damaged image.

[0055] 2、The present application adds a reconstruction branch network in the segmentation-reconstruction network model, and initializes the parameters of the reconstruction branch network by using the pre-trained network model, avoids the feature collapse of the reconstruction branch network in the training process, and improves the segmentation accuracy of the lung medical image. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a flowchart of the present application;

[0057] Figure 2 is a pre-trained network model structure diagram in the present application;

[0058] Figure 3 is a schematic diagram of the segmentation-reconstruction network model in the present application. DETAILED DESCRIPTION

[0059] The application will be described in further detail below with reference to the drawings and specific embodiments.

[0060] The terms "first", "second", and the like in the present application are used only for the purpose of description, and cannot be understood as indicating or implying their relative importance or implicitly indicating the number or order of the indicated technical features. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly specified.

[0061] Referring to Figure 1 , the embodiment of the present application provides a lung medical image segmentation method based on pre-training reconstruction, comprising the following steps:

[0062] Step S1, constructing a no-label lung medical image pre-training data set, a labeled lung medical image training sample set and a test sample set;

[0063] Step S2, constructing a pre-training network model, performing grid division on each image of the no-label lung medical image pre-training data set and shuffling the order, and training the pre-training network model using the same;

[0064] Step S3, constructing a segmentation-reconstruction network model, and initializing parameters of the segmentation-reconstruction network model using weights of the trained pre-training network model;

[0065] Step S4, performing image enhancement processing on the labeled lung medical image data set; and optimizing and training the segmentation-reconstruction network model after parameter initialization using a stochastic gradient descent method;

[0066] Step S5, testing the segmentation-reconstruction network model after optimization and training using the test sample set.

[0067] In the embodiment, the step S1 specifically comprises the following steps:

[0068] Step S11, collecting a plurality of no-label lung medical images and a plurality of labeled lung medical images;

[0069] Step S12, taking the collected plurality of no-label lung medical images as a no-label lung medical image pre-training data set;

[0070] That is, D r ={x1, x2,... x i ,..., x M}, wherein D r is a no-label lung medical image pre-training data set, is D rIn the manifold of Euclidean space; x i D represents r The i-th image in the middle, M represents the number of images in the unlabeled lung medical image pre-training dataset;

[0071] In a plurality of labeled lung medical images, a part of the images is selected to construct a labeled lung medical image training sample set, and another part of the labeled lung medical images is used to construct a test sample set.

[0072] This embodiment relates to an unlabeled lung medical image pre-training dataset and three labeled lung medical image datasets. The unlabeled lung medical image pre-training dataset is given by Kermany in 2018 on Cell, which includes 5232 lung X-ray images, of which 1349 are normal cases and 3883 are pneumonia cases. In the embodiment, only 1349 normal samples are used to construct a field-related unlabeled lung medical image pre-training dataset.

[0073] The three labeled lung medical image datasets are Montgomery County Chest Radiograph Dataset (MC), Japan Society of Radiological Technology Database (JSRT), and Shenzhen Chest Radiograph Dataset (SH), respectively. The MC dataset contains 138 lung X-ray images, each with a size of 4020x4892. JSRT is a public dataset that includes 247 lung X-ray grayscale images, each containing 2048x2048 pixels. The SH dataset includes 662 lung X-ray images of tuberculosis symptoms, each with different sizes but approximately containing 3Kx3K pixels. The MC dataset and the JSRT dataset both contain manually segmented lung mask labels, while the SH dataset is annotated by the National Technical University of Ukraine.

[0074] In this embodiment, the step S2 specifically comprises the following steps:

[0075] Step S21, as shown in Figure 2 , a pre-training network model is constructed, which includes an inference network for modeling the posterior distribution q β (z|x) and a generation network for fitting the p δ (x|z) distribution, the inference network is a pre-training encoder f β (·), and the generation network is a pre-training decoder f δ (·); a grid arrangement label prediction branch is established;

[0076] The pre-training encoder f β (·) is composed of ResNet50, and the pre-training decoder f δ(·) includes a first module, a second module, a third module and a fourth module connected in sequence, the first module and the second module are structurally identical, both of which include an up-sampling layer, a convolutional layer, a BatchNorm layer and a ReLU activation function layer connected in sequence.

[0077] The convolutional kernel of the convolutional layer of the first module is 3x3, the stride is 1, and the number of channels is 512; the convolutional kernel of the convolutional layer of the second module is 7x7, the stride is 1, and the number of channels is 256. The up-sampling layer includes a bilinear up-sampling operation that expands the input feature size by two times, a convolutional layer with a convolutional kernel of 1x1 and a stride of 1, a BatchNorm layer and a ReLU activation function layer. The up-sampling layer of the first module is referred to as the first up-sampling layer, and the number of channels of the convolutional layer inside it is 512. The up-sampling layer of the second module is referred to as the second up-sampling layer, and the number of channels of the convolutional layer inside it is 256. The third module includes three up-sampling layers connected in sequence, a convolutional layer with a convolutional kernel of 3x3, a stride of 1, and a number of channels of 32, a BatchNorm layer, and a ReLU layer. The number of channels of the convolutional layers inside the three up-sampling layers is 128, 64 and 32 respectively. The fourth module includes a convolutional layer with a convolutional kernel of 3x3, a stride of 1, a number of channels of 3, and a Tanh activation function layer connected in sequence.

[0078] In addition, the pre-trained decoder f δ (·) is used to generate a lung medical image in a normal order.

[0079] The grid arrangement label prediction branch includes an average pooling layer, a tensor reconstruction layer, a linear mapping layer, a tensor reconstruction layer and a linear prediction layer connected in sequence; the grid arrangement label prediction branch is used to predict the sequence order of image segmentation disorder.

[0080] For example: the input image is segmented and disordered in the foregoing, for example, the picture is segmented into 9 parts, and the arrangement before disorder is {1, 2, 3, 4, 5, 6, 7, 8, 9}, and after disorder, it becomes {5, 2, 1, 4, 7, 3, 6, 8, 9}. However, the network does not know the order of random disorder, so the grid arrangement label prediction branch is used to predict the order after disorder, and the predicted result is compared with {5, 2, 1, 4, 7, 3, 6, 8, 9} in terms of distance.

[0081] Since natural images and medical images have different spatial structures, the knowledge learned in natural images has poor transfer effect in medical image downstream tasks, so the unlabeled lung medical image pre-training dataset is used as the input of the pre-training network model.

[0082] Step S22, inputting the unlabeled lung medical image dataset into the pre-training encoder f β(·), and each image in the unlabeled lung medical image dataset is first resized to 300×300, and then split into regular n×n grid images;

[0083] Right now x M Represents a grid image set. In this embodiment, n is selected as 3. Each grid image is assigned a prior distribution p(z k ), where Z is the pre-trained encoder f β (·) The output latent variable has a dimension of 128; the original whole image is transformed from a prior distribution p(z k ) approximation is transformed into 9 prior distributions p(z k ) to approximate and solve the single prior distribution p(z k )The problem of insufficient approach capability.

[0084] Step S23: Randomly shuffle the numbers 0 to 8 to generate 100 permutation labels S i and arrange the labels S by i Randomly arrange the 9 grid images;

[0085] Establish a network sorting optimization loss function and randomly arrange n 2 The grid image is input to the pre-trained encoder f of the pre-trained network model β (·) and combines the network sorting optimization loss function and the grid arrangement label prediction branch to optimize the pre-trained network model and learn the potential encoding of the entire image, so that the arrangement label S of each grid can be correctly predicted. i , get the pre-trained network model after learning;

[0086] In this embodiment, the network sorting optimization loss function in step S23 is Specifically:

[0087]

[0088] in represents the accumulated grid feature set; Represents the kth grid feature in the grid feature set; K(S i ) is the symbol function, when the predicted grid arrangement label and the real permutation label S i If they are the same, the value is 1, otherwise it is 0; Is the grid arrangement tag The predicted probability of ; N is the number of samples.

[0089] Step S24, a pre-training total loss function is established, and the pre-training network model is optimized and trained through the pre-training loss function to obtain an optimized and trained pre-training network model;

[0090] In the embodiment, the step S24 specifically comprises the following steps:

[0091] Step S241, a distance loss function is established The posterior distribution q β (z k |x k ) is forced to approximate the prior distribution p(z k ), and the distance loss function can be expressed as:

[0092]

[0093] where Θ is a set of d-dimensional unit spheres , is a hyperplane, θ is an element sampled from Θ, and p(z k ) is assigned as a Gaussian distribution is an edge distribution of a random transformation, and W p (·) is a p-Wasserstein distance, also known as a heap distance, and the heap distance is expressed as:

[0094]

[0095] where ∏(p X , p Y ) represents a set of all joint distributions, d is a metric function, such as the Euclidean distance d(x, y) = ||x-y|| 2 ; p is the exponent of the metric function; W p refers to the heap distance; p x represents the probability of X; p Y represents the probability of Y; and γ represents the joint probability distribution. represents the expectation; X and Y represent two different variables, respectively;

[0096] Step S242, the output feature of the pre-training encoder f β (·) is sent to the pre-training decoder f δ (·) for image reconstruction, an image reconstruction loss function is set, and a pre-training total loss function is constructed according to the network ranking optimization loss function, the distance loss function and the image reconstruction loss function;

[0097] The pre-training total loss function is specifically:

[0098]

[0099] Step S243, using a random gradient descent optimizer and combining a pre-training total loss function to optimize and train the learned pre-training network model, to obtain an optimized pre-training network model.

[0100] In this embodiment, the image reconstruction loss function in step S242 is specifically:

[0101]

[0102] wherein, refers to a pre-training decoder f δ (·) generated by the reconstruction of the i-th image;

[0103] In this embodiment, the step S3 specifically comprises the following steps:

[0104] Step S31, as shown in Figure 3 , a segmentation-reconstruction network model is built, which includes a segmentation branch network and a reconstruction branch network; the segmentation branch network includes a segmentation encoder f (·) and a segmentation decoder f θ (·); the reconstruction branch network includes a reconstruction encoder f r (·) and a reconstruction decoder f s (·);

[0105] The segmentation encoder f (·) of the segmentation branch network is the same as the reconstruction encoder f r (·), both of which are composed of ResNet50, and the output is 4×4×2048; the segmentation decoder f θ (·) is composed of a PSPNet network upsampling framework, and the output is 300×300×1.

[0106] Step S32, using the segmentation decoder f θ (·) to output the lung prediction segmentation mask M Pre of the current image;

[0107] The segmentation decoder based on the PSPNet network is composed of a pyramid pooling module, an up-sampling module and an output module. The pyramid pooling module first up-samples the input feature to 12x12x2048, respectively passes through four average pooling and convolution operations with different kernel sizes, generates four feature maps with sizes of 1x1, 2x2, 3x3 and 6x6, and a channel number of one-fourth of the input feature, i.e. 1024 dimensions. The above four feature maps are up-sampled by bilinear interpolation to make the size consistent with the input feature, and then a feature fusion operation is performed, i.e. the above feature maps are sequentially spliced with the input feature in the depth direction to obtain a fused feature map. The up-sampling module expands the size of the input feature map through three up-sampling operations, so that it is the same size as the input medical image. The up-sampling operation sequentially includes a convolution layer with a convolution kernel of 3x3 and a stride of 1, a BatchNorm layer, a Dropout layer and a ReLU layer. Among them, the channel numbers of the convolution kernels in the three up-sampling operations are 256, 64 and 64 respectively. The output module changes the output feature channel number of the last module from 64 to 1 through a convolution layer with a convolution kernel of 1x1, a stride of 1 and a channel number of 3, and then uses a sigmoid function to convert the output pixel value to 0 to 1. Finally, the pixels with a pixel value greater than 0.5 are taken as the target pixels of image segmentation, and a lung prediction segmentation mask M is output Pre .

[0108] Step S33, initialize the parameters of the reconstruction branch network using the trained pre-trained network model, and share the weights of the segmentation encoder and the reconstruction encoder f r (·) to obtain a segmentation-reconstruction network model after parameter initialization.

[0109] In this embodiment, the reconstruction encoder f r (·) in step S31 is consistent with the framework of the pre-trained encoder f β (·), i.e. f r (·)=f β (·), and the reconstruction decoder f s (·) is consistent with the framework of the pre-trained decoder f δ (·), i.e. f s (·)=f δ (·).

[0110] In this embodiment, the step S4 specifically comprises the following steps:

[0111] Step S41, performing image enhancement processing on the labeled lung medical image dataset, and inputting it into the segmentation-reconstruction network model;

[0112] Step S42, establishing a segmentation-reconstruction network total loss function, which is specifically:

[0113]

[0114] wherein, the segmentation branch loss function is used for calculating the similarity between the segmentation true value M GT and the lung prediction segmentation mask M Pre The similarity between the segmentation true value M

[0115]

[0116] wherein |·| represents the sum of all element values, and ε is a preset constant, which is set to 1 to prevent the denominator from being 0;

[0117] Step S43, input the labeled lung medical image dataset after image enhancement processing, and use the segmentation-reconstruction network total loss function to optimize and train the segmentation-reconstruction network model after parameter initialization, to obtain an optimized segmentation-reconstruction network model.

[0118] In the embodiment, the image enhancement processing in step S41 specifically includes noise processing, blur processing, contrast transformation, brightness change, and saturation change.

[0119] In step S5, the present embodiment randomly takes two of the aforementioned Montgomery County Chest Radiograph Dataset (MC), Japan Society of Radiological Technology Database (JSRT), and Shenzhen Chest Radiograph Dataset (SH) for training and testing, respectively, and the experimental results are shown in Table 1 (using English initials to replace the full name, such as M for MC):

[0120] Table 1

[0121]

[0122] From the above experimental results, it can be seen that the present embodiment, by unsupervised puzzle reconstruction pre-training, reconstructs the input lung medical image after grid segmentation and randomization, captures global and local information to learn important semantic feature representation of the image, combines the idea of transfer learning, improves the robustness and generalization of the downstream segmentation network model, and enhances the segmentation performance of the unseen image or damaged image. At the same time, the image reconstruction branch is added in the downstream segmentation network model, and the pre-trained network model is used to initialize the parameters of the downstream related network framework, which avoids the feature collapse of the segmentation network model in the training process, and improves the segmentation accuracy of the lung medical image.

[0123] The above merely describes specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Furthermore, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the realization of the ordinary skilled person in the art, when the combination of the technical solutions appears contradictory or unachievable, it should be considered that the combination of the technical solutions does not exist, and is not within the protection scope required by the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A lung medical image segmentation method based on reconstruction pre-training, characterized by: The steps include: Step S1, constructing an unlabeled lung medical image pre-training dataset, a labeled lung medical image training sample set, and a test sample set; Step S2: constructing a pre-trained network model, dividing each image in the unlabeled lung medical image pre-training dataset into a grid and shuffling the order, and using the grid to train the pre-trained network model; Step S3: construct a segmentation-reconstruction network model, and use the weights of the trained pre-trained network model to initialize the parameters of the segmentation-reconstruction network model; Step S4: performing image enhancement processing on the labeled lung medical image dataset; The stochastic gradient descent method is used to optimize the segmentation-reconstruction network model after parameter initialization; Step S5: testing the optimized trained segmentation-reconstruction network model using a test sample set; The step S2 specifically includes the following steps: Step S21: construct a pre-trained network model, which includes an inference network for modeling the posterior distribution qβ(z|x) and a network for fitting p δ (x|z) distribution generation network, the inference network is the pre-trained encoder f β (·), generating a network that is a pre-trained decoder f δ (·); Establish a grid arrangement label prediction branch; Step S22: input the unlabeled lung medical image dataset into the pre-trained encoder f of the pre-trained network model. β (·), and each image in the unlabeled lung medical image dataset is first resized to a specified size and then split into a regular n×n grid image; Right now x M Represents a grid image collection, each grid image is assigned a prior distribution p(z k ), where Z is the pre-trained encoder f β (·) Output latent variables; Step S23: Generate multiple randomly arranged labels S i and arrange the labels S by i Randomly arrange n×n grid images; establish a network sorting optimization loss function, and randomly arrange the n 2 The grid image is input into the pre-trained network model, and the pre-trained network model is optimized and learned by combining the network sorting optimization loss function and the grid arrangement label prediction branch to learn the potential encoding of the entire image, so that the arrangement label S of each grid can be correctly predicted. i , get the pre-trained network model after learning; Step S24: Establish a pre-training total loss function, and optimize the pre-training network model using the pre-training total loss function to obtain an optimized pre-training network model; The step S3 specifically includes the following steps: Step S31: Build a segmentation-reconstruction network model, which includes a segmentation branch network and a reconstruction branch network; the segmentation branch network includes a segmentation encoder and the segmentation decoder f θ (·); The reconstruction branch network includes reconstructing the encoder f r (·) and the reconstruction decoder f s (·); Step S32: Use the segmentation branch network to output the lung prediction segmentation mask M of the current image. Pre ; Step S33: Initialize the parameters of the reconstructed branch network using the trained pre-trained network model, and at the same time, segment the encoder. and reconstruction encoder f r (·) Share weights to obtain the segmentation-reconstruction network model after parameter initialization.

2. The lung medical image segmentation method according to claim 1, characterized in that: The step S1 specifically includes the following steps: Step S11, collecting multiple unlabeled lung medical images and multiple labeled lung medical images; Step S12: using the collected multiple unlabeled lung medical images as an unlabeled lung medical image pre-training dataset; D r ={x1, x2, ...x i ,...,x M }, Among them D r It is an unlabeled lung medical image pre-training dataset. D r In Euclidean space, the manifold x i Indicates D r The i-th image in , M represents the number of images in the unlabeled lung medical image pre-training dataset; A portion of the multiple labeled lung medical images is selected to construct a labeled lung medical image training sample set, and another portion of the labeled lung medical images is selected to construct a test sample set.

3. The lung medical image segmentation method according to claim 2, characterized in that: The network sorting optimization loss function in step S23 is specifically: in, Optimize loss function for network ranking; represents the accumulated grid feature set; Represents the kth grid feature in the grid feature set; K(S i ) is the symbol function, when the predicted grid arrangement label and the real permutation label S i If they are the same, the value is 1, otherwise it is 0; Is the grid arrangement tag The predicted probability of ; N is the number of samples.

4. The lung medical image segmentation method according to claim 3, characterized in that: The step S24 specifically includes the following steps: Step S241: Establish a distance loss function, and use the distance loss function to make the posterior distribution q β (z k |x k ) approximates the prior distribution p(z k ), the distance loss function can be expressed as: in, represents the distance loss function, Θ is the d-dimensional unit ball A collection of is a hyperplane, θ is the element sampled from Θ, p(z k ) is assigned a Gaussian distribution is the marginal distribution of random transformation, W p (·) is the p-Wasserstein distance, also known as the heap distance, which is expressed as: Where Π(p X , p Y ) represents the set of all joint distributions, and d is the metric function, such as the Euclidean distance d(x, y) = ||xy|| 2 ; p is the exponent of the metric function; W p Refers to the distance between piles of soil; X represents the probability of X; p Y represents the probability of Y; γ represents the joint probability distribution; Indicates expectation; X and Y represent two different variables respectively; Step S242: pre-trained encoder f β The output features of (·) are fed into the pre-trained decoder f δ (·) Perform image reconstruction, set the image reconstruction loss function, and construct the pre-training total loss function based on the network ranking optimization loss function, distance loss function, and image reconstruction loss function; The pre-training total loss function is specifically: in, is the total loss function of pre-training; is the image reconstruction loss function; Step S243: Optimize and train the pre-trained network model using a stochastic gradient descent optimizer in combination with a pre-training total loss function to obtain an optimized pre-trained network model.

5. The lung medical image segmentation method according to claim 4, characterized in that: The image reconstruction loss function in step S242 is specifically: in, refers to the pre-trained decoder f δ (·) The reconstructed i-th image generated.

6. The lung medical image segmentation method according to claim 5, characterized in that: The reconstruction encoder f in step S31 r (·) with the pre-trained encoder f β (·) is consistent with the framework, that is, f r (·)=f β (·), the reconstruction decoder f s (·) with the pre-trained decoder f δ (·) is consistent with the framework, that is, f s (·)=f δ (·).

7. The lung medical image segmentation method according to claim 6, characterized in that: The step S4 specifically includes the following steps: Step S41: performing image enhancement processing on the labeled lung medical image dataset and inputting it into the segmentation-reconstruction network model; Step S42: Establish a segmentation-reconstruction network total loss function. The segmentation-reconstruction network total loss function is specifically: in, is the total loss function of the segmentation-reconstruction network; Segmentation branch loss function, used to calculate the segmentation truth value M GT and the lung predicted segmentation mask M Pre The similarity between them, the segmentation branch loss function can be expressed as: Where |·| represents the sum of all element values, and ε is a preset constant set to 1 to prevent the denominator from being 0; Step S43: input the labeled lung medical image dataset after image enhancement processing, and use the segmentation-reconstruction network total loss function to optimize the segmentation-reconstruction network model after parameter initialization to obtain the optimized segmentation-reconstruction network model.

8. The lung medical image segmentation method according to claim 7, characterized in that: The image enhancement processing in step S41 specifically includes noise processing, blur processing, contrast transformation, brightness change and saturation change.

Images